Why BitTorrent Causes Latency and How To Fix It 315
Sivar recommends an article by George Ou examining why BitTorrent affects performance so much more than other types of file transfer and a recommendation on how to fix it. The suggestion is to modify P2P clients so that, at least on upload, they space their traffic evenly in time so that other applications have a chance to fit into the interstices. "[Any] VoIP [user] or online gamer who has a roommate or a family member who uses BitTorrent (or any P2P application) knows what a nightmare it is when BitTorrent is in use. The ping (round trip latency) goes through the roof and it stays there making VoIP packets drop out and game play impossible."
Re:Wait, wait wait! (Score:5, Interesting)
I don't mind traffic shaping [slashdot.org] at all, anywhere. QoS is a good thing, even when the ISPs do it. What I mind a whole awful lot is traffic blocking, ala Comcast.
Traffic shaping works but fair-queue works better. (Score:5, Interesting)
Traffic shaping and QOS are not usually able to make that guarantee. A straight priority queue with bandwidth guarantees can, as long as you are able to actually classify the torrent traffic differently from your other traffic.
Part of the problem is that it is often not possible to distinguish between the batch and the interactive traffic with Shaping/QOS. Not only is QOS almost universally set wrong, but the simple fact is that one can mix interactive and batch traffic over the SAME ports (http, ssh, dynamically allocated ports)and that can make it virtually impossible to use traffic shaping or QOS to keep the mess away from your interactive traffic.
The best general solution is to use a straight priority mechanic with minimum bandwidth settings to separate as much of the bulk traffic out as you can, and then run fair-queueing at each priority level to take care of any that leaks through. This will do a very good job cleaning up the traffic. DragonFly has a fair-queue implementation for PF that does this. There is also at least one fair-queue implementation for PF in the wild.
Fair-queueing essentially classifies connections (the one in DFly uses PF's keep-state to classify connections), generates a hash and indexes a large array of mini-queues. One packet is then pulled off the head of each mini-queue. One enhancement I would like to make to the DFly implementation which I haven't done yet is to use the keep-state to actually determine which connections are batch and which are interactive, and have a parameter that allows the queue to give additional priority to the interactive connections by occasionally skipping the hoppers related to the batch connections. A quick and dirty way to do that is to simply check the queue length for each mini-queue.
In anycase, its a problem for which solutions are available. Regardless of what you use it has become apparent in the last few years that the only way one can classify the traffic well enough to properly queue it is by building keep-state knowledge on a connection by connection basis.
-Matt
Re:QoS? (Score:3, Interesting)
When I'm not using http to download something then nntp can download at full speed. When I do something on http it will get the full bandwidth. It's not instant though so it takes a few seconds to kick in. I suspect it's dropping ACKs for the nntp traffic or something like that so that the nntp server stops sending so much data.
You could do the same thing with bittorrent as long as you know the ports.
Re:Use randomized time rather than even spacing (Score:3, Interesting)
There are better tools for measuring link properties, for example there is "pchar" - http://www.kitchenlab.org/www/bmah/Software/pchar/ [kitchenlab.org]
Looking at the URL you gave, there is nearly zero description about the software or how it works or how to use it.
In addition, i went ahead and downloaded the source hoping there might be some documentation giving a clue about this, and then i noticed:
Like i said, ive been out of network engineering for a few years, but i still have to question this method as 'better'
Re:QoS, but only on the Telco Side (Score:5, Interesting)
Hell, Azureus has a plugin to test ping an IP address/website, and if it takes longer than a set time, it slows down your uploads. uTorrent has a feature like that, as well.
Re:QoS? (Score:1, Interesting)
I hope you don't pay too much for that service. I have Road Runner too but I get 5Mbps down and I've been able to get up to 360kbps up. I don't know what the advertised upload bandwidth is off hand. I get that for $45/mo.
Re:Traffic shaping works but fair-queue works bett (Score:4, Interesting)
When I went from a T1 to a DSL line to save some money I immediately noticed the missing cisco. That little 2620 was so nice. PF couldn't hold a candle to what the 2620's fair-queue could do so I sat down and wrote a fair-queue implementation for PF (for DragonFly). It still isn't as good as what Cisco has, but it gets a lot closer then the other PF queuing mechanisms get.
I think the bit I'm missing is the batch classification. My fair-queue can still get overwhelmed by dozens of batch TCP connections if I happen to not be able to classify their traffic (and they wind up on the standard queue instead of the bulk queue). The set-up is a priority queue with minimum bandwidth guarantees plus a fair-queue at each priority level.
I keep hoping someone will take up the flag and finish it.
-Matt
TCP Capture effect (Score:3, Interesting)
It works like this: if the upstream bandwidth is saturated, TCP ACK packets get delayed and the sender slows transmission so the downstream bandwidth does not get fully utilised.
There is no solution other than throttling the upstream senders (AFAIK good P2P software has settings). Note larger send buffers in broadband modems actually exacerbate the problem by taking longer to flush. Best to keep them empty, and th only way is throttling.
A Better Solution (Score:2, Interesting)
-
Set your total bandwidth minus the guaranteed bandwidth you want to allocate to priority traffic masked/identified either by port/protocol/src/dest or by a deep packet (perl based) inspection.
-
If any app OR host OR connection OR port starts encroaching on the latency of other others, it gets chucked into memory jail for a fixed number of escalating milliseconds.
-
This has a two fold benefit for latency and bandwidth contention issues. One, by chucking the hogs in memory jail, queue space is cleared up to allow priority traffic through on a more consistent basis.
-
Imagine you've got a city bus that goes to sports stadium of your chosing. When it's not a
game day, there are no problems boarding the bus. You get 3 or 4 people on their way home from or to work. On a game day though, you get those same 3 or 4 people plus an additional 20 drunk people who don't know how to board a bus. What usually happens is the 3 or 4 regulars either don't get a seat or have to wait until the next bus. Not good or fair.
-
The way linux bandwidth arbitrator solves the problem is to tell the bus driver to watch for drunk idiots. If she sees more than three cubs fans in a row, she shouts out to them to step back, close their eyes and count to 10. While they're doing this (they're cubs fans, you know they would), she waves the regulars to the front of the line.
-
So, it's good for latency and jitter. It's also good for bandwidth. All those bad packets you threw into memory jail? Well, a good portion of them are probably TCP which means there's a remote end of the connection waiting on an ACK before it sends more data. Even in the case of connectionless protocols like UDP, chances are there is some sort of app or session layer check that will defacto hold back sending more data until it receives a response. You've managed to stem the firehose that's half of the problem without resorting to cheap ass tactics like false RSTs.
-
The beauty is, done right you don't need to know anything about the problem causing traffic to mitigate its effects. Assume you tag your priority packets with a deep inspection based on port/src/dst/prot (This is what almost never gets done). Whether it's eMule is running on port 443 or through a proxy or if it's a worm or the file sharing program of tomorrow, it gets throttled back.
-
The drawback is it's a bitch and a half to get installed and three more bitches worth of pain to get configured and tuned. Once you do though, it rocks.
-
[I kid about cubs fans. I lived off of Addison Ave and dealt with my fair share of that scenario, but the bulk of the people in line were ok.]
Re:Simpler solution (Score:1, Interesting)
Low-end SonicWall boxes top out at around the same amount of connections - pretty awesome that a $50 box can keep up.
Bullshit (Score:4, Interesting)
In other words, your DSL line is perfectly capable of handling an uplink that is actually used for more than an occasional HTTP request without bogging down. The reason it doesn't do it is poor engineering of the DSLAM. With better tuning and queue management algorithms like RED (Random Early Drop) they will cooperate with TCP congestion control to avoid overloading the uplink buffers. Your DSL line will work just fine without a third-party bandwidth management tool.
Why is the DSLAM poorly engineered? The simple explanation is incompetence. Conspiracy theorist would probably claim that it's intentional because ISPs don't want you to use bandwidth-intensive applications. The truth is probably somewhere in the middle: the original flaw was a combination of lazy engineers and the fact that most users don't really use their uplink so much. It's not being fixed beacuse it serves the interests of the ISPs.
Re:QoS? (Score:3, Interesting)