Why BitTorrent Causes Latency and How To Fix It 315
Sivar recommends an article by George Ou examining why BitTorrent affects performance so much more than other types of file transfer and a recommendation on how to fix it. The suggestion is to modify P2P clients so that, at least on upload, they space their traffic evenly in time so that other applications have a chance to fit into the interstices. "[Any] VoIP [user] or online gamer who has a roommate or a family member who uses BitTorrent (or any P2P application) knows what a nightmare it is when BitTorrent is in use. The ping (round trip latency) goes through the roof and it stays there making VoIP packets drop out and game play impossible."
QoS? (Score:5, Funny)
What? Oh, damn Linux! What? Oh, Windows can do it too now? Why do I always have the good ideas about 10 years too late?
Re: (Score:2, Informative)
Re: (Score:3, Insightful)
Re: (Score:2)
Re: (Score:2, Funny)
They got a cable connection just for their internet-enabled toaster? Now THAT'S luxury!
Re:QoS? (Score:5, Insightful)
Re: (Score:2, Insightful)
Re:QoS? (Score:4, Insightful)
Re: (Score:2, Flamebait)
I do also have a Vista machine, and in all fairness, it has never been infected with anything worse than Vista itself.
Re: (Score:2)
Don't hold your breath, I don't think we have viruses that bad yet.
Re: (Score:2)
Re: (Score:3, Interesting)
Re:QoS? (Score:5, Funny)
I am sorry for your loss (Score:3, Funny)
Re:QoS? (Score:4, Funny)
Re: (Score:3)
Re:QoS? (Score:5, Insightful)
Re: (Score:2)
Re:QoS? (Score:4, Informative)
UNITS! (Score:3)
850 KBps = 6.6 mbps
70 KBps = 560 kbps
I use RR also, and those are both reasonable numbers.
Re:QoS? (Score:5, Informative)
L7-filter can even manage traffic at the application layer. Just set Bittorrent to "Bulk" and put Skype and Xbox live as "Premium."
Managing traffic on the router level is a lot easier than on the PC level, especially when you have several devices on a single network competing for scarce bandwidth.
Re: (Score:2)
I have L7 on dd-wrt, but the torrents start using encryption on port 443 almost immediately. So I block 443, then they start using random ports. Its like a game of whack-a-mole with a large fleet of computers. Try to meter one port and two more pop up. The end result is a computer saturating bandwidth with endless connections over the full range of ports.
Re: (Score:2)
Re: (Score:2)
QoS, but only on the Telco Side (Score:5, Informative)
This issue is with the queue on the Telco's DSLAM, or on the other side of the cable from the modem. This is more like an invited DDOS, which no amount of filtering at or behind the modem can resolve, because the modem is getting the traffic from the DSLAM after it goes through the queue.
The only way to have QOS solve this issue would be to ask the telco to do the QOS for you, and the amount of processing power to do that nicely isn't trivial.
Re: (Score:3, Insightful)
I love these home geek "i know how to flash DD-WDT and click on a GUI" networking experts, who fail to grasp your point above (i.e. QoS = OUTBOUND).
Since downstream QoS from telco aggregation router is not practical to implement, the best fix is to throttle the clients on the end user PCs, free and just a few clicks away.
Or if you want to be really advanced, QoS outbound from a second router (or linux gateway or firewall etc.) behind your WAN router but really that's overkill for 99% of users.
Re:QoS, but only on the Telco Side (Score:5, Informative)
Re: (Score:2)
Re: (Score:2, Informative)
UDP or IP protocols do not care at all, and TCP sessions don't slow down until they realize packets are being lost which can take up to 10 packets per connection.
So when remote BT clients hit with 6 incoming TCP sessions, that is at least 60 packets without any rate limit. And BT will do that over and over again.
Re:QoS, but only on the Telco Side (Score:5, Informative)
Once it's done at the network level the same can be applied down to the user level with the packets as they're tagged.
What we lack is ways for routers to signal upstream routers for dynamic QOS to the customer network.
Re: (Score:2)
My upload and download are the same.
But, why does it matter what the upload/download ratio is? P2P programs try to maximize transfer, and if several people are uploading to a single person, that could easily overwhelm a normal download pipe, like a bittorrent that has many seeders and few downloaders.
Re: (Score:3, Informative)
Upload speed makes a huge difference ... so cutting your torrent upload to half your upload bandwidth solves the problem:
1. the fewer packets your torrent app sends, the fewer replies it receives, so more bandwidth available for other data such as web pages, gaming data, etc.
2. the fewer packets your torrent app sends, the more upstream bandwidth your other apps have to request data such as web pages, gaming data, etc.
Re:QoS, but only on the Telco Side (Score:4, Informative)
But that isn't what the article is about. The article is looking at a download link that is saturated from P2P transfers from other people. Since the DSLAM queue isn't in the users control, it is a bit harder to prevent the P2P traffic from saturating the link.
Re: (Score:2, Informative)
Re:QoS, but only on the Telco Side (Score:5, Informative)
Re: (Score:2)
Re:QoS, but only on the Telco Side (Score:5, Interesting)
Hell, Azureus has a plugin to test ping an IP address/website, and if it takes longer than a set time, it slows down your uploads. uTorrent has a feature like that, as well.
Re:QoS, but only on the Telco Side (Score:4, Funny)
I'm hoping you meant 20ms...
That's not even lag, that's simply not being connected to the server!
Re: (Score:3, Interesting)
When I'm not using http to download something then nntp can download at full speed. When I do something on http it will get the full bandwidth. It's not instant though so it takes a few seconds to kick in. I suspect it's dropping AC
Re: (Score:2)
this is why i like smoothwall, the best part of smoothwall is that it will run on slow, cheap computers, some have even managed to get it to run on 386's. I know old computers use more power than a linksys, but you can get a new computer based on cheap System on a chip parts, that uses about as much power as a linksys, but
Re: (Score:2)
I don't want to use L7 because I'm trying to
Re:QoS? (Score:5, Informative)
There are two key points:
For reference, here is the script that I use to set up the traffic shaping. It might prove useful to you.
Re: (Score:3, Informative)
The key point that I've missed is the master speed throttler at the trunk of the tree - of course the router's just throwing stuff at the modem as fast as it can so its queues are never full.
Thankyou for taking the time to reply, and making my kick myse
Re: (Score:2)
(All-important disclaimer: I don't use Windows myself, of course, but I might at least be able to help people who do)
Re: (Score:2)
Re: (Score:2)
Free as in beer, smoothwall express http://www.smoothwall.org/get/vmware.php [smoothwall.org]
vmware player http://www.vmware.com/products/player/ [vmware.com]
you do have to play around with your network configuration to route it through smoothwall in the vmware player, and i don't know if you can have vmware player automatically load the smoothwall vm on boot, but there probably is a way.
a smoothwall VM will need a little cpu resource and a little ram, not as much as a full desktop linux would nee
Re: (Score:2)
My Linksys router has a section conveniently called "QoS" Is there any way to adjust these settings so I don't get severe lagging while downloading the latest Ubuntu ISO? (Or something else?)
Also, I turned on Encryption inside Azureus and my download speed jumped 4 times the rate I was getting. Just a hint for everybody who has Comcast.
Re: (Score:3, Informative)
Why is slashdot linking to stories by a troll like George Ou? His treatment of Peter Gutmann [cypherpunks.to] is unforgivable.
What's so bad about his treatment of Gutman? Gutman wrote a crazy tinfoil hat piece about how Vista's DRM will steal your soul and George flamed the hell out of him. From your link.
http://www.cypherpunks.to/~peter/zdnet.html [cypherpunks.to]
Schneier is a moron if he thinks telling Hollywood no will force them to use non-DRM content. All you need to do is look at the CableCard fiasco. You give Hollywood the finger and they give you the finger right back because they'd
rather NOT have any content on the PC to begin with. Like Apple, Microsoft
will humor Hollywood so they come join the party. Once they're in, they'll
get screwed out of their DRM protections because Microsoft won't patch the DRM
holes and let their customers bypass DRM. The latest DRM stripper for Windows
Media has worked for almost 2 months now and Microsoft hasn't patched it yet.
Ok, so it's nasty to call someone a moron. And it's not really true either. It's ideology that causes Schneier and all the Web 2.0 'experts' to say this. He's no fool but he can't differentiate between it would be good if something being true and something being true. It would be good
Re: (Score:3, Informative)
Cracking DRM is illegal in some countries. Is George Ou saying its better to break the law in this way than not have access to certain media?
No, he isn't, and I'm beginning to see why he gets angry arguing with people who don't understand what they are talking about and won't read what he says.
Let's take the whole thing from the top.
1) Microsoft's marketing department decided that Vista needs to support BluRay.
2) The BluRay Disk association [wikipedia.org] said that if they want to do this they need to support protected media paths and all the other nonsense.
3) Microsoft did that.
4) The net result is that you can Windows Vista and a software player to play Blu
short answer: (Score:3, Funny)
Re: (Score:2)
My Roommate owes me 5000g (Score:3, Funny)
Do you know how many times I've died in WoW because of his porn downloading?
He's paying up, I need my epic flying mount...
Re:My Roommate owes me 5000g (Score:5, Funny)
As long as you haven't signed a contract with your roommate, then you could throttle him
Re:My Roommate owes me 5000g (Score:5, Funny)
eewww. he no doubt can handle that himself.
Next on /. (Score:5, Funny)
Simpler solution (Score:2, Insightful)
Re: (Score:3, Informative)
Re:Simpler solution (Score:5, Insightful)
Re:Simpler solution (Score:4, Informative)
That doesn't address the number of open connections issue. Bittorrent clients can often have hundreds of open connections while a browser or a game may only have 1 or 2 connections open. So when the game sends a packet, the router gets it and recognizes that it is connection 99 of 100 open connections. If the router equally prioritizes every packet, then the app that only utilizes a single connection can still wait before being serviced.
It also doesn't solve the problem of having a roommate who will leave bittorrent on indefinitely.
The real solution is to come up with a way to analyze packets and determine which packets should have the highest priority. This is called Quality of Service (QoS). Linux and routers based on linux have access to a number of different QoS schemes, but the off the shelf routers may not have good enough hardware to run it. For example I bought a ddwrt compatible router. I dumped the original factory firmware and installed ddwrt. I turned on QoS and put http and other types of traffic at higher priority than the rest. It worked great when the router could handle the traffic. I could let the bittorrent client eat as much as it wanted but when I hit a webpage, the page loaded just as fast. But every once in a while the router would crash or become really slow and inaccessible (can't access it through ssh or http). Turning off QoS alleviated that issue but of course bittorrent would starve out the other apps. In the future I plan on buying a router with a faster cpu so I can leave QoS on.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
If the router equally prioritizes every packet, then the app that only utilizes a single connection can still wait before being serviced.
While it is possible to allocate bandwidth per connections routers rarely bother, and can't tell the difference between one connection sending 1000 packets, and 1000 connections sending one packet. The problem with TCP is when you receive an ACK packet you typically send a whole window size of data to your peer. If you receive multiple ACK's from different peers in a short space of time, you can easily flood the transmit / receive buffers of the device at your choke point (usually a modem). However if you
Wait, wait wait! (Score:2, Insightful)
Re:Wait, wait wait! (Score:5, Interesting)
I don't mind traffic shaping [slashdot.org] at all, anywhere. QoS is a good thing, even when the ISPs do it. What I mind a whole awful lot is traffic blocking, ala Comcast.
Uh, yeah? (Score:5, Insightful)
Re: (Score:3, Insightful)
Re: (Score:2)
So, if the ISPs do traffic shaping "to improve the service" it's bad, but we admit that on the small scale (when it affects ourselfs) there is a real need for traffic shaping! Thats interesting....
Despite the fact slashdot is not one mind, i still don't believe any sensible person here on slashdot has ever had a problem with traffic shaping.
Sure, there are a ton of people complaining about liars (IE they do traffic shaping to an extreme and lie about that fact claiming they don't, wasting hours of resources on our end tracking down a problem that is their fault), and when an ISP simply lies on their bills claiming you used more bandwidth than they sold you and is stated you will get in their ads, an
Re: (Score:2)
If its MY network with MY router, i have the choice of what sort of bandwidth usage will occur.
When i PAY for bandwidth from my ISP, they shouldn't limit it.
Re: (Score:2, Informative)
How clever (Score:4, Funny)
Traffic shaping works but fair-queue works better. (Score:5, Interesting)
Traffic shaping and QOS are not usually able to make that guarantee. A straight priority queue with bandwidth guarantees can, as long as you are able to actually classify the torrent traffic differently from your other traffic.
Part of the problem is that it is often not possible to distinguish between the batch and the interactive traffic with Shaping/QOS. Not only is QOS almost universally set wrong, but the simple fact is that one can mix interactive and batch traffic over the SAME ports (http, ssh, dynamically allocated ports)and that can make it virtually impossible to use traffic shaping or QOS to keep the mess away from your interactive traffic.
The best general solution is to use a straight priority mechanic with minimum bandwidth settings to separate as much of the bulk traffic out as you can, and then run fair-queueing at each priority level to take care of any that leaks through. This will do a very good job cleaning up the traffic. DragonFly has a fair-queue implementation for PF that does this. There is also at least one fair-queue implementation for PF in the wild.
Fair-queueing essentially classifies connections (the one in DFly uses PF's keep-state to classify connections), generates a hash and indexes a large array of mini-queues. One packet is then pulled off the head of each mini-queue. One enhancement I would like to make to the DFly implementation which I haven't done yet is to use the keep-state to actually determine which connections are batch and which are interactive, and have a parameter that allows the queue to give additional priority to the interactive connections by occasionally skipping the hoppers related to the batch connections. A quick and dirty way to do that is to simply check the queue length for each mini-queue.
In anycase, its a problem for which solutions are available. Regardless of what you use it has become apparent in the last few years that the only way one can classify the traffic well enough to properly queue it is by building keep-state knowledge on a connection by connection basis.
-Matt
Re: (Score:2, Informative)
NBAR on any current cisco IOS feature set will detect pretty much anything you need to prioritise without seriously impacting performance.
Juniper has something similar on their gear as well.
Easy QoS: Low latency queueing = fair queue with a priority queue as you described.
tag real time traffic as priority queue and allocate enough bandwidth depending on your capacity engineering. tag your important apps and put them in the second queue. Rest in default class.
This is really all
Re:Traffic shaping works but fair-queue works bett (Score:4, Interesting)
When I went from a T1 to a DSL line to save some money I immediately noticed the missing cisco. That little 2620 was so nice. PF couldn't hold a candle to what the 2620's fair-queue could do so I sat down and wrote a fair-queue implementation for PF (for DragonFly). It still isn't as good as what Cisco has, but it gets a lot closer then the other PF queuing mechanisms get.
I think the bit I'm missing is the batch classification. My fair-queue can still get overwhelmed by dozens of batch TCP connections if I happen to not be able to classify their traffic (and they wind up on the standard queue instead of the bulk queue). The set-up is a priority queue with minimum bandwidth guarantees plus a fair-queue at each priority level.
I keep hoping someone will take up the flag and finish it.
-Matt
Re: (Score:2)
At least one ISP here in AUS is looking at ways for customers to adjust their own shaping on the ISP end, so that you can get the perfect connection
Use randomized time rather than even spacing (Score:5, Informative)
When fixed repeating intervals are used, separate instances of a protocol (and other protocols that use repeating intervals) slowly tend to fall into lock-step patterns with pulsating waves of traffic in accord with those patterns.
In other words, fixed protocol timers can create the traffic equivalent of the Tacoma Narrows bridge.
By-the-way, ping (ICMP Echo request/reply) is a terrible way to measure network latency. ICMP is often a disfavored form of traffic as it crosses routers, sometimes even rate limited.
There are better tools for measuring link properties, for example there is "pchar" - http://www.kitchenlab.org/www/bmah/Software/pchar/
I worked on a method to do even better measurements, but I put it aside several years ago: Fast Path Characterization Protocol at http://www.cavebear.com/archive/fpcp/fpcp-sept-19-2000.html
Re: (Score:3, Interesting)
By-the-way, ping (ICMP Echo request/reply) is a terrible way to measure network latency. ICMP is often a disfavored form of traffic as it crosses routers, sometimes even rate limited.
There are better tools for measuring link properties, for example there is "pchar" - http://www.kitchenlab.org/www/bmah/Software/pchar/ [kitchenlab.org]
Ok, I've been out of network management for a couple years now, but I have never heard of pchar.
Looking at the URL you gave, there is nearly zero description about the software or how it works or how to use it.
In addition, i went ahead and downloaded the source hoping there might be some documentation giving a clue about this, and then i noticed:
As of pchar-1.5, this program is no longer under active development, and no further releases are planned.
So, to me it seems like you are saying ICMP, which is supported by literally every single device that speaks IP, is disfavored, and the current method is to use a
Re:Use randomized time rather than even spacing (Score:4, Informative)
Pchar is derived from Van Jacobson's pathchar; there is a lot of very good and very deep knowledge behind those tools.
Yes, Ping is better than nothing, and a lot better than things like DNS round trip times. But if you are probing basic connectivity of a single hop the best protocol is to use is ARP.
But pings, as I mentioned, are often rate limited or slow-path switched or even blocked. And an increasing number of folks don't even reply to 'em. Moreover, they usually don't reveal the fate of large packets to things like MTU constraints or very noisy wireless paths that tend to clobber larger packets (as in bittorrent or HTTP) more often than small ICMP packets.
By-the-way, a lot of folks have commented on how to use the Linux traffic control system to manage outbound traffic. I commercially build a small box to do this for folks who don't want to mess with "tc" commands.
But the bigger issue for outgoing links is that the providers don't keep the outbound bandwidth constant; many providers tweek the outbound pipe size fairly rapidly. This makes it quite difficult to maintain the aggregate outbound rate so that the queues build up in the user's box (where the user can do sane management) rather than the provider's box (where the provider does whatever is good for the provider.)
wondershaper (Score:2)
Works in Linux since 2002.
*yawn*
what about? (Score:2)
Layer7 traffic shaping (Score:2, Informative)
Re: (Score:2)
Even a home with only computers under your control may not be entirely under your cont
Uplink vs Downlink (Score:4, Informative)
Downlink bandwidth can be controlled in numerous ways. The easiest way is to actually run the incoming packets through a bandwidth limiter with a very large packet queuing capability. This will cause a ton of packets to build up in front of the limiter and eventually fill the TCP windows of the senders. The packets that get through the limiter will cause a stream of ACKs back from your machines at the desired data rate. The combination of the two will cause the remote senders to band-limit the packets they send to the bandwidth you desire.
when running incoming packets through a limiter you still need to traffic-shape/QOS, priority-queue, or priority-queue + fair-queue the packets going through the limiter. If you don't then your interactive traffic can wind up getting stuck in a packet queue with hundreds of packets in it. In addition to that you may have to control the advertised TCP window or even implement RED on your limiter to prevent the hundreds of packets built up in front of the limiter from turning into thousands of packets.
If you can classify the bulk traffic then you can use virtually any queueing mechanic. If you can't classify all of the bulk traffic then the only mechanic that will work reasonably well is, again, going to be a fair-queue.
Fair-queueing is not the holy grail but it is typically the most effective mechanism when combined with another queueing mechanic, such as a priority queue.
-Matt
Ok, I'll bite... (Score:2)
"Incoming data from from multiple sources via the fast core of the Internet can sometimes clump closely together when multiple sources happen to transmit data around the same time."
More like technology for idiots.
It's simple. TCP/IP has a built-in backoff mechanism. It works wonderfully when two or three TCP (and other similar, more or less polite) streams compete for bandwidth. The mechanism is stream-based and not port-based, so when one app (one port) has 200-300 active streams, yo
Does George Ou have ANY credibility left? (Score:3, Informative)
Re: (Score:2)
From the Great Geek Philosopher Hypocrates (Score:2, Troll)
The geeks of slashdot acknowledge that P2P use strangles traffic on their LAN, and feel that some modification needs to happen to address this.
However, when service providers complain about the negative effects of millions of people using P2P on their backbones, and take action to correct this, same said slashdot geeks get their panties in a bunch and cry fowl.
I'm not taking one position over another. I'm just saying that I think this may be a big reveal about why a lot of norms
Re: (Score:2, Insightful)
There is a huge difference between a corporation not giving customers what they have paid for, and the customers using that bandwidth how they see fit.
Just my 0,02
Re: (Score:3, Insightful)
Look, I'm not for legislation, but a little common sense will tell you that it simply isn't right for a small minority of the customers to use a massive percentage of available bandwidth, using applications that they themselves say wreak havok on their local network.
Yo
Re:From the Great Geek Philosopher Hypocrates (Score:4, Insightful)
Re: (Score:3, Insightful)
There's nothing wrong with reasonable traffic shaping. ISPs, however, DON'T want to do that. They want to damn near cut-off Bittorrent traffic enti
It's about control. (Score:4, Insightful)
More seriously: Me shaping my own traffic is very different from someone else shaping my traffic against my will.
To borrow another poster's analogy:
I have no problem with choosing what kind of food I eat. If I had kids, I'd have no problem choosing what kind of food they eat.
I would very much not like the grocery store to choose what kind of food is best for everyone.
Fortunately, it's in the grocery store's best interest to give customers what they want. For some reason, ISPs think it's not in their best interest to do the same.
TCP Capture effect (Score:3, Interesting)
It works like this: if the upstream bandwidth is saturated, TCP ACK packets get delayed and the sender slows transmission so the downstream bandwidth does not get fully utilised.
There is no solution other than throttling the upstream senders (AFAIK good P2P software has settings). Note larger send buffers in broadband modems actually exacerbate the problem by taking longer to flush. Best to keep them empty, and th only way is throttling.
A Better Solution (Score:2, Interesting)
-
Set your total bandwidth minus the guaranteed bandwidth you want to allocate to priority traffic masked/identified either by port/protocol/src/dest or by a deep packet (perl based) inspection.
-
If any app OR host OR connection OR port starts encroaching on the latency of other others, it gets chucked into memory jail for a fixed number of escalating milliseconds.
-
Bullshit (Score:4, Interesting)
In other words, your DSL line is perfectly capable of handling an uplink that is actually used for more than an occasional HTTP request without bogging down. The reason it doesn't do it is poor engineering of the DSLAM. With better tuning and queue management algorithms like RED (Random Early Drop) they will cooperate with TCP congestion control to avoid overloading the uplink buffers. Your DSL line will work just fine without a third-party bandwidth management tool.
Why is the DSLAM poorly engineered? The simple explanation is incompetence. Conspiracy theorist would probably claim that it's intentional because ISPs don't want you to use bandwidth-intensive applications. The truth is probably somewhere in the middle: the original flaw was a combination of lazy engineers and the fact that most users don't really use their uplink so much. It's not being fixed beacuse it serves the interests of the ISPs.
Re: (Score:2, Troll)
No one ever had 56K, no one. If you got over 50, you were doing good.
These days, the darkside^Wtelcos are degrading dialup on purpose.
Re: (Score:2)
The OP was simply saying he got in fights over who could use the line for telephone, and who could use the line for internet on a 56K modem.
Not that he got 56K out of his modem.
But argueing against that, I ocasionally had 96K (bursts, lasting about 15 seconds) out of my 56K (USRobotics v.92), ie: 12 kilobytes a second rather than 7 kilobytes a second (Telus dial-up, on brand new phone lines around 2002) and almost a perminent 7kb/s (donwload).
Re: (Score:2)
It's called data compression [wikipedia.org].
Re: (Score:2)
Do they really need to do that? I mean, really?
Re: (Score:2)
BRI ISDN is 2 64kb B-channels, and 1 16kb D-channel.
Re: (Score:2)