Forgot your password?
typodupeerror
The Internet

Georgia Tech's ShaperProbe Detects ISP Traffic Manipulation 113

Posted by timothy
from the shaping-is-such-a-euphemism dept.
An anonymous reader writes with this excerpt from Ars Technica: "Two researchers at Georgia Tech can tell you exactly how American ISPs shape Internet traffic, and which ones do so. Bottom line: of the five largest Internet providers in the country, the three cable companies (Comcast, Time Warner, Cox) employ shaping while the telephone companies (AT&T, Verizon) do not — though that fact is less significant for the user experience than it might first sound."
This discussion has been archived. No new comments can be posted.

Georgia Tech's ShaperProbe Detects ISP Traffic Manipulation

Comments Filter:
  • Is shaping the same as throttling?
    • Re:Question (Score:4, Informative)

      by Anonymous Coward on Monday May 23, 2011 @11:03PM (#36224540)

      no, shaping is done per-protocol, and throttling is done per pipe

      • Re: (Score:3, Funny)

        Pretty sure the internet is made of tubes, not pipes.
      • by DJRumpy (1345787)

        According to TFA, the shaping being done isn't protocol specific (although it can be). They equate it to a bucket of tokens, where you start your transfer with a full bucket, and at specific intervals, you must use a token to transfer X amount of data. What that means is that when you start your transfer, you would have full bandwidth until you empty your bucket, after which you have to wait for the bucket to be filled again to continue your transfer. This has the effect of giving you a full pipe during you

    • Re:Question (Score:5, Insightful)

      by ozmanjusri (601766) <(aussie_bob) (at) (hotmail.com)> on Monday May 23, 2011 @11:05PM (#36224550) Journal

      Is shaping the same as throttling?

      Sort of.

      Online, shaping and throttling are something network companies do to customers. In meatspace, throttling is what customers want to to to network company executives.

      HTH.

      • Surely we could think of some creative shaping of said executives as well - preferably applying shaping before throttling.

    • Looking at http://en.wikipedia.org/wiki/Traffic_shaping [wikipedia.org] it sounds like "shaping" is throttling based on packet type that kicks in when bit rates get to high.
      • by AK Marc (707885) on Monday May 23, 2011 @11:43PM (#36224742)
        I find that Wikipedia is good at giving a few people's opinions of terms, but not actually backing up the vernacular definition.

        Shaping: controlling bandwidth among various protocols (whether DPI or QoS, port number, etc.). This can be enforced by throttling some traffic or by prioritization.

        Throttling: capping or reducing the bandwidth available to some identifiable clump of traffic (I use clump because all the other appropriate terms I can think of have some technical definition more strict than what I want to say). It can be done solely in response to congestion, or in the absence of congestion. It can be done on some subset of a subscriber's traffic, or to the entirety of it. Throttling is a slowing or capping of traffic. Most shaping is a subset of throttling. Oversubscription could be considered a form of throttling. Throttling is much more general of a term than shaping.
        • I find that Wikipedia is good at giving a few people's opinions of terms, but not actually backing up the vernacular definition.

          As opposed to a single poster on slashdot? At least Wikipedia insist on citations.

          So in what way are your definitions superior to those on the linked Wikipedia page? What did the wiki get wrong?

          • by AK Marc (707885)
            There wasn't a single cite to the definition. There was a definition asserted without citation, followed by the implications of that or any number of other definitions.

            What was "wrong"? Nothing worth dealing with. They give a mostly useless definition that doesn't help differentiate it from anything else, then talk about the technical implications and implementations without regard to the vernaclar, which for the average person is much more important than the overly-cited long and dry article about a te
    • Re:Question (Score:5, Informative)

      by Ungrounded Lightning (62228) on Monday May 23, 2011 @11:24PM (#36224642) Journal

      Is shaping the same as throttling?

      Shaping is when they give you a rate and enforce it. (The faster burst at startup is because you had accumulated some credit by not using your bandwidth in the immediately preceeding time.) There may be separate shaping mechanisms for different protocol families and there may also be shaping on aggregates - like total bandwidth across multiple users of a common DSLAM.

      Throttling is when, after they notice that you've used a lot of bandwidth lately, they turn down the rate on the shaper ("traffic manager").

      Shaping is mainly about things like keeping protocols from interfering with each other (by giving different classes of them separate allowances) and avoiding congestion and queue-too-full latency (by limiting the traffic sent to a following box to the amount it can handle.)

      Throttling is about keeping a user's resource consumption down by slowing him down after he's run fast for a while.

      • by AmiMoJo (196126)

        Not quite right.

        Throttling is where they simply slow down your entire internet connection. Typically when you download more than a certain amount of data the ISP punishes you with throttling. Kind of like going to McDonald's and ordering a Supersize Big Mac meal, then a member of staff comes over and punches you in the gut as punishment for overeating.

        Shaping is where the ISP tries to slow down certain traffic to give priority to others. The typical use is to slow down P2P and large downloads so that web br

        • Throttling is where they simply slow down your entire internet connection. Typically when you download more than a certain amount of data the ISP punishes you with throttling.

          I note that this can be accomplished in a hierarchical traffic manager / policer: The first level of the hierarchy might be each of several classes of traffic to (or from) your machine, while the second level might be the aggregate composed of all of this traffic. Throttling would consist of turning down the second level's programmed

    • Re:Question (Score:4, Insightful)

      by L4t3r4lu5 (1216702) on Tuesday May 24, 2011 @06:39AM (#36226114)
      It's car analogy time!

      Shaping is like putting a bus lane / car pool lane on the motorway / freeway; Buses and car pool drivers can move through quicker, at the expense of car traffic having one less lane on the motorway, much like VOIP would be given priority over BitTorrent, or somesuch, but the cars and buses are all capable of going at maximum speed (should traffic allow).

      Throttling is like variable speed limits. In the interest of keeping traffic moving freely across the whole motorway, the speed of heavily trafficked areas is slowed down so it doesn't cause congestion. 70MPH becomes 50MPH in the same way that 10Mbit becomes 2Mbit.

      Data caps are like a bastard child of toll roads; You've driven a certain distance on this road which is covered by vehicle excise or fuel tax, now you have pay a toll. To travel further on this road, you pay more tolls. You can drive only so far each month on the toll roads for free.

      HTH.
      • by ArisGT (1984622)
        A better analogy is during rush hour they have those traffic lights on the on ramps controlling the rate of cars entering the interstate.
  • by Anonymous Coward on Monday May 23, 2011 @10:54PM (#36224480)

    I work for a college, and we shape / police traffic to / from the Internet.

    This was a necessity on our 3Mb link of many years ago, but has still been useful on our 1Gb link of today.

    This policy has greatly improved the user experience. Interactive protocols have low latency, bulk transfer protocols get sent to the end of the line. Where we do slow down things, it isn't really noticed by most folks. After first implementing this many years ago, we immediately got positive feedback. Now it is just "how things are."

    Hell, I shape / police traffic at home to my cable modem. VOIP and interactive ssh are still usable even with huge downloads going on now, and users hammering the public wifi I provide to my neighborhood.

    • by Urza9814 (883915)

      It's great that it improves the quality of service...but here's the problem: My neighbor and I both pay for the same service. So why should he get better service because he's using a more interactive protocol? Why should my file download slow down because my neighbors are all on VoIP calls? Depending on how the traffic shaping is set up, isn't it possible that someone who's paying the exact same amount I am for internet service could be getting several times more real data throughput simply because of what

      • why should he get better service because he's using a more interactive protocol?

        "Better service" has multiple definitions here. I would say that interactive sessions get "worse service" when the latency increases, perhaps because you have some huge download that is eating away at bandwidth that an interactive session could be using. Do you want your VoIP calls to get choppy, or do you want your downloads to be a few minutes faster?

      • by Anonymous Coward

        I think it equates to the freeway, you have small cars that are tiny and capable of getting to their destinations quickly, then you have huge semi trucks that trundle along. Should it be fair to only have 1 lane and have the small cars get stuck behind the huge slow trucks, no. So you create lanes and ask that slow traffic keep right so the rest of the internet can move along.

      • by Lehk228 (705449)
        your neighbor isn't getting better service, your neighbors VOIP packets are getting more effort to delivering them without excess latency while your torrent packets show up where they can be fit in, but get a lot of bandwidth
      • It doesn't hurt your download if there's half a second nothing, then half a second double speed. The average data rate is still the same. However it massively hurts the VoIP connection.

      • by GooberToo (74388)

        See, that's the problem. You're a selfish, self entitled idiot. That's the only problem here in this discussion.

        You're argument boils down to, everyone else's experience must suffer and frequently become unusable so you can download something a second or two faster. That's stupid. That's selfish. That's idiotic. And the simple fact is, its non-discriminatory. When you require interactive services, you get the same quality of service.

      • by AvitarX (172628)

        For now anyway, interactive services are low bandwidth, for example, a typical skype call is 400kbps / 400kbps (high-quality), or 1.2mbps/1.2mbps (HD).

        Torrenting a popular 24hour+ old file uses significantly more (I get at least 2.4mbps on comcast, peaking at 3.6, I throttle the upstream to between 400 and 800 mbps, I peak at 1.2mbps otherwise). I download news articles at 16mbps, putting me about 6 times the maximum a skype call will use (highest bandwidth interactive connection I can think of).

        An FPS is 1

    • by Idbar (1034346) on Monday May 23, 2011 @11:20PM (#36224620)
      Hum... that's the issue. You're providing for a bunch of students that don't have a contractual agreement with you. Now imagine that the company you contracted the 1Gb link to, started shaping your traffic. That eventually comes to you, not receiving the 1Gb you paid for.

      I understand you sometimes need to shape the traffic to prioritize services you want to perform better. But these companies are getting their money offering services they don't completely provide: First they charge you for access they can shape. And second, they put a cap just in case you get away with it.

      Again, I understand your work case, because if you don't pay for it, normally you tend to abuse it. But if you pay for it, why would they need to mess with your traffic?
      • by Dhalka226 (559740)

        But these companies are getting their money offering services they don't completely provide

        I hesitate to use words like "every," but I can think of no ISPs who advertise their plans as anything other than "up to X Mbps." Why you believe that not getting that is somehow a service they don't provide eludes me, particularly when the reason you are not receiving it is legitimate shaping activity*.

        And second, they put a cap just in case you get away with it.

        I tend to agree with you on this one, for a lot of r

      • because if you don't pay for it, normally you tend to abuse it. But if you pay for it, why would they need to mess with your traffic?

        Usually when I'm on public wifi (even if paid for), I tend to put downloads on a slow trickle, overnight if possible. Ye olde wget is awesome for that:
        wget --limit-rate=20k http://www.example.com/bigfile [example.com]
        will limit the download speed to 20KB/s. All platforms have wget available.

        (By the way, why does Slashdot make text URLs into hyperlinks even in code sections?)

      • by AmiMoJo (196126)

        ADSL ISPs in the UK are pretty much forced to shape and set ridiculously low caps because of the amount the owner of the telephone lines and other equipment, BT, charges them for access.

        BT initially charged on a per-user basis, but then they switched to charging by the amount of bandwidth they supply to the ISP. So where as before an ISP paid £X per user regardless of how much data they consumed, under the new system they would pay for say a 144Mb "pipe" and choose how many users to put on it. O

        • by markxz (669696)

          ISP are able to install their own equipment/connections in BT exchanges and use the unbundled local loop for the last stage of the connection.

          Users connected to smaller exchanges don't benefit from this as there is not enough customers to make it economic for the ISP. These users are forced to use the BT backhaul network at excessive cost.

      • It's "Up to 1Gb!"(tm)
      • Students pay, maybe even a "technology" fee. The difference is schools operate on a (mostly) non-profit basis and can be trusted to maintain the network for the benefit of the users, unlike ISPs that are more concerned with the shareholders. Contracts or regulation can't really keep them honest, since some shaping is probably always necessary. The answer is more competition, but that is hard to maintain in any market let alone a utility market.
      • by StikyPad (445176)

        You're providing for a bunch of students that don't have a contractual agreement with you.

        You mean besides the enrollment contract, but otherwise you're spot on. Just like I have no contractual agreement with my ISP other than the one I signed when they installed my service.

    • I work for a college, and we shape / police traffic to / from the Internet.

      This was a necessity on our 3Mb link of many years ago, but has still been useful on our 1Gb link of today.

      This policy has greatly improved the user experience. Interactive protocols have low latency, bulk transfer protocols get sent to the end of the line. Where we do slow down things, it isn't really noticed by most folks. After first implementing this many years ago, we immediately got positive feedback. Now it is just "how things are."

      Hell, I shape / police traffic at home to my cable modem. VOIP and interactive ssh are still usable even with huge downloads going on now, and users hammering the public wifi I provide to my neighborhood.

      You make a good case, and I agree. I'd like to know whether or not you told your customers how you were shaping their traffic.

      I have no issue with enforcing (your idea of) quality of service on a network. What bothers me about Comcast is the general lack of transparency behind it all. Their policies should be public and open to scrutiny, minimally so I know what's going on with the service I'm paying for and ideally so they can be held directly accountable if they implement an absurd form of shaping.

    • Shaping traffic based on the protocol is generally a good thing, assuming you get it right (i.e. give the protocols that need priority priority). Shaping is compatible with network neutrality, as long as you are not using the address packets originate from or the payload as part of the shaping rule.

      Unfortunately, I simply cannot trust that an ISP like Comcast will stick to shaping rules that only use the protocol. This is one of those cases where regulation is needed (particular given how many hand-out
    • by Anonymous Coward

      Does your college run the last mile through public land that was taken or "borrowed" from the home owners? Does your college charge a monthly fee to the same owners of that land that was taken? Does your college negotiate a monopoly with the local government to ensure they are the only ones that can use that last mile? Effectively making the residents pay for that last mile over and over again and still only have one choice of internet provider over those lines?

      I'd rather pay the local government for the

    • by arkhan_jg (618674)

      There's nothing wrong with QoS and putting some traffic at the front of the router queue. It just means light, latency important packets don't get drowned in a torrent of bulk traffic that doesn't really care what order the packets arrive in. But you still have the same amount of capacity for traffic at the start as the end, you're just determining which app gets to use it first, and which app you're prepared to throw away packets for first if necessary at heavy load.

      The article is not about that, but bucke

    • by wye43 (769759)
      What you describe is indeed good, but it should not be done with shaping. It's a functionality implemented at another layer by the TOS flag (http://en.wikipedia.org/wiki/Type_of_Service). While there are some bad apples out there, any decent TCP/IP stack(Linux has it since the stone age) has implemented this at least in its most primitive form (interactive/bulk cases).
    • by GooberToo (74388)

      Exactly. You can always tell when you've discovered a moron, and idiot, and a downright knuckle-dragger, because they immediately equate all traffic shaping as bad. In fact, traffic shaping is in wide use and is generally a good thing.

      Traffic shaping frequently means your SSH session stays fast and responsive despite massive FTP downloads elsewhere. It means your games remain responsive and fun despite your neighbor stealing movies over the next several weeks. It means your HTML is delivered quickly, allowi

  • As long as its done in a neutral manner based on whats being sent as opposed to whos sending it.

     

  • by drtsystems (775462) on Tuesday May 24, 2011 @12:12AM (#36224860)

    This is said (although almost in passing) in the article. But I will repeat it because i know how few of us RTFA. Time Warner advertises its PowerBoost feature (and Comcast has something similar) where you get like double your usual bandwidth limit for "burst" downloads and then you get throttled back to your limit after the burst is complete. This is a FEATURE they advertise, not something bad. It allows you to (for example) get 15mbit when download a web page or small file on your 7mbit plan. Notice its a 7 mbit plan, they are not throttling you below your plan's rated speed. They are giving you faster downloads for a quick burst. There is plenty wrong with Time Warner, but this isn't one of the the problems.

    • Re: (Score:2, Informative)

      by Anonymous Coward

      My neighbors add a couple more tiers to PowerBoost. Weekends the cablemodem borders on useless for any low latency or sustained streaming service (for example hulu, netflix). Weeknights interactive games are tolerable with hit & miss reliability for sustained streaming. Weekday mornings flawless service.

      • Weeknights interactive games are tolerable with hit & miss reliability for sustained streaming. Weekday mornings flawless service.

        Ok, so I've designed this game network protocol which gives users with good bandwidth smooth gameplay and high precision, but gives lower bandwidth users smooth gameplay and lower precision -- client side prediction and synch rate are determined by a test -- a test at the beginning of your connection. This test says you have a monster awesome responsive network, and I'll give you the best experience you expect -- Until your BS ISP plan decides that you only get that for "bursts", and now you're in a fire-f

    • by Anonymous Coward

      Trying to give messing with your bandwidth a positive spin, sounds like newspeak to me.

    • by Anonymous Coward

      Not true, when I had Time Warner they throttled back my Netflix. I could tell because at first it came through at DVD quality and then dropped down to circa 1990 DIVX blocky crap quite quickly. But the speed was claimed to be 7 Mbit, which should be plenty fast enough for streaming DVD quality, let alone Netflix H.264 high compressed DVD stream. Dropped them for Grande, who was also 7 Mbit and the Netflix was crystal clear every time, all the time.
      They may be fixed now, but I initially dropped them as they

    • by EmagGeek (574360)

      It wasn't anywhere near "mentioned in passing" in the article. It was stated quite plainly and directly.

    • This is true, and furthermore "shaping" is one of the nicer/friendlier methods of managing traffic contention.

      The "my plan says 10 Meg I demand 10 Meg!" argument is simply not technically valid. If the ISP has 10G of upstream to a carrier and more than 1,000 customers, it's not physically possible for everyone to run unlimited 10 Mbps all the time. So now that there's a possibility of contention, the good/bad lies in how the provider limits you, not if.

      Possibly the worst is usage caps. After transferri

  • But it could be because it is already too late to tell.

The 11 is for people with the pride of a 10 and the pocketbook of an 8. -- R.B. Greenberg [referring to PDPs?]

Working...