Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
×
Government Networking The Internet Technology

FCC Relying On Faulty ISP Performance Data 89

alphadogg writes "The FCC recently used speed test results from comScore as an absolute indicator of specific ISPs' performance. Consulting firm NetForecast analyzed comScore's testing methodology and data to assess whether it accurately reflects broadband ISP performance, and to assess the appropriateness of using the data to reach general conclusions about the actual performance ISPs deliver to their subscribers. NetForecast uncovered problems on both counts. They found that the effective service speeds comScore reports are low by a large margin (PDF) because its data calculations under-report performance and place many subscribers in a higher performance tier than they purchased."
This discussion has been archived. No new comments can be posted.

FCC Relying On Faulty ISP Performance Data

Comments Filter:
  • Waiting for an ACK before transmitting the next packet doesn't seem like a way of measuring bandwidth. Sounds like a measure of bandwidth + latency.
    • Re:Wait for ACK? (Score:4, Interesting)

      by topham ( 32406 ) on Monday March 29, 2010 @03:02PM (#31661428) Homepage

      TCP/IP doesn't wait for the ACK. It keeps sending until the Window is full, or the ACK is received. If the Window fills it will wait until the ACK is received (or timeout and retry, etc).

      If the test is trying to automatically place the users in specific Tiers then there could be a problem, however the rest of the issues are mostly a red herring. I use Speedtest.net and can readily attest to it's general accuracy, and I seriously doubt any other services are all that different.

      by the way, I'm not in the U.S., I actually get what I pay for.

      • What do you pay or and what do you get? (just curious)

        The speedtest.net you reference shows that the EU* is about 1 Mbit/s behind the US (which is 7.8 Mbit/s average). Now maybe your specific member state is faster overall, but the top US state is no slouch either (Delaware at 12.5). The only continent-sized federation faster than the US is the Russian Federation.

        *
        * Yes I'm making a big jump here that you live in the EU.

        • Re: (Score:3, Interesting)

          by topham ( 32406 )

          I pay for 15 Mbit down and .5 Mbit up.
          I get 15-20 Mbit down and .5 Mbit up.

          I even tested it with the FCC test since I posted the first message. It rated my connection slightly faster than speedtest.net did. (Not significantly, and I'm sure it would vary).

          Until recently I paid for 20 Mbit / 1.0 Mbit; but I wanted to save some money.

          The general illusion in the U.S. is many markets are the numbers for a zip-code are good, or even fantastic, but only a tiny fraction of the zip-code may actually get any service

        • Re: (Score:3, Informative)

          by QuantumRiff ( 120817 )

          Course my Zipcode has 10MB charter available in it. Of course, thats the homes right around the golf course. And the only ones that have cable. The rest of us get either Satellite, 3G, or Wisp coverage that is spotty, and drops alot. But according to the FCC, I have high speed available, since those houses on the golf course have it, and were all in the same zip code.

          And I live a few miles from a very large town. So if the majority of the EU is at 7.8Mb/s and they have access to that, I think its aweso

          • >>>So if the majority of the EU is at 7.8Mb/s

            6.8 Mbit/s

            >>>I have 600k\s. I pay $50/month for that, and feel lucky.

            That's nuts. I have DSL through my phone company and it only costs $15/month. If I were in a position of power (nudges FCC), I would mandate DSL must be supplied to any customer that requests it. The wires are already there - all that's needed is to install the ~$1000 DSLAM at each central station.

      • Comment removed (Score:4, Informative)

        by account_deleted ( 4530225 ) on Monday March 29, 2010 @03:19PM (#31661650)
        Comment removed based on user account deletion
      • I prefer the speedtest that you can download and put on your own servers.

        There are far too many unknown when testing from point A (your PC) to point B (the speedtest server). Because it's a public speedtest server, that hans people are beating on it all the time. Is the server capable of handling it? The uplink? Is there a congestion problem between you and them. mtr helps to give a better image of potential problems during the test.

        I have access to servers

  • by Bob_Who ( 926234 ) on Monday March 29, 2010 @02:35PM (#31661130) Journal
    I just am so surprised. Its run by a bunch of government employees, and they are rarely faulty.
    • Re:FCC is faulty? (Score:4, Insightful)

      by Otterley ( 29945 ) on Monday March 29, 2010 @03:01PM (#31661420)

      And we all know employees of private companies are infallible.

      • Private companies don't have access to my paycheck.

        For example when Comcast yanked TCM off my cable without notice (and in violation of FCC rules), then sent me some paperwork that I could get TCM back by getting a "free" digital converter box at $5 per month rental (times 3 sets), I mailed them a photo of my middle digit attached to a formal complaint to the FCC, and asked them to cancel my cable effective the day they yanked TCM w/o telling me.

        Now when Comcast mails me a letter asking me to "come back" I

        • by Otterley ( 29945 )

          Private companies can use legal process the same way the Government can to obtain access to your paycheck. Ever heard of "garnishment"?

          • >>>>>Private companies don't have access to my paycheck. For example when Comcast...I asked them to cancel my cable
            >>
            >>Private companies can use legal process..... Ever heard of "garnishment"?

            Are you saying Comcast can drag me to court, convince the judge to FORCE me to be their customer, and suck/garnish a $70 subscription fee out of my paycheck each month??? - Dude. That has to be the most stupid idiotic thing I've ever heard anyone say. Of course they can't do that.

            Now c

            • by Otterley ( 29945 )

              No, but a judge can hold you liable if you violate a contract or damage a private party. I wasn't trying to imply that private parties can levy taxes.

              • While you are correct, that a judge can hold you liable for contract violation, in this case Comcast unilaterally changed the terms of the contract. The GP was within his rights to insist that the contract be honored, or be terminated without damages. He chose "terminated".

    • Re: (Score:3, Interesting)

      Bad data == good data for a politician.

      Especially when it's in favor of whatever they desire to happen. Politicians wanted healthcare so they generated a faulty "42.5 million americans uninsured" statistic. How? Using a couple mail-in postcards from voluntary recipients. Hardly scientific. (Real numbers from scientists estimate the number as 5-15 million uninsured U.S. citizens. +9 million if you include illegal non-citizens/intruders.)

      And of course if the FCC stats show that ~40 million Americans do

      • Real numbers from scientists estimate the number as 5-15 million uninsured U.S. citizens. +9 million if you include illegal non-citizens/intruders.

        Senator Orrin Hatch [wikipedia.org], a scientist???

        Besides, here are his exact words. He doesn't even dispute the figures (because he does admit that they're uninsured). He just questions the assumption that those uninsured people even want insurance (which is a separate argument of its own, if you want to argue that, argue that, don't change the freaking numbers).

        "By the way, of that 47 million people, when you deduct the ones who could have insurance through their employers but don't, you deduct the 11 million that basically qualify for CHIP or Medicaid but don't realize it (and) are not enrolled, you deduct those who are over $75,000 a year in income but just won't purchase their own health insurance, and then 6 million people who are illegal aliens, my gosh, when you put that all together, it leaves about 15 million people. So we're going to throw out a system that works for 15 million people."

        And by the way, people without health insurance, whether they want that health insurance or not, do affect the rest of us. And unless Senator Hatch wants to re

        • Correction: I should have said that "that prevents [them] from turning away patients". By saying "mandates" instead of "prevents", I actually said the opposite of what I meant.
        • Never mind the fact that Hatch cleverly ignores the fact that he's double-dipping on the exclusions.
      • (Real numbers from scientists estimate the number as 5-15 million uninsured U.S. citizens. +9 million if you include illegal non-citizens/intruders.)

        [citation needed]

        My five minutes of googling has not come up with sources that agree with your figures. Typically, lowball figures like yours are due to the following errors:

        1. They include only people without any insurance for the entire year, though at any given time during the year, a significantly higher number of people are uninsured.
        2. They double-dip

        • There's a simpler answer: The "45 million uninsured Americans" that politicians kept talking about actually ARE insured - by the government's existing programs. You're old, or you're a child, and when you enter the hospital you are instantly signed-up for Medicare or SCHIP. That drops the number to about 20 million instantly.

          Also the stats come from about 5,000 mail-in surveys from voluntary respondents (the other persons threw-out the surveys). NOT an accurate scientific polling, therefore the

          • There's a simpler answer: The "45 million uninsured Americans" that politicians kept talking about actually ARE insured - by the government's existing programs. You're old, or you're a child, and when you enter the hospital you are instantly signed-up for Medicare or SCHIP. That drops the number to about 20 million instantly.

            Please provide a citation for that. No figures I have been able to find agree with that number.

            Also the stats come from about 5,000 mail-in surveys from voluntary respondents (the ot

    • by blair1q ( 305137 )

      They have jobs.

      Now compare with your lot. All of you.

    • by cmacb ( 547347 )

      Obviously you don't understand the methodology here.

      The failure of government is meant to convince us that we need to spend more money on government so they can do better testing to convince us we need to spend more money on government.

    • Exactly. While the commercial employees function perfectly... while working hard to rape the money out of your as hard as they can. ^^

  • How Much (Score:2, Interesting)

    by DeanFox ( 729620 ) *

    I wonder which ISP owns comScore. Who got the worst rating?
    • Think about it; What other ISP's name starts with com?

    • by skids ( 119237 )

      Better question: who owns NetForcast?

      Actually I'm inclined to chalk this one up as plain old sniping between competing performance testing companies, rather than either the FCC/pols or the ISPs trying to fudge numbers.

  • by Spazmania ( 174582 ) on Monday March 29, 2010 @02:45PM (#31661254) Homepage

    comScore got the data more or less right. The OP's main complaint seems to be that the speed is under-reported because packet loss causes the TCP session they used to slow down. Guess what? Packet loss causes the TCP session to slow down. Customers on ISPs with noticeable loss rates experience slower performance than the line's rated speed. Hello!

    • by amorsen ( 7485 )

      The OP's main complaint seems to be that the speed is under-reported because packet loss causes the TCP session they used to slow down.

      Other things cause TCP sessions to slow down too though. Like Windows XP with its lack of window scaling.

      • Like Windows XP with its lack of window scaling.

        What are you talking about? There are all sorts of tweaking tools that let you enabled window scaling in Windows XP.

        • And what percentage of Windows XP users do you think used those tools? I know I didn't.
          • Re: (Score:1, Troll)

            It doesn't matter. He said it lacked window scaling. This is false because you can enable it through the registry or tweak tools.

            • Re: (Score:3, Insightful)

              by Bigjeff5 ( 1143585 )

              Technically false, but true in practice.

              Better would have been to say Windows doesn't come with any such tools, and therefore the vast majority of people don't have access to such a feature because they lack the technical ability to get it.

        • >>>>>Like Windows XP with its lack of window scaling.
          >>>
          >>tweaking tools that let you enabled window scaling in Windows XP

          I see a lot of XP bashing but even if it didn't have "window scaling" I still consider it the best Microsoft OS. Still better than Windows 6.1's (7) ribbon interface. I liked this OS at first, due to it fixing the Vista Beta release, but now I'm sick of it. The menus keep randomly moving around instead of staying put. Well..... at least it's not as bad

      • If you're not causing latency with QoS and you're not going through a satellite loop, lack of window scaling doesn't become a major factor until you're in the 100mbps range, well beyond any but the largest of today's consumer broadband links.

        • XP was written in the day when most Internet connections were either dialup, or broadband that was slow by today's standards (128, 256, 512 or 768 kbps DSL, for instance). A small window size is preferable for that kind of situation, and as you say, provided the latency is low, it scales up well to the 100Mbps range. So in the US with your sub-100ms latency to the majority of hosts, it's not really an issue. But if you are more geographically distant, window scaling matters a lot.

          Here in Australia, the XP T

          • If I correctly recall the article, comScore tested against the nearest Akamai server. That would have all but eliminated latency due to geographical distance.

            As for ADSL2, it's mainly a political thing with the phone companies. They're required by law to unbnudle and sell copper pairs but if they deliver fiber they have an unregulated monopoly. So anywhere they'd seriously consider ADSL2 is getting fiber instead.

            • Ah true. I was speaking more generally about whether XP's lack of window sizing 'mattered'. In relation to the comScore tests though you are right ... doesn't matter at all.

    • by COMON$ ( 806135 ) on Monday March 29, 2010 @03:05PM (#31661462) Journal
      Ya I pay for the "Extreme" Roadrunner in my area. which gives me a better upstream for my telecommuting wife. supposedly 10M/1M but it is more like 3M/768k, most of this is due to really high latency and dropped packets. When it works it works, so I guess by this guy's definition I get my 10/1, just as long as you don't count the packet loss...
  • At least they don't have data saying that our speeds are faster than they really are. This way the problem of 3rd world net speeds can be addressed.

    And really the argument 'these scores are low because X slowed them down' is really not sound. If X really exists then the connection is slowed in real life. I would bet this is the results of what people expierence over the net. And should be plenty fine to help determine what needs to be done to get back in the same ballpark as the rest of the industrialized
    • >>> This way the problem of 3rd world net speeds can be addressed.

      What? The US has an average speed of 7.8 Mbit/s. Third world speeds are 3.6, 2.5, 1.8 in Chile, Brazil, or Argentina respectively. Or how about 1.9 for South Africa. Or 2.1 in China.

      Compared to other continent-sized federations, the US is #2, behind Russia but ahead of the EU. The US is faaaaaar faster than a "third world speed"

  • by guruevi ( 827432 ) on Monday March 29, 2010 @02:58PM (#31661378)

    Both sides need to learn more about statistics.

    The report fails to mention that across a large enough population, the results will be more-or-less correct within a certain percentage point because as he mentions, some people will test with a lot of bandwidth available at a certain point but others will test with their available bandwidth constricted. Overall, out of a large enough population the outliers are washed away.

    comScore needs to realize that correlation != causation. It's not because your bandwidth correlates with other users' high-bandwidth plans, that it is caused by you actually buying the plan. But even then, even in the report the statistics show that it evens out pretty good with only a small percentage error.

    Off course this brief report reeks more like paid research. Off course comScore measures the users' experience connecting to large-bandwidth centers like Akamai which has a lot of large sites on it and it doesn't accurately measures what the provider offers in the last mile. I don't care that I actually get my 10Mbps connecting to my neighborhood (unless a bunch of my neighbors actually host the Linux-ISO torrent I want) I care about getting on average getting maybe 50% of what I pay for which I usually don't get (I get closer to 1-10% depending on what I'm doing). comScore accurately reflects the poor status of broadband in this metropolitan area - dual-ISDN speeds (early 90's) on the best high-tier packages money can buy in this area. The only alternative is DSL which is horribly outdated.

  • by irn ( 1773184 )
    I guess I need a new ISP. According to the article there is roughly 17% of the population who at any given time is getting MORE bandwidth from their ISP than what they're paying for? Is that right? Did I misread the article? I'm sure comScore would have at least put me in a much lower tier than what I pay for. Something here doesn't seem right.
    • by Belial6 ( 794905 )
      I must be in that 17%. I get 20Mb/6Mb on my 12Mb/3Mb Comcast Business line a good 90% of the time. I have rarely seen it drop down to the 12Mb/6Mb that is advertised, and in the last year and a half, I have seen it drop down to 3Mb/1Mb for a total of about 10 hours across three different instances.

      Now, I do pay $60/Month for my connection which is 3x the cost of Residential Comcast, but the extra $40/Month as been well worth it to me.
      • Probably just a slight slip, but isn't Comcast Business Basic 12/2 for $60?

        FYI, next town over from me has Comcast, and their residential lowest tier service (15/3) is $43 a month. You can get it for $20 for 6 months if you bundle with Comcast Cable or Voice.
        • by Belial6 ( 794905 )
          It may be 12/2 and not 12/3. I might be forgetting given that I am almost always at 20/6. If the $20 rate is only for 6 months, then it is even a worse deal to go with Residential.
  • As if dslreports.com isn't useful?

    Sheesh

  • So here is the outline of their claims, with responses.
    Data gathering errors
    Only one TCP connection is used
    Basically valid, it's a pretty rare net activity nowadays that actually maxes out the connection by itself, no idea if the promises the ISPs make contractually include any wording about per-connection performance.
    Client-server delay is variable
    Tough, this is a reality of how the network operates, if an ISP promises

  • "The Secure features a fingerprint scanner and a thermal sensor 'so that the finger alone, detached from the body, will still not give access to the memory stick's contents."
    I'm sure that if someone went through the trouble of removing the finger to access the Secure Pro, then they'd go through the trouble of warming the dead finger up so that they could have access. seems kinda gimmicky to me...

"The voters have spoken, the bastards..." -- unknown

Working...