Thumb On the Scale? Study Finds 5 of 7 Broadband Meters Inaccurate 114
stox writes "For the 64 percent of Americans whose internet service provider imposes a broadband cap, and for those lucky enough to have a meter, I have some bad news. The president of the firm who audits many of the country's broadband meters says that he can't certify the measurements produced by five out of seven of his clients' meters because they don't count your bits correctly
are you suprised?? (Score:3, Insightful)
Re:are you suprised?? (Score:5, Informative)
Re: (Score:3)
Re: (Score:2)
Yes, because by under-reporting they can charge you ...wait you're just one of those idiots that doesn't RTFA right?
Well...yes. Because unless your ISP uses cumulated reports that span at least two months, you're getting ripped off every month and in turn being over-charged. Though to be honest, I've never seen a ISP bandwidth monitor that under reported. All of them I've seen at least here in Canada have overreported, some have reported even when the customer has the modem unplugged/off. Which makes me think that these things really aren't collecting the stats, they're making guesstimates based off of usage over X t
Re: (Score:1)
Try not to use "seen" for hearsay, it's not appropriate.
Re: (Score:2)
Re: (Score:2)
I'm actually serious.
Router with DD-WRT firmware. (Score:2, Interesting)
DD-WRT has a meter I find it to be very accurate. I guess it could be used as evidence if things do not match.
Re:Router with DD-WRT firmware. (Score:5, Insightful)
DD-WRT has a meter I find it to be very accurate. I guess it could be used as evidence if things do not match.
Depends on your ISP, I'd wager. You might get reasonable people in the billing department you can argue with.
If not, good luck with that. It'd be nice if everyone and their mother had a non-shit router, the ability to understand metrics, and the willingness to go to small claims court, but, as a wise woman once said:
Ain't nobody got time fo' dat.
Re: (Score:1)
I'm sure their TOS say that their meter is the only thing that counts.
Re: (Score:2)
Evidence only counts in court. This is them sending you a bill and you paying it or losing your internet. Good luck.
Re: (Score:2)
Re: (Score:2)
Comcast used to be close (Score:5, Interesting)
When they were enforcing the 250 gig cap, they were within 1% of my dd-wrt tally. Now that they're not enforcing the cap, their reading is waaaaaay under my actual usage. I wonder if they're no longer counting traffic that stays in the Comcast network.
Re: (Score:3, Interesting)
I recall that Comcast does not count some streaming video services against the data caps. There were some (like Netflix) who complained this was not in the spirit of net neutrality. So maybe they are still not counting some of the video you stream, depending on where you are streaming it from? Just a guess.
Here's some more info on this: http://arstechnica.com/tech-policy/2012/03/net-neutrality-concerns-raised-about-comcasts-xbox-on-demand-service/ [arstechnica.com].
Re: (Score:2)
If they're under-reporting by a large factor and keep it that way when they reintroduce caps, I may consider switching from their Business service to Residential. The reasons why I got Business are no caps, supposedly better service, supposedly faster incident resolution, and a hint that my traffic would take priority during congestion. I work from home two days a week and can't be concerned about caps. As for the rest of the supposed benefits of their Business service is a load of BS; when I've had a probl
Weights and Measures? (Score:5, Insightful)
Perhaps we need a weights and measures type certification for ISPs?
In the US it's per County, so that will be interesting!
Re: (Score:2, Funny)
As long as it's not metric.
My cable modem gets 5 megabitrods to the hogshead, and that's the way I likes it.
Re: (Score:2)
Re: (Score:3)
it also depends on altitude.
as you go higher up (mountains, etc) the weight of 1KB is lower.
(wait, wut?)
Re: (Score:2)
Re: (Score:2)
Re:Weights and Measures? (Score:4, Insightful)
This is exactly what we need. It would solve a LOT of problems. What is 20mb/s service anyway? I know we have our (logical) definition. But I guarantee your ISP has an entirely different definition that has absolutely no bearing on the speed of your connection and more to do with the price you're paying. (I work for a large ISP btw)
Government regulation is bad in almost all respects when it comes to the economy. Capitalism works best when it's unfettered and transparent. The laws the government should impose should not put chains on businesses or consumers. What the government should be doing is making the market more transparent. Don't make derivatives illegal, make describing exactly whats in them required before sale. Don't dictate what speeds or services ISPs can offer, require the ISPs to use common terms and conditions that consumers can understand. Just as you say, a certain speed should be exactly that. None of this "up to" bullshit. If there are limits on how much you can download, that should be clear and upfront, not buried on their website. Their traffic shaping policies should be clear and understandable. The way they measure your use should be standardize. It would help both the ISPs and the consumer. We need something like the FDAs nutrition labels but for technology.
Data cap? y/n
Limit = ###
Max speed = ##
Minimum speed = ##
Average Latency = ##
% time down in your town over the past 12 months: ##
Average time to resolution for customer outages: ##
Your ISP HAS all of this information already. It's all a mater of making it law that they have to give it to you before you sign a contract. Simple as that.
Re: (Score:3)
And make the pricing transparent. No more adding in fees, especially fees that a carrier cannot tell you how much they will be before selling the service, but magically, can calculate them after you have signed on the dotted line.
I am just waiting for a carrier to offer a 1c/month plan (plus fees and taxes), where the fees and taxes are somewhere between $50 and $100/month.
Re: (Score:2)
The problem was my IP packets go over PPPoE which then goes over ATM and then over ADSL. ATM uses 53 byte cells with 5 byte overhead. That's about 10% in overheads (esp if you include everything else).
Re: (Score:2)
I'd say you've got it exactly right.
They should do the same for health insurance - not dictate how much it costs, or what is or isn't covered, but rather regulate how the coverage is described. Perhaps define various "standard" levels of covered, give them names, and then allow insurance to use those names for their products so long as the products they meet the official description. But still allow other packages, so long as the covereages and costs are describe in compliance with the regs.
Re: (Score:1)
Re: (Score:2)
In the US it's per County, so that will be interesting!
Since internet traffic crosses state boundaries, the federal government has jurisdiction according to current interpretations of Article 1 Section 8 of the Constitution. These sorts of things are usually defined by NIST, and enforced by the appropriate regulator, such as the FCC, FTC, or Department of Commerce.
Re: (Score:2)
Yea, this makes it complicated. For example, a "gallon" is set by NIST, but it is a county official who goes to the gas pump and verifies that what it says is a gallon is actually a gallon, then puts a seal on the pump to certify it. Not sure of any US governing body that actually tests weights and measures.
Re: (Score:2)
Don't be silly [Jen], the internet doesn't weigh anything!
Re: (Score:1)
Telcos are thieves (Score:5, Insightful)
no other explanation is necessary. For the old folks here who used to have a landline phone service in the old days, do you remember all those mysterious little "charges" they tacked on your bill? Like $1.05 "User Service fee" and $0.87 "DCF Maintenance fee" or some crap like that? Well even the federal gov't realized they were just plain thieves and sued them, which they settled for a few dozen million dollars. And went right back to doing it again.
Also there was the dial-up modem scam the telcos used to pull... Dvorak's summary [pcmag.com]
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
no other explanation is necessary. For the old folks here who used to have a landline phone service in the old days, do you remember all those mysterious little "charges" they tacked on your bill? Like $1.05 "User Service fee" and $0.87 "DCF Maintenance fee" or some crap like that? Well even the federal gov't realized they were just plain thieves and sued them, which they settled for a few dozen million dollars. And went right back to doing it again.
Also there was the dial-up modem scam the telcos used to pull... Dvorak's summary [pcmag.com]
I guess I'm old? I still have a land line since no cellphone I've ever heard sounds anywhere close quality wise.
Re: (Score:1)
All my voicemails are on vinyl. It just sounds warmer.
Re: (Score:2)
Yes.
So... (Score:5, Interesting)
So this means that they can't legally do 'metered billing,' as the meter is known and proven to be inaccurate, right?
right?
anybody?
rrdtool & /proc/net/dev (Score:4, Interesting)
DATA1=`grep eth0 /proc/net/dev | sed -e 's/ /:/g' -e 's/:\+/:/g' | cut -d: -f 3,11` /path/rrd/eth.rrd N:${DATA1}
rrdtool update
?
Re: (Score:2)
So you have a direct ethernet connection to your ISP? Most people use cable, dsl, or some type of wireless across different kinds of head end equipment.
Re: (Score:1)
And why should bytes that don't even make it to my machine count towards my usage?
Because life's not always fair.
However, from TFA, it appears that a lot of the bytes that DO make it to your machine don't count towards your usage. I'm wondering if they're only tracking TCP....
Re: (Score:2)
And why should bytes that don't even make it to my machine count towards my usage?
I think the answer to that contains the phrase "because they can". ;-)
Re: (Score:3)
Well, it works just like cars. For every thousand cars that pass QC, one or two fail and need to get recycled into parts. So when you buy one of those cars, if it ends up not working, you just eat the cost and shrug and buy another one.
Right?
Re: (Score:2)
Fast and wrong, or slow and correct. (Score:5, Informative)
Building an incorrect bandwidth meter is easy. Incorrect meters will calculate your bandwidth like ( 'MTU size' * 'number of packets' = usage), which will over estimate usage by a large margin (30% off is common), since a large number of packets are much smaller then MTU, DNS replies for example. It is 'somewhat' more accurate to take ('average packet size' * 'number of packets') per user, since different usage will come up with a different avg_pkt size. Counting each packets size and keeping track of it is the most accurate, but also the most resource intensive therefore the least likely to occur in bulk by the ISP.
Another place that can cause a significant skew in total bits is where bandwidth is monitored. Most ISPs count traffic before the restriction of your slow connection, therefore packet loss and re-transmits get counted against you (if the ISP uses no, or a bad queuing discipline this can end up being a significant amount of bandwidth). Monitoring how much was downloaded is best done on the CPE, such as the cable modem or dsl modem, but that would lead to firmware hacks and such to lie to the provider.
Re: (Score:2)
Building an incorrect bandwidth meter is easy.
That's fine and dandy as long as they can guarantee that in case of an error they under-count rather than over-count.
Most ISPs count traffic before the restriction of your slow connection, therefore packet loss and re-transmits get counted against you
And that's the problem. A combination of "black box" meter and a claim that "measuring bandwidth exactly is hard" cannot be allowed. They should have to chose one out of two.
Re: (Score:2)
Counting bytes is just as easy as counting packets, so I don't know why you assume they can do one but not the other. When counting bytes there's no need to consider packet size at all.
Re: (Score:3)
http://www.google.com/search?q=interface+byte+counter+incorrect [google.com]
It's also amazing how often equipment gets that byte count wrong (I've seen it often with ISP head end equipment). Also there is no shortage of equipment that can overflow 32-bit byte counts faster than the ISP samples.
Also many interfaces are encapsulating the data (PPP-OE) and most ISPs do some mathematics so you are not charged for the overhead.
Re: (Score:2)
I get my internet by WiMax, and I'm pretty sure that over half of my metered usage is error packets and resends.
Re: (Score:2)
It's fairly common to use NetFlows or similar protocols to measure per-IP bandwidth at either the inner or outer edges of the network (depending on whether it's billed over all traffic or just external traffic to the ISP). NetFlows has a few issues where it under- or over-reports the traffic used by a network flow due to insufficient detail supplied by the protocol.
In most cases where an LNS is in use (such as with any PPPoX DSL connection), they can report down-to-the-byte accurate statistics, but often d
we recommend... (Score:2, Insightful)
when I've called my ISP to complain about low speed, they usually start out by telling me to go to a specific site to check my speed. (they do the same thing when they send out a tech)
Thanks, no. I'll go to a different site. Anywhere besides the one you just suggested to me. Using what they recommend is like the used car dealer recommending you get a second opinion from his brother Jim.
Re:we recommend... (Score:5, Funny)
Re: (Score:2)
When I'm experiencing slow internet, I often go to the speedtest site my ISP recommends just to get my downloads to speed up for 10 minutes (YMMV, depending on how your ISP's traffic shaping is configured).
Re: (Score:2)
It can still be handy to use a speed test attached directly to the provider's core network to determine where the problem is - my home ISP has a few large files on a customer website you can download, and their customer support will get you to try both an internal and external test site.
If the internal test is slow, then there's a problem on the backhaul between you and the ISP. If it works but external sites are slow, there's either congestion on the edge (usually pretty obvious on a traceroute, and your
Re: (Score:2)
"you have a brain-cloud. you only have a few months to live"
something like that?
yeah, never trust the 'go to my shop and get an estimate'.
If this were Electricity, Water or Gas meters... (Score:2)
Re: (Score:1)
If 5 out of 7 Electricity, Water or Gas meters were inaccurate, you can bet people would be screaming and government would be cracking down.
Also, imagine if the cost (to the provider) of providing Water at near-capacity was roughly the same as the cost of not providing any water, because the bulk of the expense is in maintaining the pipes... Imagine the outrage against the greedy provider who strictly limits the amount of water anyway.
Re: (Score:2)
Re: (Score:1)
Do you think anyone would know? How many people have real clue how to measure how many kw of electricty they use, or how many gallons of water flow through their house each month?
I have no idea how many gallons of water I actually use, my bill could be off by a factor of 10 and I wouldn't be able to tell.
And truthfully I wouldn't really care, what I care about is if my bill is in line with others that appear to use the same as me.
They can call the units anything they want to call them as long as I can get a
Re: (Score:2)
Water, Gas and Power meters are required to be required to be accurate by federal law. These laws included the establishment of agencies like Underwrites Labratory to certify power meters.
You know all those laws and regulations your parents enacted to prevent people from cheating and keep the system as fair as possible, that certain people today think should go away because all government regulation is bad. Where the reality is they are trying to propagandize people who aren't smart enough to realize what t
Not new (Score:1)
It's long been industry practice to count bandwidth as # of Packets * MTU.
It's always been bullshit, and it always will be, since the government nicely handed over practical monopolies to them
Re: (Score:2)
32-bit byte counters and long sample times can be an issue(you can count 500/1000/1500 times the number of bytes as packets without overflowing the counter) But most of it is legacy, packet counts were the way it was done on a lot of equipment even though it's not the most correct method.
Re: (Score:2)
Your ISP doesn't have a dedicated interface just for you. And Bob who's on EFM. And Jill on VSAT. And Jim's Mowing with its 2 aggregated DSL lines.
Most ISPs I've seen collect flow statistics from routers, which works no matter what tail is in use and collects many customers' statistics at once.
Related issue: How much of that usage is YOU ... (Score:4, Insightful)
... and how much is ads?
Moving to a new client's site gave me a taste of using a browser without noscript and flashblock. I discovered a number of sites are displaying multiple ads that consist of flash movies.
To view a few paragraphs of text (a couple kilobytes or so) I USED to be downloading perhaps a quarter megabyte of graphic imagery. Now I'm downloading perhaps a minute of video for each of several self-starting video ads.
Not to mention popovers-on-mouseover - including some that that darken the whole page rather than just obscuring part of it - and if I want to kill them without "pushing a 'close' button" supplied by the popover ("Push me! Push me! I'll just close the window and not download malware! I promise!) I have to reload the page all over again. Listen to that meter whir!
What bytes are we measuring? (Score:5, Interesting)
I don't know how many on slashdot know networking, but there are different ways of measuring bandwidth. Are we measuring layer 2, or layer 3? Further, layer 2 can often have multiple encapsulations before even taking layer 3 into consideration. Take for example DSL which frequently uses PPPoE, which means we have both PPP and Ethernet frames in addition to the IP data and everything encapsulated therein. And if you include DSL interleaving, then do we also include the packets that had a bad checksum and were therefore discarded? (in many cases there are a lot of these) That *is* data usage by all definitions. Do we also include ingress packets that were dropped due to bad checksums? Again, that is data usage.
In my opinion, the problem is that there aren't any standards defined for measuring bandwidth. Also in my opinion, that definition should be layer 3 traffic only and nothing else.
Re: (Score:2)
Also something to add to what I said, even if you do only layer 3, you still won't have 100% accuracy. Some egress packets will get dropped long after leaving the ISP's routers, and it's impossible for any bandwidth meter anywhere to be able to tell what didn't make it, mainly because by design IP can't (and shouldn't) provide the facilities for doing so.
Re: (Score:2)
That's what I'm getting that. If you metered data, you can't know whether or not it made it at its destination.
And yes, you can get horrible packet loss without the ISP being involved at all. In fact, that is currently happening at the peering level with AT&T and Cogent. They've had a few saturated links at their border routers somewhere near Illinois for several months now and haven't done a damn thing about it. It is mainly effecting several west coast ISP's whose users get horrible latency with blizz
Re: (Score:2)
Re: (Score:2)
Why would that be asking for somebody to develop a protocol that does that? The end result would be lower data rates for the end user. I mean what, they would do it just for the purpose of annoying the ISP? Besides, if that was their goal, they could easily do that using UDP with error correction at the application layer, no need to develop a new protocol.
And when I speak of checksums, I mean at the CPE. For example if somebody is behind a NAT, your outside interface will trash the packet before the host co
Re: (Score:2)
Indeed there are lots of variables needed to define what "how many bits were sent in this time period" means. And even more to define "offered bandwidth".
This is classic work for standards committees. I'd bet there is a standard for calculating "net weight" on cans of olives. I haven't heard of relevant standards for data. Maybe T1 could be convinced to address the need.
Re: (Score:2)
layer 3 is routing layer.
that includes routing updates (ospf, etc etc).
you want tcp and udp byte counters, riiiight?
Re: (Score:2)
End users won't be sending dynamic routing protocols to the ISP. If they did, I'd be a bit concerned if the ISP didn't filter them out as that's just asking for trouble. End users should only be using static routes, and consequently static routes are the only form of routing that most consumer grade equipment supports.
And yes, TCP and UDP overhead should count towards that, as should *any* layer 4 encapsulation. The ISP should only be doing layer 3, when you start going into the transport and session layers
Goodput should be measured (Score:2)
In other words, the amount of data layer 4 can send/transmit in a given unit of time.
Layer 3 is nearly as close, but it may include more data that's not inherently "usable" but necessary (TCP overhead, ICMP packets, etc).
Layer 2, with its many levels of encapsulation on the ISP side is less valuable because it involves so much overhead.
They're UNDER-reporting! (Score:5, Informative)
Re: (Score:2)
I guess it really doesn't take any facts for the idiots to start clamoring about how all business' are evil.
This.
1,000x This. I suppose RTFA is a lost art...
Re: (Score:2)
I guess it really doesn't take any facts for the idiots to start clamoring about how all business' are evil.
Dunno about that. But since most ISPs have a gross margin of 90% [wsj.com] on their broadband traffic, they can under-report by 2x and still be vastly over-charging.
Re: (Score:2)
Gross profits also doesn't include the losses from other parts of the company. Those aren't relevant when talking about bandwidth caps.
Re: (Score:3)
I guess it really doesn't take any facts for the idiots to start clamoring about how all business' are evil.
Yeah, when we know that in reality, only about 90% of them are evil. And those 90% are giving the other 10% a bad name.
(Dunno if I need a ;-) here or not ...)
How about some honesty in bandwidth numbers? (Score:2)
How about some honesty in bandwidth numbers?
Comcast is what I'm thinking of specifically -- they provision your *modem* to talk to the local head end at ridiculous bandwidth numbers but in my experience, once you go over about 25 Mbps in most areas you never see it, even if you are provisioned at 50 or even 100 Mbps.
I've been caught in the middle with customers who have equipment provisioned at the 100 Mbps level before who see nowhere near this, even on Comcast's own cheesy bandwidth meter. The customers
Re: (Score:1)
Sounds good to me- I prefer porn with more 0's and fewer 1's.
Re: (Score:1)
That's called "ebony."
Re:We need broadband meter readers (Score:4, Informative)
I'm sure these meters make mistakes both ways right? Occasionally under counting.
From the article:
"They are wrong by missing numbers by one way or another - sometimes it's over reporting, but more frequently the error is under reporting," he said. Under reporting should be a relief to those facing overage charges or service termination for going over their meters, but if the meters aren't counting the data properly, it is still a problem.
Re: (Score:2)
Re: (Score:3)
Someone who thinks it should be at -1 even though it's not troll, offtopic, redundant, or flamebait?
For example, wrong information, or a superbly silly statement/question that still manages to apply to the topic. Not that this applies here, I'm just answering your question.