Google Caught in Comcast Traffic Filtering? 385
marcan writes "Comcast users are reporting 'connection reset' errors while loading Google. The problem seems to have been coming and going over the past few days, and often disappears only to return a few minutes later. Apparently the problem only affects some of Google's IPs and services. Analysis of the PCAP packet dumps reveals several injected fake RSTs, which are very similar to the ones seen coming from the Great Firewall of China [PDF]. Did Google somehow get caught up in one of Comcast's blacklists, or are the heuristics flagging Google as a file-sharer due to the heavy traffic?"
Not me... (Score:3, Informative)
Gmail Notifier (Score:5, Informative)
Comcast is really pissing me off. But what's my other option: Qwest DSL.
Re:It could be technical incompetence (Score:3, Informative)
Re:Get the facts (Score:2, Informative)
Wow, -1 Troll? Do people even think before moderating? For those who aren't subtle enough to get it on their own, the parent post is being sarcastic.
Edit : ha, nevermind, someone had the common sense to mod it Funny.
Edit #2 : Oh yeah, didn't you know? Now you can edit your posts on Slashdot.
Re:First hand experience here (Score:5, Informative)
Did you actually flush your DNS caches like, say, the one in your router, the one in your linksys box, the one on your PC? You can do it manually but the quickest way for a lot of equipment is to reboot. Hence the suggestion.
Additionally, it was quite likely google because something on your machine (maybe yourself "trying" the connection) had accessed google while the DNS redirection was in place (that was how they "redirected" you to their page). Once you'd done it once it'd linger until the TTL's had expired all the way back to your computer. Ping, nslookup, etc. would ALL show the Comcast IP until that happened, which could be minutes, hours, days, months, depending on your setup.
In your case, it looks like it was less than 24-hours, because it worked the next day without having to reboot. If you had rebooted immediately, it would have all worked when it came back up. That's WHY he was telling you that.
Before you start throwing accusations around, delve into such things just a little bit deeper.
Re:Can we please pay attention to the dates... (Score:3, Informative)
Not comcast (Score:3, Informative)
Your OWN COMPUTER was redirecting you to Comcast (maybe you should be indignant towards Microsoft? >_>). It's called DNS caching.
In Windows a simple ipconfig /flushdns can take care of that, although some applications, such as Firefox, keep their own DNS caches which must also be cleared (In Firefox there's a DNS cache timeout in about:config somewhere, you just set it to 0 and then back and that should flush the cache).
Also the tech was almost right... restarting your computer WOULD have fixed it (since DNS caches are only kept in memory and would have been wiped when you rebooted) although it wouldn't have been the OPTIMAL solution.
Let me take you through the steps your computer took.
Wikipedia page (Score:5, Informative)
The way it's written now, everyone should use Sandvine - it sounds like wonderful software.
Re:Not me... (Score:5, Informative)
It's not that they can't figure it out, it's that they aren't even bothering to try and shape traffic. They'd rather interfere with it.
Back in my ISP days we ran our entire operation (400 dial-in lines and about 60 WISP clients) off two un-bonded T-1s (they went to different POPs for redundancy). We couldn't afford to add more bandwidth at the edge, so I hacked together a traffic shaping setup using Linux. It prioritized ssh, telnet, TCP ACKs, icmp packets, and the VPNs of our business clients. VoIP wasn't a big concern in those days but had it been I would have prioritized it as well. When online gaming started becoming big we started giving that traffic priority over bulk transfers as well.
The bulk downloaders/p2p'ers didn't notice or complain. They still got the lions share of the bandwidth -- and are you really going to notice if your transfer gets 139KB/s instead of 140KB/s due to that ssh packet moving ahead of you in the queue? During peak hours my T-1s were running at 90-95% of capacity but my users were all still humming along quite nicely, none the wiser. There was more to this then just traffic shaping (we also had a pretty slick squid setup), but the point is we got along just fine with our limited resources.
If we could fucking do it, then sure as hell Comcast could. They have apparently decided that it's better to block/drop the traffic then shape it. If they had real competition they'd probably pay for this over the long run.
Re:Get the facts (Score:1, Informative)
Troll: Ex. "Windows(or whatever) Sucks." Ideas that do not bring any useful information to the topic and usually just there to abuse the people. Lately it is has been use as I disagree with your point of view or you are missing some what the moderator considers correct facts.
Flaimbate: Ex. "Windows is the Best OS out there" Simular to a Troll but usaually used to spark emotional responce from the users, with little descussion of the actual topic... Latly moderators like to use this for contaversal topics that they disagree with the premis. The difference is one is based on some logical reasoning while the other is just to spark emotional fighting.
Redundant: Ex. "Just as everyone else said this
Offtopic: This post... Post that tangant to far off the topic or are compleatly unreleated to the topic. Often used now for posts with 2 or more paragrahs and/or relate to the topic by analogy.
Overrated: For cases where they are rated a 4 or 5 but really not worth that spot... Thay are now used as a loop hole around metamoderation. For I disagree with your topic.
There is less abuse in the positive moderation, except for the fact the earlier posts get moderated faster then older ones and there is little double checking to see if a moderated down post is really woth that.
Underrated: For those posts that deserve some notice but doesn't fit any of the other moderation. This is not used too often but often it is used for low moderation such as trolls or flaimbates that are not worth the -1 penality but still is a troll or flaimbate but it is minor and deserves a 0 or 1
Funny: It made you chuckle so it is moderated funny. One Mans funny is often an other mans Troll.
Interesting: It made you stop and read it fully to understand what the heck they are talking about. They brought up a lot of good points. Used now as I agree.
Insightful: Put new perspective on things you, a view that is new, and worth consideration was brought up... Now it is used for Yea that was what I was thinking...
I think there should be a flag where the user can debate their moderation if they disagree with it and give it priority in meta moderation. While on the whole it doesn't mean much, but still people like to get credit where credit is due.
Slashdot requires you to wait between each successful posting of a comment to allow everyone a fair chance at posting a comment.
It's been 22 minutes since you last successfully posted a comment you guys need to fix that.
Re:Not me... (Score:5, Informative)
Furthermore, the problem is very likely far more simple and less sophisticated than this issue of packet spoofing.
Set up a continuous ping to something "nearby" (your gateway, your DNS ser ver, your neighbor, whatever) in your Comcast network and tee it to a file. Leave it up for days and you'll likely see periods of time where you have no service for patches of time... often long enough to kill sessions.
I very often have problems with any sort of sessions (SSH, VPN, etc.) staying up for long periods of time because the underlying line level reliability is so poor. I can watch my cable modem logs and see many resets, timeouts, etc.
I laugh whenever asked about phone service via Comcast. Sadly, however, this pathetic reliability also precludes Vonage and the like. And I find this a bit sad since while I do not consider Comcast capable of running a world class network, I loathe the phone company. Those guys are more competent but much more directly evil.
Re:Not me... (Score:3, Informative)
Re:Get the facts (Score:3, Informative)
You wouldn't happen to be one of the people I talked about attempting to dispel knowledge of this are you? There we go, the tinfoil hat is back in place and everythign feels right again.
Either look around or keep your eyes shut. It doesn't matter much to me. But I call them as I see them. I haven't been wrong often.
Re:Not me... (Score:4, Informative)
If we look at what is promised, what is purchased, what is possible, and compare that to what is experienced, it is clear that some ISPs suck, and there is a reason that they suck. Suckiness is not 'normal' or 'average' or acceptable. With the FCC ruling to allow multiple ISP connectivity to many homes, the quality of service should improve to prevent customer churn. My advice is to switch if complaints are not resolved if you can. If not, register a complaint with the authority who gave your ISP broadband monopoly in your area. Document the complaint process and responses. The BBB, I believe, can be consulted in cases where they clearly are not giving you what you paid for.
From the guy in the second link (Score:3, Informative)
Here's the condensed version:
* Pings work fine, other websites work fine - only HTTP to google.com with a "google.com" host header is affected
* HTTP 1.0 without host header isn't affected
* Going to one of google's web servers by IP works fine (no "google.com" host header)
* I am typically seeding torrents and was at the time of each service interruption
* TCP RSTs follow a specific pattern. 2 RSTs in rapid succession in response to the initial GET statement (1 with a valid SEQ, one with a SEQ in the 12xxx range), followed by a second batch of the same. As the article here states (and as I posted in the linked thread), this matches perfectly with results from the China firewall
* The problem went away at almost exactly 12:00am EDT this morning (give or take a minute)
* This is from a Comcast subscriber in Grand Rapids, MI.
For more detail, visit the thread linked. I have links to the raw packet capture data in
Three reasons that makes NO sense... (Score:1, Informative)
B) Why only Comcast? They'd have to make a list of all Comcast IPs, then figure out what path the packet took so that they could forge an RST from an appropriate router. Oh, and the TTL would probably be off. They also don't have very long to do this on each connection, and they never know when the routes will shift underneath them.
C) What's Google's motive? To get caught and make Comcast look good? You have to be stupid not to think this will get caught and analyzed. If there's stupid, I'm betting it's at Comcast more than Google. Especially given that Comcast HAS a clear motive: to stop P2P from clogging their shared and overtaxed cable bandwidth.
Comcast & DNS (Score:3, Informative)
Using tcpdump showed that all the bad dns queries stopped after 4 frames, while the successful ones went 68 or 70 frames.
Switching from Comcast's regional DNS servers to their national DNS servers fixed the problem immediately.
Makes me wonder what they're doing on the regional ones.
Re:First hand experience here (Score:3, Informative)
I believe that 4.2.2.1 - 4.2.2.5 (or maybe 6) are all DNS servers for Level3, in case you want multiples available.
Re:iptables fake RST detector (Score:1, Informative)
It's already been discussed over on broadbandreports that the Sandvine devices *are not* being used as actual routers (read: packets do not "flow through them"), but rather sit on a switch port with mirroring enabled. The Sandvine does signature analysis on the packets and when it declares a match, along with numerous other rules (e.g. inject RST every X number of packets), sends two falsified packets out a separate network port connected to the router that *does* have packets flowing through it. The router configuration prioritises those injected packets to be processed first, so that there's no race condition.
Otherwise, if what you said is true, the Sandvine is a bottleneck in every piece of Comcast's network in every geographic region -- and this is highly unlikely.
The sequence numbers are indeed properly forged by the Sandvine, because the Sandvine appears to only inject a TCP RST *after* a preceding legitimate packet (with sequence number included, obviously) was sent. Thus, the Sandvine does sequence number state tracking along with overall TCP state tracking.