Duke Research Experiment Disrupts Internet Traffic 80
alphadogg writes with this excerpt from Network World about an experiment gone wrong which affected a big chunk of internet traffic yesterday morning: "It was kicked off when RIPE NCC (Reseaux IP Europeens Network Coordination Centre) and Duke ran an experiment that involved the Border Gateway Protocol (BGP) — used by routers to know where to send their traffic on the Internet. RIPE started announcing BGP routes that were configured a little differently from normal because they used an experimental data format. RIPE's data was soon passed from router to router on the Internet, and within minutes it became clear that this was causing problems. ... [f]or a brief period Friday morning, about 1 percent of all the Internet's traffic was affected by the snafu, as routers could not properly process the BGP routes they were being sent."
Wow (Score:3, Interesting)
A big chunk? (Score:3, Interesting)
1% isn't big in my book.
I would have liked to see what would happen if they kept going with this.
Hmm... (Score:3, Interesting)
Re:A big chunk? (Score:3, Interesting)
I seem to recall that CERN produces about 1% of all data that goes through the internet every day. Hmm...
Not surprising (Score:2, Interesting)
I can't believe we don't see more of this, considering the trust-based nature of BGP. I'm not saying that's a bad thing, I'm just wondering out loud why this is so unusual.
Re:Hmm... (Score:3, Interesting)
BGP is, like all routing protocols, very secure in and of itself. The difficulty is that a router peering with all routers on the internet can "inject" bad routes, and the "mail" gets reliably delivered to a wrong address. This is ONLY a difficulty if you can somehow gain access to a router that is directly connected to a backbone, and has peering status. You will have to have your own Autonomous System number also, although I am sure you could fake that.
The only time that I have seen even isolated internet routing issues is due to mis-configuration of the router by the owner. Well, that and the extremely rare (yes, really, it is rare) OWNing of an edge router.
I am not all- knowing on this subject; far from it. If someone has something to add/update/correct, please do.
Re:Wow (Score:5, Interesting)
Maybe, yes [wired.com]. BGP has been identified as vulnerable for a long time, and this is further proof. On the other hand, this research is probably motivated by fixing the problem. But the Internet is no longer something you can just shut down or reboot to upgrade; you must operate on a live patient. It does make you wonder, though, if well-intentioned people can do this trying to help, what somebody malicious could do. Hopefully governments will decline to use this as a weapon - like poisoning the ocean.
Re:Hmm... (Score:5, Interesting)
Any ISP network engineer has some good BGP stories.
For me I was I fighting for over a year to get some of MY blocks back from another provider. They simply continued to announce the routes for them and made it uttererly worthless. It was also fairly horrible to get any upstream traction against the offender.
Eventually, we simply started announcing the routes for those blocks and caused turmoil for those who were using them. It didn't take long to get that issue cleaned up afterwards. Though it was funny because they had asked my guys to stop announcing.
BGP is a bit of a trust relationship, but it isn't the end of world when everything goes to shit.
Admins will get up for their beds and start clearing issues. Things will be sluggish for a bit, but eventually things work out.
Re:Not surprising (Score:3, Interesting)
You don't see more of it because you don't get told, and you don't look.
Then there are the snit fits [pcmag.com] peers have to indulge in.
And the occasional stupidity [zdnet.com].
Go check out the Internet Taffic Report [internettr...report.com] from time to time. Today it looks like there was significant event. Wonder what happened.....?
Now don't get me started on PMTUD. How do I explain to a user that it is not 'our' network that is the cause, we have MILLIONS of users working just fine, but everyone in their office can't get on because we broke something just to annoy them? And of course, since they can see the same error a different, unrelated site, it MUST BE US. Yeah. I'm the designated PMTUD expert on the team now, because I let their ISP talk itself into the solution. And I can read packet captures. Yay me, think I'm going off decaf for a few days...
The Internet is not perfect.
Re:Wow (Score:4, Interesting)
I wouldn't say countless. There have probably only been less than 10 blackhole type events with BGP/routing that affected a significant amount of Internet traffic in the past 15 years. The big one being back in 1997. There is a website somewhere that keeps track of them and explains what happened.
Re:Wow (Score:4, Interesting)
But the Internet is no longer something you can just shut down or reboot to upgrade; you must operate on a live patient.
That's a really important point that often goes undiscussed - it's been suggested that if the Internet did go down (major solar storm, EMP, etc.) that it's not likely that it, or the interconnected systems (electrical grid, etc.) could come back up. Too many race conditions, mostly unknown/undocumented. Sure, eventually it would all get back on track, but it could be weeks-to-months. I'm planning to hike the Appalachian Trail while it gets straightened out. ;)
Hopefully governments will decline to use this as a weapon - like poisoning the ocean.
That sounds like a major societal vulnerability that needs to be patched. Nuclear weapons marked an important turning point in history where governments became too dangerous to keep around.
Re:Hmm... (Score:5, Interesting)
Fake it? Not in the last five years!
unless you know of some BGP peers that refuse the standard peering protocol, 1) they are required to only listen to routes from known surrounding peers, 2) will not be listening to what's being advertised by your router unless you have instructed them ahead of time what AS you manage and what prefixes you will be advertising to them.
No. Period, fucking no. Most BGP sessions run between customers and carriers are still basically allowing whatever. Even the big boys basically don't care what you advertise. It would cause too many problems to go and begin filtering, so only regions that seem to have routing DBs (RIPE region) are even remotely participating in this. For the most part, thats a few places here and there, but the carriers will let you do what you want.
Don't believe the hype: BGP is still as weak in public IP as it ever has been. The difference is that if you do decide to hijack someone else's prefixes (don't even include bogons, because the carriers will probably let you advertise those!), everyone will know and you will get your upstream looking at you.
Re:Hmm... (Score:2, Interesting)
My networking class instructor in college had a doozy of a story.
He and a coworker were working for a company, and while they weren't supposed to have the passwords for the BGP routers and whatnot, they did as a matter of expediency. (you know, someone not wanting to walk over somewhere to just enter a code, etc). Anyways, the coworker executed a hard re-computation of the BGP routes rather than a soft one, bringing the entire company's network down for about a half hour until everything was recomputed. The only reason they escaped with their jobs was because they weren't supposed to have the passwords, and thus not suspected.
That Could Explain the Steam Spike... (Score:4, Interesting)
Re:Wow (Score:2, Interesting)