Forgot your password?
typodupeerror
Networking The Internet Technology

Chinese ISP Hijacks the Internet (Again) 171

Posted by Soulskill
from the phase-two-test-complete dept.
CWmike writes "For the second time in two weeks, bad networking information spreading from China has disrupted the Internet. On Thursday morning, bad routing data from a small Chinese ISP called IDC China Telecommunication was re-transmitted by China's state-owned China Telecommunications, and then spread around the Internet, affecting Internet service providers such as AT&T, Level3, Deutsche Telekom, Qwest Communications, and Telefonica. 'There are a large number of ISPs who accepted these routes all over the world,' said Martin A. Brown, technical lead at Internet monitoring firm Renesys. Brown said the incident started just before 10 am Eastern and lasted about 20 minutes. During that time the Chinese ISP transmitted bad routing information for between 32,000 and 37,000 networks, redirecting them to IDC instead of their rightful owners. These networks included about 8,000 US networks, including those operated by Dell, CNN, Starbucks, and Apple. More than 8,500 Chinese networks, 1,100 in Australia, and 230 owned by France Telecom were also affected."
This discussion has been archived. No new comments can be posted.

Chinese ISP Hijacks the Internet (Again)

Comments Filter:
  • Accident (Score:5, Insightful)

    by rmushkatblat (1690080) on Friday April 09, 2010 @03:41PM (#31794568)
    It was an accident, of course.
  • by Michael Kristopeit (1751814) on Friday April 09, 2010 @03:41PM (#31794572)
    now you can order iPad direct from china through apple.com
    • Re: (Score:3, Interesting)

      by jc42 (318812)

      now you can order iPad direct from china through apple.com

      Nothing new here. When I ordered this Macbook Pro last year, I was able to follow online its progress from the warehouse in Shanghai to my porch. Apple is now effectively a delivery and customer-support service for Asian manufacturers.

      Maybe eventually they will cut out the middleman, as IBM did a while ago with its Thinkpad laptops. Now you order them directly from Lenovo, which is a Chinese firm. The pretense that they were an IBM product has en

      • by jon3k (691256)
        Without Apple the Chinese manufacturers would have nothing. Apple's business strategy, marketing, design and software development - the "magic" that actually separates Apple from every other tech company, none of this exists without Apple.

        You're basically saying that if I put together a jigsaw puzzle I invented it.
  • Not unintentional (Score:5, Interesting)

    by Nickodeemus (1067376) on Friday April 09, 2010 @03:42PM (#31794598)
    All that data routed to the wrong place accidentally... hmmm sounds like a perfect excuse to me - for intelligence gathering. If it passes through their routers, they have the data.
    • by Ruede (824831)
      i thought the same thing. information gathering... maybe some passwords....
      • Re: (Score:2, Informative)

        by robmv (855035)

        and add to that a Chinese CA certificate inside Firefox and even SSL could be sniffed

      • If the networks that your traffic is being routed to doesn't simply melt sure.

        This has happened before quite a few times, it's a side of the internet which is surprisingly fragile.

    • by billstewart (78916) on Friday April 09, 2010 @04:08PM (#31795040) Journal

      Limited-scope attacks like the Pakistani YouTube diversion are much more likely to be a deliberate attack; broad-spectrum attacks are obviously either mistakes (or really clever DDOS.) Advertising that you're the best route to half the world isn't exactly un-stealthy enough for intelligence gathering - and China doesn't have the bandwidth to handle that much traffic, either inside their entire country's network or especially across the Pacific; the only carriers with a chance of absorbing some fraction of AT&T's plus Level3's traffic are Verizon or possibly Google, and they're both competent enough not to do that.

      This kind of thing happens occasionally with BGP, which was designed to be run in a relatively trusted environment by relatively-to-extremely-competent people, which means that it only explodes occasionally and most major carriers do a good job of filtering routing announcements that look seriously wrong, and detecting when other people advertise bogus information about their networks. The typical cause used to be bad conversions between external BGP routes and internal OSPF or RIP routes, especially back when some random customer would have left autosummarization on so they'd take their two Class C subnets, combine them into the Class A that they're both in, and announce to everybody in the world that they were the best route to reach the Tier 1 carrier who's their upstream (or who's the upstream of their local ISP, who wasn't bothering to filter their BGP announcements.)

      The first time this happened in a big way was a bit of a surprise, as some little ISP announced that their T1 line was the best way to reach all of MAE-EAST (i.e. half the world), so suddenly there were gigabits of traffic headed that direction, at least until their self-DDOS killed off most of the BGP sessions and somebody fixed it. Since then, if you try to advertise being the best route to some large carrier who has a /8, you'll find they're also advertising a pair of /9s (which win), and that they'll be calling your upstream carrier within a couple of minutes to get your BGP session shut down. On the other hand, if this happens, it also means your upstream carrier wasn't filtering your BGP announcements for sanity, so they may also not be good at having somebody who can answer the phone and quickly resolve that level of problem.

      • they'll be calling your upstream carrier within a couple of minutes to get your BGP session shut down

        I wonder what would happen if there were no voice circuits anymore and everybody used VOIP? Would network operators use dedicated radio circuits to coordinate operations? I have this vision of them pulling up their own 80 metre antennas to ensure voice communication or maybe RTTY.

    • Re: (Score:3, Interesting)

      by TreyGeek (1391679)
      Sounds a lot like "Stealthy IP Prefix Hijacking" [sigcomm.org]. Advertise a BGP route that will be accepted by some people to attract their traffic. Do it correctly, it may be less noticeable than a full prefix hijacking (though it was obviously noticed in this case). You can also attempt to moderate the amount of traffic you receive so that you don't DOS yourself with the incoming flow and you can analyze the traffic easier. BGP is a pretty insecure protocol and depends a lot upon the upstream providers filtering an
    • Intelligence gathering, or just general probing of ability to control the Internet (if only for a somewhat short period of time - and how much time do you need, really?)
    • by Anonymous Coward

      nuff said. Ok, I will ellaborate, but that shouldnt be neccecary. Do you really need to read more?

      This may be a cyberwar between a multinational corporation and China. Google will of course win this war. The war is secret, and not fought with bullets. Oh, you want to know even more? That is hardly neccecary, but I will go on.

      Also, we will need to equip an army of female acrobatic tech-warriors wearing tight-fitted latex with large open cleavages. That can probably keep the kung-fu chinese hackers at bay. No

    • "Once is happenstance, twice is coincidence, three times is enemy action." - Ian Fleming

      The next time one of these stories comes around, then you can jump to conclusion. Right now, well....

  • Blacklist 'em (Score:5, Interesting)

    by DogDude (805747) on Friday April 09, 2010 @03:43PM (#31794604) Homepage
    Until China learns how to act as responsible Internet citizens, I'll continue to blackhole as many of Chinese subnets as I can find both at work and home. Spam, malware, and every kind of crap comes from China, and I don't do business with any Chinese, so it's a no-brainer.
    • Re:Blacklist 'em (Score:5, Informative)

      by pv2b (231846) on Friday April 09, 2010 @03:45PM (#31794648)

      Blacklisting China's IP ranges would do nothing to protect you against bad routing - something you as an end user don't have any control over.

      • I second the AC above: If someone has a link for all Chinese Internet-routable subnets in order to drop, that'd be cool.

        No, it won't protect against malicious fake routes, but it protects against attacks/scans/connections from legitimately Chinese networks.

      • Re: (Score:3, Interesting)

        by beadfulthings (975812)

        Of course, you are right about the routing. But since giving in to my baser impulses and blacklisting the entire country on my one humble web server, I've had a remarkable decrease in my annoyance factor in terms of crap like port scans, login attempts, comment spam in the blogs, and even a respite from the damned Baidu spiders who won't observe anybody's robots.txt file. Along about the fall of last year, I began observing what looked like attempts at ddos attacks--all originating from China. None of them

        • Re: (Score:2, Informative)

          by pv2b (231846)

          Baidu's real spiders obey robots.txt. However there are plenty of malicious spiders out there who pretend to be Baidu in their User-agent string - giving Baidu a bad name in this area.

          • Sorry, but if Baidu wants us to believe that their spiders behave lawfully, they should arrange to receive lawful communications regarding them. As for me, I'm enjoying the respite--since the spiders stopped when I terminated the communication.

    • Re:Blacklist 'em (Score:5, Interesting)

      by PNutts (199112) on Friday April 09, 2010 @03:54PM (#31794806)

      Until China learns how to act as responsible Internet citizens, I'll continue to blackhole as many of Chinese subnets as I can find both at work and home. Spam, malware, and every kind of crap comes from China, and I don't do business with any Chinese, so it's a no-brainer

      Well, since more SPAM comes from the US I assume you'll block those subnets too? http://www.spamhaus.org/statistics/countries.lasso [spamhaus.org]

      Also, in March the US was the source of most malware, but since you already have that blocked for SPAM you should also block Korea who for some reason in the month of April took the lead. http://www.infosecurity-us.com/view/8547/korea-reigns-as-king-of-malware-threats-/ [infosecurity-us.com]

      In regard to China learning how to act as responsible Internet citizens, you are not leading by example.

      • Re: (Score:2, Insightful)

        by Kenz0r (900338)
        I get thousands of SSH log in attempts coming from China every day. I also get some from South-Korea, some from Brazil, but none from the US.
        Spam is not the only factor to consider.
        It certainly DOES make sense to blacklist China in its entirety unless you're doing business with them.
      • by jon3k (691256)
        I do business with some Americans that might come from those netblocks, so I can't block them. I don't do any business with anyone in China (or a statistically negligible amount, anyway) so I can block those networks.
    • by DarkOx (621550)

      yes than your traffic can get router there anyway when the start advertising American, and European subnets.

      • Re: (Score:2, Funny)

        by Anonymous Coward
        English, motherfucker. Do you speak it?
    • Good walls work both ways. To "help" China from being tainted by the evil ways of us westerners let's just cut them off completely.

    • While at it, I offer you to query my own Zebra server, I guarantee to only return the best available routes ;-))

      http://www.gnu.org/software/zebra/ [gnu.org]

      Contact me off-line if you are interested.

      Seriously, I have some friends who do like you, they start by blocking China, then Korea, then end up blocking half of the world to enhance their security.

      In my humble opinion, this is not a valid security approach, I actually use some requests or connection attempts from these countries to test and strengthen my security.

      • by jon3k (691256)
        100% of my thousands of failed SSH attempts come from (Chinese) APNIC address space. I will humbly disagree with your conclusion that blocking the source of all attacks doesn't increase security. Like anything it's one layer of defense. When they start relaying it off American hosts then we'll come up with a Plan B, which will most likely be cutting off the relays, since ARIN and US companies within US jurisdiction are a little easier to work with :)
        • by ls671 (1122017) *

          > 100% of my thousands of failed SSH attempts come from
          > (Chinese) APNIC address space

          I call bs on this and I have logs and automated complaint reports to prove it. Also, I have other additional means to deal with this issue.

          There are compromised machines on every network, most of these attempts are done by botnets without the knowledge of the IP owner as I found exchanging with remote network admins. ! ;-))

          Here is one report, I edited out my own IP for obvious reasons.

          Date: Tue, 13 Apr 2010 15:27:10

  • An old saying... (Score:4, Insightful)

    by marmoset (3738) on Friday April 09, 2010 @03:44PM (#31794614) Homepage Journal

    "Once is an Accident, twice is a Coincidence, and three times is a Pattern."

  • by Archangel Michael (180766) on Friday April 09, 2010 @03:45PM (#31794650) Journal

    Any sufficient level of Incompetence is indistinguishable from Malice.

    Solution however is exactly the same.

    • by Nerdfest (867930)
      I think they're hoping that the people don't notice that the opposite can be true as well.
  • by Turzyx (1462339) on Friday April 09, 2010 @03:47PM (#31794678)
    The ISP in question only controls 30 networks, yet other routers blindly accepted thousands. Why isn't there basic verification of such re-configurations? I'm actually very shocked, the potential for abuse is huge; and TWICE as well.
  • built to spill (Score:1, Insightful)

    by Anonymous Coward

    ... faulty by design.

  • Fall guy (Score:4, Interesting)

    by Manip (656104) on Friday April 09, 2010 @03:50PM (#31794732)

    Why can one "small" ISP do this? I mean from a technical point of view how can they spread routing information for endpoints their network doesn't own? While they have clearly dropped the ball, I struggle to understand how they could accomplish this even if they tried, that is if everyone else's equipment is configured correctly *cough*

    • Re:Fall guy (Score:5, Informative)

      by Paralizer (792155) on Friday April 09, 2010 @03:55PM (#31794826) Homepage
      The internet runs the BGP routing protocol. It is by design a 'trust' system. You explicitly neighbor with autonomous systems you want to directly connect to and you freely exchange routes. It's possible to filter that routing information if you wanted (both in and out), but because you explicitly connected with them there's a certain level of "I trust anything you tell me, as I you should of me."
      • IP V6 everywhere
        static herarchical routing everywhere based on geographical IP addresses prefixex.
        like in the old telecom way.

        • by billstewart (78916) on Friday April 09, 2010 @04:29PM (#31795336) Journal

          By "old-school principles", you did mean "pre-ARIN IPv4 Swamp Addresses", didn't you? :-)

          Yeah, the people who designed IPv6 hoped that by having a big enough address space with no pre-existing reservations, they could make routing simpler and cleaner and delay the problem of routers running out of special route table memory and routing protocol horsepower, but that was pretty much a pipe dream:

          • Medium-large businesses want to own their own address space instead of using provider-owned space so they've got the ability to change carriers without renumbering,
          • businesses that want multi-homing for diversity need to have routing table presence regardless of what size their address blocks are,
          • geographical addressing may be ok for single-site businesses, but tends to fail for businesses with multiple offices (at least multiple offices with public presence),
          • and anybody who wants to be an early adopter (i.e. actually be using IPv6 long enough to be stable before the IPv4 ship sails off the edge of the world and everybody else notices the dragons and their ISP does something useful about IPv6) is likely to spend the ~$1250 to get their own public IPv6 space as opposed to just building a tunnel to SiXXs or Hurricane Electric,

          so the IPv6 world's going to be a non-hierarchical mess just like the IPv4 world.

      • As several other people have commented, the ISPs they connect to are responsible for doing some sanity filtering on the routes they announce. It's not universal, especially for connections between ISPs (as opposed to connections from end-user customers that use BGP for multi-homing, where ISPs usually do a better job), and there's nothing close to universal agreement about address range registration systems or how to validate BGP information.

        • by ShakaUVM (157947)

          >>there's nothing close to universal agreement about address range registration systems or how to validate BGP information.

          Given this same problem happened before back in the 90s, you'd think that they'd at least not allow negative route lengths to be propagated.

      • by diamondsw (685967)

        However, why should a network be able to advertise routes for subnets that are out of its control? Even if we accept multiple levels of peering relationships, there should be some safeguards against overly broad routes and "hijacking" of networks known to be authoritatively announced by other peers.

        (Note: I'm genuinely asking, as I'm fairly ignorant of the design of BGP - I'm much more LAN than WAN.)

        The whole idea of "trust" on the network is something of an anachronism. The internet is not the secure, safe

        • by jon3k (691256)
          Well these bgp sessions are customer -> provider or in a peering arrangement between provider provider. It's not like some anonymous service available to anyone. They delivered the Internet circuit to the customer. If the customer fucks up bad enough they just turn them off. When you have someone by the balls you can afford a certain level of trust.
      • It reminds me of a scenario we had at work. We come in one day and find that about half the computers in the building are getting bad IP addresses, and as such, weren't able to connect to the email servers or the internet. We found out it was a rogue router on the network, dishing out 192.168.1.x/24 addresses when that specific building was under 172.21.30.x/20. We were lucky that it was obviously a default linksys setup, we were able to log into it once we found the IP and disable DHCP. Then we had to go t

        • One of the lab computers and the lab instruments need to be on a 192.168.1.x/24 subnet because the lab instrument software is programmed terribly.

          One of the joys of being on the 172.x.x.x network scheme internally is finding devices like that. Everyone knows about 192.168.x.y and 10.x.y.z, but the 172 range tends to be overlooked.
      • by jon3k (691256)
        That's not entirely true. There's lots of route filtering going on. Typically a very small ISP shouldn't be capable of doing this because their BGP announcements should be filtered by their ISP. The closer you get to the core of the Internet the more difficult this becomes as you start dealing with Tier 1 carrier's peering sessions with thousands and thousands of routes being announced and changing constantly. But those aren't the type of people who make these types of "mistakes" as we're calling it.
    • Re: (Score:3, Insightful)

      by lukas84 (912874)

      The small ISP can't do this if the big ISP would've done it's job properly.

    • If you are expecting a router to pass GOOD data, how hard is it to believe that someone can trick you into accepting BAD data?

      This is no different than you downloading a Windows Update that bluescreens your computer. Clearly your equipment isn't configured correctly.

      In actuallity, in order to route things through China, you have to trust China, and yes, that sucks, and yes, I'm using way too many comas.

      • by dave562 (969951)

        I'm using way too many comas.

        If you were putting your comas to good use, you wouldn't have enough consciousness left to over-use the commas.

  • How rare/common is such screwups? Or are we just bashing Chinese (not that I mind it all that much, don't let me get in the way)?
    • Re:Chinese bashing? (Score:5, Interesting)

      by Blackbrain (94923) on Friday April 09, 2010 @04:03PM (#31794976)
      This kind of thing happens all of the time. Subscribe to the operators list at http://www.nanog.org/ [nanog.org] and you will see reports of mis-announced prefixes every month or two. This is just China bashing and media sensationalism. (Which I do mind very much, thank you)
      • Re: (Score:3, Insightful)

        Yeah, I'd be interested in knowing if I'm paranoid against China and this type of thing for no reason, but (and maybe it's just my paranoia talking) I think there's pretty good reason to believe this is intentional. The only time I've ever heard of large scale screwups like this are with China and once with Pakistan.

        Are you saying this is truly a selection bias, or are the Chinese screwups more global in scope? Seems like propagating a small ISP to a large ISP to the entire Internet would be something
        • Re: (Score:3, Interesting)

          by Blackbrain (94923)

          Don't get me wrong, this was a really big mistake. It doesn't happen often at this scale, but it does happen.

          In this case the prefixes what were mis-broadcast were sequential for the most part and covered several networks and countries, not a specific target. The bulk of the misrouted addresses were actually in China. They also didn't leak the routes (as in the Pakistan incident) but re-originated the prefixes, pre-pending their AS number to the announcement. This means "origin AS" based filters would have

          • Interesting - thanks for the technical info. If you hear of something this big in the future which involves a country other than China, I'd really appreciate an article, and I bet the /. editors would too. I think it would do a lot to remove the paranoia, if this does indeed happen in other places.

            Thanks again.
  • by MrTripps (1306469) on Friday April 09, 2010 @03:55PM (#31794820)
    Obviously the only way to protect the Border Gateway Protocol is to build a fence around it. (Spits. Scratches ass.)
  • by Beelzebud (1361137) on Friday April 09, 2010 @04:08PM (#31795046)
    This should really be cause for alarm. Does China also use the Narus systems that the NSA is using to spy on all Americans?
  • by StuartHankins (1020819) on Friday April 09, 2010 @04:17PM (#31795170)
    Someone had to say it.
  • by zenchemical (1468505) on Friday April 09, 2010 @04:28PM (#31795320)
    This is sort of the nature of BGP, at least when you are in the habit of trusting BGP peers. Methinks the large carriers should probably be in the habit of filtering BGP updates from chinese carriers, at least until they can pass "peering 101"
  • by Anonymous Coward

    So while this was going on could the chinese save off the network traffic? They have the infrastructure Cisco routers, etc.
    Could they decrypt SSL packets ? It may take awhile but they're not doing this real-time.
    Go through any interesting attachments ? Spreadsheets, documents, ...
    I think I'll read up more on asymmetric warfare and the Red Army officer's paper on the subject.

  • This is unlikely to be the last this will happen. What can be done to protect against this sort of issue?

  • Tier 1 & 2 ISP's should really be filtering all subnets they own. A lot of them do, but also a lot of them do not or think their Tier 2's are handling it. I've seen a company who was assigned a /24 misstype a number and suddenly they're claiming a /16 and disrupt a bunch of our customers.

    Unfortunately many companies are ill equipped to detect this type of error, internally they may see everything is fine, but it's external traffic that's being detected.

    It's easy if you can setup a server to check who'

    • by Kymermosst (33885)

      Oh yeah?

      What ISP owns 17.0.0.0/8? In fact, how does any ISP know what other ISP is allowed to advertise that prefix or a subnet of it?

      • by jon3k (691256)
        Depends on the relationship. But the easiest is Tier 1 to Tier 2 where the Tier 2 is a customer of Tier 1.

        In which case, the Tier 1 should filter announcements for anything other than:
        • address space they gave the tier 2 isp
        • portable address space owned by the tier 2
        • address space neither owned by the tier 1 or tier 2 but for another ISP who has provided a LOA to allow the tier 2 to announce that particular address space

        And to answer your other question, Apple owns 17.0.0.0/8
        OrgName: Apple Inc.
        OrgID

  • It seems like Slashdot has been hit hit by China.

    If I try:

    http://slashdot.org/firehose [slashdot.org]

    or

    http://slashdot.org/~ls671/ [slashdot.org]

    I have been getting this for the past half hour:

    Error 503 Service Unavailable
    Service Unavailable
    Guru Meditation:
    XID: 147127282289
    Varnish

  • RIPE is pushing to have all route announcements signed by 1.1.11 and the other four RIRs are following suit. Personally, I can't wait for this to happen :)

    • by jon3k (691256)
      I don't see how signing announcements stops this from happening? All that would do is make it (nearly) impossible to forge announcements. That's not and hasn't ever been a problem that I'm aware of. We know where the bad routes are coming from.
      • by RichiH (749257)

        You know where they are coming from, but it seems the tier 1 & 2 ISPs are not willing to filter incoming routes from China. With signed announcements, they would have something to filter on.
        While that won't help if they just strip all intermediary AS numbers out of the routes (unless upstreams are also verified, at some point), it will still improve the overall situation.

The typical page layout program is nothing more than an electronic light table for cutting and pasting documents.

Working...