Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×
Bug The Internet Technology

Transatlantic Cable Fault Disrupts Internet In UK 394

Posted by timothy from the electrify-the-atlantic dept.
An anonymous reader submits "Web traffic between the U.S. and Europe has been hit after an undersea cable developed a major fault on Tuesday. Because the TAT-14 cable network is shaped like a ring, it should be able to cope with one such failure -- but unfortunately the consortium that owns it hadn't fixed an earlier problem, just off the U.S. coast. Just shows how systems with build-in redundancy can still go badly wrong...."
This discussion has been archived. No new comments can be posted.

Transatlantic Cable Fault Disrupts Internet In UK More

Transatlantic Cable Fault Disrupts Internet In UK

Comments Filter:

  • Tin foil hat, please. (Score:5, Funny)

    by grub ( 11606 ) <slashdot@grub.net> on Wednesday November 26, 2003 @11:32AM (#7569516) Homepage Journal

    LINX, the London Internet Exchange, which carries nearly all UK Internet traffic and over half of Europe's Internet traffic

    I guess the Echelon boys got to go home early that day.

  • Ok let me get this straight.... (Score:5, Funny)

    by Neck_of_the_Woods ( 305788 ) * on Wednesday November 26, 2003 @11:34AM (#7569537) Journal

    We have a link from the US to the UK.

    It is redundant, unless we have 2 faults.

    We have a single fault...but we don't repair it.

    So then we have anouther one!

    I would really like to ask if these guys ever thought of putting together a startup....because let me tell you, they already have the right frame of mind.

    • Re:Ok let me get this straight.... (Score:5, Insightful)

      by bugbread ( 599172 ) on Wednesday November 26, 2003 @11:36AM (#7569572)
      My guess is that the initial problem may have been an undersea cable. Those generally take 2 or more weeks to fix, and if the weather is really bad, they have to pull the boats back in, delaying things further.

      No evidence, of course, but it seems like the most logical reason. Cables like the TAT-14 don't stay unfixed just because someone in management is lazy.
      • I have to agree here on that you can't fix something overnight. I was just pointing out what was best stated below:

        Oh, ho, ho, irony! Oh, no, no, we don't get that here. See, uh, people ski topless here while smoking dope, so irony's not really a high priority. We haven't had any irony here since about, uh, '83, when I was the only practitioner of it. And I stopped because I was getting tired of being stared at.
        -- Steve Martin in Roxanne

      • Cheap management, not lazy management (Score:5, Funny)

        by swb ( 14022 ) on Wednesday November 26, 2003 @12:22PM (#7570169)
        You're new here, aren't you? Management is cheap, not lazy. Redundancy means that when something breaks they save money by not fixing it, not that they can keep running while they do fix it.

        They delays in repair may also be due to the bids they have out to fix it: A Greek sponge diver, the "Polynesian" pearl diver from an unnamed Florida amusement park and a crew from Bangalore with no diving experience or equipment, but a willingness to follow the diving script. There's also a chance that an unnamed "muff diver" may be employed as well, but executives are downplaying it as part of their don't ask, don't tell policy.

        Management originally wanted the crew from "Ghost Ship" because the chick was hot, but when they found out it was only a movie they had to look elsewhere.
    • after reading the article, it says that the first fault happened within the last month, and that it should be fixed within the week.

      these cables lie under several fathoms of ocean, they are not that easy to just fix.
    • It's very likely that the initial fault has been in the process of being repaired for quite some time. In the best of conditions, it can take a while to fix fiber. Now imagine trying to fix it on the high seas.

    • probably was a startup (Score:4, Funny)

      by ciaran_o_riordan ( 662132 ) on Wednesday November 26, 2003 @11:41AM (#7569658) Homepage
      It probably was a startup, y'know, back in the ol' under-water-cable boom of 1999. But then the bubble burst, and all the people that dropped out of college to lay cables on the ocean floor had to find real jobs.

    • Re:Ok let me get this straight.... (Score:5, Informative)

      by gorilla ( 36491 ) on Wednesday November 26, 2003 @11:47AM (#7569757)
      Remember that fixing faults in undersea cables isn't exactly an activity that you can do in 30 seconds. You've got to get a ship out to the location of the fault, hook the cable, and get it to the surface, and then fix it. There are going to be a limited number of cable ships [seafarers.org] which have the capability, and they might be busy elsewhere. Even once they start acting on the repair, they are going to take time to get to where the fault is (14 knots cruise speed isn't exactly the fastest ship around the QE2 cruises at 28 knots, and still takes a week to cross the atlantic. Remember a cable ship might be off in the other side of the pacific when it becomes free), and then time to get the cable and repair it. Therefore 'earlier this month' not being repaired is perfectly reasonable.

    • Re:Ok let me get this straight.... (Score:5, Insightful)

      by Anonymous Coward on Wednesday November 26, 2003 @11:52AM (#7569819)
      The real problem is in the design of networks. Information networks are designed to be fault-tolerant (famously but erroneously attributed to a desire to withstand nuclear attacks) -- multiple connections and a "mesh" network mean that if nodes break, traffic is routed elsewhere and the network continues to function. This works great, and there's no problem with it. But the problem is, humans don't build networks this way, and economics is against doing so.

      If you're buying a network connection, you buy it from the best provider available, which naturally means network connections become concentrated to a few suppliers, who in turn find economies of scale and provide lower prices, thus attracting more customers. Thus the economics of building networks naturally produces networks that have a few or even single points of failure: we noticed this on September 11th, when the knockout of the huge links through New York noticeably slowed transatlantic traffic, even to sites other than CNN and the other news sites that were being toasted by demand at that point. Centralisation is something that we naturally do because it's economically efficient, but centralisation leads to problems for networks.

      In the energy sector, things are even less flexible, because energy connections are a lot more expensive to set up and difficult to maintain than information links. The US powercut was caused by the cascading failure of a daisy-chain of power stations around the great lakes. Nobody would build an information network that way any more, but it's still the natural way to build a power network. Italy's powercut was caused by a huge reliance on foreign power, supplied by JUST TWO LINKS to France -- one fell over, instantly overloading the second and knocking it out too.

      Yes, we are critically reliant on these fragile networks. And yes, economic realities tend to cause these problems, but not because of privatization: it's simply because humans naturally tend to build poor networks, because those are cheaper -- no matter who pays the bills. To solve the problem, we need to pay more attention to networking theory when building all of our networks, and provide regulatory incentives to build better networks of both kinds.

      Or one day, a critical failure will cause a cascading catastrophe, and it will be nobody's fault. We built the network to fail that way.

  • That's totally fuct (Score:3, Insightful)

    by siphoncolder ( 533004 ) on Wednesday November 26, 2003 @11:35AM (#7569561) Homepage
    Imagine... some big cable that's thousands of miles long connecting continents...

    That's just a weird idea. You gotta wonder who makes those things and how, exactly, they're maintained. Let alone set up in the first place. Do they just sit along the ocean floor? Are they suspended in mid-water? I have absolutely no idea. Just mind-boggling to me, the logistics of it.
    • It's literally one big, long, armored cable drapped across the ocean floor between the U.S coast and England.

      hard to imagine, ain't it.

    • Re:That's totally fuct (Score:5, Informative)

      by bugbread ( 599172 ) on Wednesday November 26, 2003 @11:40AM (#7569642)
      The sit on the ocean floor. In low water areas, they are cladded in steel, to prevent anchors, etc. from ripping them up. Recent ones are apparently treated to prevent sharks from chewing on them, which was an old problem. Fixing them involves sending out a big ship that hauls up cable from the ocean floor (they have a lot of slack so that this is possible), hanging the cable across the deck, fixing it, and lowering it back into the water.

      Yes, I'm a WAN administrator, why do you ask?
    • My thoughts exactly. That's really an impressive technology. Kinda creepy, too. All that information streaming through the dark sea bed.....

      • Re:That's totally fuct (Score:5, Funny)

        by pla ( 258480 ) on Wednesday November 26, 2003 @11:51AM (#7569803) Journal
        That's really an impressive technology. Kinda creepy, too. All that information streaming through the dark sea bed...

        Now you know why the high-pressure methane breathing aliens (the ones who live under the sea for convenience of maintaining "atmospheric" pressure in their domes) know all about us, but we know almost nothing about them.

        We've just given them a high-bandwidth line that we have almost no ability to monitor between the two endpoints.


        (For the humor-impaired... Laugh).
      • I'm not sure how impressive it is, considering that we've been doing the same basic thing since 1858 [ciolek.com]. All we've done since then is increase the speed of each link, and the numbers of links in each cable.

    • Re:That's totally fuct (Score:3, Interesting)

      by gr8_phk ( 621180 )
      How?

      1) put a big spool of cable on a ship.
      2) anchor the cable to shore.
      3) set sail for the other side of the ocean.

      I had no idea it could be so simple and obvious when I heard it either.

    • Check out this great article by Neal Stephenson [wired.com] in wired. It talks about running an even longer cable. Beware though. He's in his typical verbose form. The article is 56 pages long.

    • Re:That's totally fuct (Score:5, Informative)

      by velo_mike ( 666386 ) on Wednesday November 26, 2003 @11:50AM (#7569793)
      You gotta wonder who makes those things and how, exactly, they're maintained.

      Check out Global Marine Systems [globalmarinesystems.com] the company that laid it, and some of their cool [globalmarinesystems.com] toys [globalmarinesystems.com] (er, if you're into big assed boats).

  • Say it ain't so! (Score:4, Funny)

    by Sanity ( 1431 ) * on Wednesday November 26, 2003 @11:35AM (#7569566) Homepage Journal
    With the Brits off the Internet, who are we going to rely on to correct our grammar? Who is going to tell us that every plot of every US scifi show was done in the 70's by Dr Who? Who is going translate Alan Cox' Welsh weblog for the rest of us? Who will fight the other side of the "Who invented the first computer?" debate?

    This really is a great loss for the Slashdot community.

    • You forgot the tea advocacy :-)
    • Not to worry. Us Canucks are still here. We even have the 12 hour time difference.

    • How about an expat living in Canada? Hahar: for my skills are kept sharp by their resistance to the onslaught of American spellings and date formats! Okay, so part of the time - they seem confused as to which to go these days: whenever I drive past Canadian Tire I think of people getting sleepy, not the bits of rubber on my wheels.
    • Speaking as a Brit, this isn't flamebait, it's funny!

    • Re:Say it ain't so! (Score:2, Funny)

      by Anonymous Coward
      Reports of us Brits being off the net have been greatly exaggerated

    • Re:Say it ain't so! (Score:3, Informative)

      by arevos ( 659374 )
      I'm from Britain, and I'm still here :)

      As regards to who invented the first computer, I'd imagine it was more a debate between the German's Z3 and Britain's Colossus. The Z3 was Turing-complete, but only if you hacked it a bit, and it wasn't originally designed to do that, nor was it likely to have ever run that way (if I recall, it involved literally forking the punch tape, and taking advantage of a bug in the mechanical reader to similate an "if" function). Colossus was Turing complete computer, and was
    • Er, if the Brits are off the internet then you're not going to be able to read AC even in the original Welsh. As far as I know (living 12 miles from the border) Wales is still a part of Britain (despite the best-laid plans of the English to cut it loose and float it off into the Irish Sea ;-P)
  • lay some new cables... Why run one when two or three will obviously be better.
    • You know this kind of fibre isn't cheap don't you? :)

    • Re:Time to (Score:3, Informative)

      by gorilla ( 36491 )
      Since this one is named TAT-14, it's not suprising that there are other TransAtlantic cables. There are currently active 8 different cables that AT&T [att.com] use crossing the atlantic TAT-8 through TAT-14, and BUS-1. Cables TAT-1 through TAT-7 are retired.

  • I was suprised this wasnt in (Score:5, Informative)

    by Pingular ( 670773 ) on Wednesday November 26, 2003 @11:36AM (#7569578)
    the article, but here's [google.com] the link to the linx (badum tsh) website, with another [macworld.co.uk] news site for the article.

  • First Cable (Score:5, Interesting)

    by General Sherman ( 614373 ) on Wednesday November 26, 2003 @11:36AM (#7569581) Journal
    Honestly, this reminds me of the first transatlantic cable. They kept getting faults and they couldn't figure out what it was. Turns out the paying-out machine had the cable rubbing against some fine metal shavings which would occasionally get stuck in the casing and ground the cable to the sea-water.

    I wonder what happened to this one?
  • I'm in the UK, and I haven't noticed any problems. I've even had a realaudio stream running without interruption.
    • I didn't notice any specific difficulty accessing US sites, but that was probably because NTL's cable internet service was entirely down all night (certainly in Nottingham) last night. I assume it was unrelated to this.

  • Eh? (Score:2)

    by Walterk ( 124748 )
    So, I'm in Britain, and obviously not doing my work, as I'm supposed to, and I wasn't on tuesday aswell, but I didn't notice any fault?

  • Comment removed (Score:3, Funny)

    by account_deleted ( 4530225 ) on Wednesday November 26, 2003 @11:37AM (#7569600)
    Comment removed based on user account deletion

  • old news (Score:5, Informative)

    by Anonymous Coward on Wednesday November 26, 2003 @11:37AM (#7569601)
    this was all over the service provider lists yesterday...

    The latest from the rumor mill....

    FYI, for some history on the TAT's

    http://davidw.home.cern.ch/davidw/public/SubCabl es .html

    still seeing decent ping times. anyone detect an actual outage or issue? Best info we have is that there are two outages. One has existed
    for the last 3 weeks or so between Tuckerton (New Jersey) and Bude (UK). It takes out the "southern path" across the atlantic.

    There is a second outage between Bude (UK) and Katwijk (NL). For circuits that landed in London or France this (should have) taken out the redundant path for those circuits.

    Circuits from Tuckerton (New Jersey) or Manasquan (New Jersey) to Katwijk (NL), Norden >(DE), or some city in Denmark who's name I
    forget should still be up on the northern path.

    > So, if you're in London or France your circuits are likely to be down, however some people in those locations used Contentinal capacity to link up to Katwijk, in which case they might still be operational.

    I confirm that France is having some problem with TAT14.

    France Telecom International Backbone (Opentransit) is currently running with
    non TAT14 capacity (10G) and one oc48 direct to Copenhagen (that is ok).

    We (Opentransit) are currently not experiencing any congestion but are implementing a new 10G circuit to secure our topology until TAT14 is back to life (one leg at least).

    Both problems are undersea issues, so don't expect speedy resolution if you are down.

    Yep .. i heard days ... not hours :-(

    -Opentransit (France Telecom)

  • The redundancy didn't go wrong.. (Score:3, Insightful)

    by Keck ( 7446 ) on Wednesday November 26, 2003 @11:37AM (#7569602) Homepage
    Just shows how systems with build-in redundancy can still go badly wrong...."

    Um, the built in redundancy worked as it should, apart from the maintainers not fixing the first fault. Their maintenance is what went wrong. Nobody will ever be able to afford or build a system like this with so much redundancy that you aren't required to maintain it.

  • Noticable impact (Score:5, Informative)

    by rf0 ( 159958 ) * <rghf@fsck.me.uk> on Wednesday November 26, 2003 @11:38AM (#7569604) Homepage
    It was definitly noticable as our customer reported

    1) Website traffic down at least 30%
    2) Around 75% packet loss from the EU -> US
    3) Slow delivery of email

    Basically it caused a massive amount of headaches and you have to wondered WTF didn't they fix the first problem when it came up. Its like running a RAID Array on one disk.

    Well least things seem to sort of be getting back to normal

    Rus

  • No... (Score:5, Insightful)

    by PSaltyDS ( 467134 ) on Wednesday November 26, 2003 @11:38AM (#7569608) Journal
    "Just shows how systems with build-in redundancy can still go badly wrong...."

    No, it shows how well designed redundancy can be overcome by bad management decisions! Engineering brought low by bean counters... Gee, when has that ever happened before?!

    Any technology distinguishable from magic is not sufficiently advanced.

  • She added that the Internet was not broken, as traffic was rerouted through other networks.

    Au contraire. The Internet *IS* broken, regardless of damage to this particular cable.

  • Well, that's consortiums for you. (Score:5, Funny)

    by HarveyBirdman ( 627248 ) on Wednesday November 26, 2003 @11:38AM (#7569618) Journal
    Can't they get some quantum wormhole tunneling dealy-widgets-whatzits going so we don't need cables anywhere anymore?

    Man, the FBI is going to have to interview *every* *single* *fish* in the area for Al-Queda connections.

    No one will even suspect the dolphins because they are supposed to be, like, higher mammals or something.

  • Statistics (Score:2, Informative)

    by Etherwalk ( 681268 )
    Anybody have hop count & RTT statistics?
  • You only have to loose one, and suddenly you have no reduandancy.

    I would think for such an important link, there would be at least 3 (so you still have some redundancy durring repairs if one fails).

  • Oh no the Internet almost broke? (Score:3, Funny)

    by hellfire ( 86129 ) <deviladv AT gmail DOT com> on Wednesday November 26, 2003 @11:44AM (#7569723) Homepage
    She added that the Internet was not broken, as traffic was rerouted through other networks.

    I read this and I couldn't help but think of a CDW commercial:

    Clueless pointy-haired boss to the camera: "Fred? I think I just crashed the Internet."

  • You'd Think (Score:3, Interesting)

    by The Dobber ( 576407 ) on Wednesday November 26, 2003 @11:44AM (#7569724)
    Because the TAT-14 cable network is shaped like a ring

    That on a geeky, tech oriented site such as this, we could have a slighty better description.

  • That's Strange, I'm in the UK (Score:5, Funny)

    by Mirk ( 184717 ) <[ku.gro.rolyatekim] [ta] [todhsals]> on Wednesday November 26, 2003 @11:46AM (#7569739) Homepage
    That's strange, I'm in the UK and SlashDot is hosted in America, so according to this story, I should be having problems -- but in fact, everything is working just fiFgfdgf3gf4h32hh%$$$424452
  • Seems my connection is just fine. In fact, if someone hadn't told that there was 'widespread disruption to Internet services in the UK' I would not have noticed.

    Perhaps it's only an issue for certain networks/ISPs?

  • Just wiggle the cable! (Score:5, Funny)

    by MrNybbles ( 618800 ) on Wednesday November 26, 2003 @11:47AM (#7569756) Journal
    Why not do what my friend Mike does when he has a problem with a bad cable and just jiggle it a little? It works great for his monitor cable so why not for a giant bundle of fiberoptics/wires/whatever in the ocean? What could possably go wrong? Jiggling the cable has got to be cheaper than going down to BestBuy and buying a new cable and running it from the US to the UK. Don't get the extended warrenty though, it's not worth it!
  • I don't know what everybody's complaining about. I mean we've got these big, thick cables connecting us to our friends in the west. I'm in the UK and I'm not having an$**!#@j pr83

    NO CARRIER

  • actual information... (Score:5, Informative)

    by Anonymous Coward on Wednesday November 26, 2003 @11:50AM (#7569790)
    One failure, occured on Oct 30, 2003, has existed for the last 3 weeks or so between Tuckerton (New Jersey) and Bude (UK). It takes out the "southern path" across the atlantic.

    The new failure is between Bude (UK) and Katwijk (NL). For circuits that landed in London or France this (should have) taken out the redundant path for those circuits.

    more info at

    www.tat-14.com /.s since yesterday actually

    http://www.kddiscs.co.jp/e/business/02_15.html
    http://davidw.home.cern.ch/davidw/public/SubCables .html

  • For those who are too lasy to GooGle: TAT-14 (Score:5, Informative)

    by B5_geek ( 638928 ) on Wednesday November 26, 2003 @11:52AM (#7569830)

    About the TAT-14 Cable Network
    This transatlantic cable system is in full service, connecting the United States to the United Kingdom, France, The Netherlands, Germany, and Denmark.

    The cable system is a dual, bi-directional ring configuration using DWDM multiplexing with 16 wavelengths of STM-64 per fiber pair. The system also utilizes reverse direction protection switching in the event of failure of the service fiber.

    It has a dual route, transatlantic capacity of 640 Gbits on 2 service fiber pairs backed up by 2 protection fiber pairs. This configuration provides a capability of transporting 4,096 STM-1's or approximately 9,700,000 circuits across the ocean.

  • Read The Article (Score:2, Insightful)

    by Ever Dubious ( 686307 )
    It isn't like the US-side fault was just being ignored: "According to BT, the US-side fault should be fixed by the end of this week, which will bring the cable network online again." Given the logistics of repairing a fault, and without knowing when the US-side fault occurred, it is difficult at best to imply that the cable operators were somehow negligent in their actions.
  • I notice most of the Didn't-Read-The-Article crowd seem to be missing that this is an undersea cable. They're Not Easy to work with, and besides, this outage is a great excuse to point you to Neal Stephenson's great geeky essay
    Mother Earth, Mother Board [wired.com]
  • I remember reading a book like this from one of Tom Clancy's junior authors. The whole story was about cartel's splicing into undersea fiber cables and moving the data elsewhere. I definitely have to check my library when I get home.

  • US Navy Cable Ship (Score:5, Informative)

    by PSaltyDS ( 467134 ) on Wednesday November 26, 2003 @12:01PM (#7569927) Journal
    The USNS Zeus (ARC-7) [navy.mil] is the Navy's cable laying and repair ship. The cable is laid mostly on the surface of the bottom, but at vulnerable points and at both ends (near shore) is its ploughed in to the mud/sand on the bottom. When a cut or fault occurs, the location of the fault is determined with a TDR or O-TDR [atis.org], the same way it works with a land based cable. They know the cable length to the fault and have a survey map of where the cable was layed. It is physicaly located with side-scanning sonar and robotic submersibles, then hooked and brought on deck for repair (each end in case of a break). Once repairs are complete, the cable is unceremoniously shoved over the side, or re-ploughed depending on the location and mission of the cable.

    Any technology distinguishable from magic is not sufficiently advanced.

  • Map of Trans-Atlantic Cable Network 14 (TAT-14CN) (Score:5, Informative)

    by Anonymous Coward on Wednesday November 26, 2003 @12:04PM (#7569958)
    Here's a map that I found which shows the "ring" of TAT-14...

    TAT-14 Cable Route [kddiscs.co.jp]
    • Interesting!

      Play with numbers:

      Sprints info on TAT-14 [sprint.com]

      As stated, the TAT-14 is 16 pairs of STM-64 fiber. With a help from google, the average cost was $6000/km per cable.

      6000 * 16 = $960,000/km For All 16 pairs.

      The total length of the cable is around 15,000km long.

      $960,000 * 15,000 = $1,440,000,000

      The cost of a transatlantic link cost almost 1 and a half billion dollars that is capable of 640Gbits throughput!

  • More than one cable system (Score:3, Interesting)

    by bvark ( 58049 ) on Wednesday November 26, 2003 @12:10PM (#7570033) Homepage
    There's more than one cable system linking US with Europe, it just happens that several carriers (Above.Net being one) only have capacity through TAT-14.

    Other carriers have working circuits on TAT-14 and another link (e.g. Apollo, Tyco, AC-1, Gemini) and may have some degraded service (depending on whether their transatlantic links are less than twice the size of their peak demand). FranceTelecom OpenTransit is an example of one of them.

    Interestingly, not many EU ISPs use TAT-14 North route, since it has a propagation delay of around 110ms (which is 40ms or so more than TAT-14 South from the UK and more than most other transatlantic cables)

    Most ISPs in Europe that I can see are fine. Certainly the big international transit ISPs (Sprint, L3, C&W, MCI et al) aren't showing any more trouble than normal.

    At the risk of being accused of Karma whoring, This page [home.cern.ch] and This wired article from the late 90s [wired.com] are are good summary and a great story about undersea cables, respectively, despite being a little out of date.

  • Whose fault? (Score:5, Funny)

    by iamdrscience ( 541136 ) on Wednesday November 26, 2003 @12:14PM (#7570073) Homepage
    Whose fault were these faults? Were these faults the fault of the oceanic faults? I've heard some people say that these faults were the fault of faulty maintenance, but it seems to me that you can't fault them for the faults. Now perhaps that's a faulty assertion, but I really believe that the fault of the faults lies squarely upon the techtonic faults and not the fault of this supposedly faulty maintainers. I really doubt that the faults are their fault.

    I'm sorry.

    ...this post was all my fault. :-(

  • A fly on the wall would likely have heard.... (Score:5, Funny)

    by Newer Guy ( 520108 ) on Wednesday November 26, 2003 @12:16PM (#7570100)
    As TAT-14 is a dual, bi-directional ring of cable, a single serious fault should not be enough to break it, as traffic would still be able to flow between the countries on the ring. Unfortunately, a part of the cable near the US coast had already suffered a technical fault earlier this month, which meant there was no built-in redundancy to cope with Tuesday's failure.

    I can hear it now:

    "Hey boss, half of the cable just failed. We need to get on this right away".

    The cable's still working, right?

    "Yeah, but if something else goes wrong, we're screwed".

    Look, that cable hasn't failed in ten years; let's put off repairing it until January. That way it won't affect our 2003 budget.

    But these things generally happen in pairs, and with no back up - well, we're taking an awful risk. If something else fails, most of Europe - well, I don't have to tell you the consequences. Plus, remember..the weather in January -

    cuts him off-- Not gonna happen! Put it on the schedule for mid January!

    Hopefully that manager is no longer employed....but don't be surprised if he winds up at Clear Channel! He sounds like just their kind of guy!

  • this affected the whole of Europe (Score:3, Interesting)

    by viktorVaugh ( 727348 ) on Wednesday November 26, 2003 @12:26PM (#7570226)
    As far as I'm aware the problem wasn't just limited to the UK but to the whole of Europe. One of our transit connection to the US using this Fiber, was disrupted. Following message we received from our transit provider:

    We are currently experiencing a catastrophic failure on the fiber ring that is
    affectively isolating Europe. We are researching the possibility of
    alternative connectivity, and will update you as we get more information.

    One more problem which was caused by this link outage is that our dns-servers (and those of multiple providers) where hit with a lot of dns lookups for lockdown.zonelabs.com (seems zonelabs firewall, queries that name). As the dns-server for that zone wasn't reachable anymore (no more traffic to the abovenet network in the US) the dns-servers had to do a query for each new lookup which caused a huge load. And effectively killing the customer dns servers, impacting traffic even more.

  • Mother Earth Motherboard (Score:4, Interesting)

    by elliotj ( 519297 ) <slashdot.elliotjohnson@com> on Wednesday November 26, 2003 @12:32PM (#7570289) Homepage
    I've found the whole notion of undersea cables fascinating ever since I read Neal Stephenson's Mother Earth Motherboard [wired.com]

  • Mirrored Drives (Score:4, Interesting)

    by ryanw ( 131814 ) on Wednesday November 26, 2003 @12:33PM (#7570300)
    Reminds me of how we went through all this trouble at [UNNAMED CORPORATION] of making sure to mirror the root disks for all 3000+ servers, but nobody setup alerts or notifications of a disk failure. So even though all the disks were mirrored if one drive failed, nobody knew. So we ended up running most our boxes off one drive until the other drive went out. So sure, mirroring delayed a major problem, but the major problem still existed.

    We also had a similar problem with Fiber Storage. For all the servers they had run two seporate fiber runs to each box that needed to use the "SAN". Each server would have two fiber cards installed. This way if one network went out, it would just fall back to the other card. Well, of course, both cables were plugged into the same switch.. Smart. Yes, we did have a fiber switch go out once.

  • So, then you might say.... (Score:3, Funny)

    by switcha ( 551514 ) on Wednesday November 26, 2003 @04:08PM (#7572528)
    As TAT-14 is a dual, bi-directional ring of cable, a single serious fault should not be enough to break it,...

    ...customers may currently be experiencing problems with all Internet access, including Web browsing, email, ftp and newsgroups.

    So....there was just one ring to screw them all?

Slashdot Top Deals

"The vast majority of successful major crimes against property are perpetrated by individuals abusing positions of trust." -- Lawrence Dalzell

Close