Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
The Internet

Time Warner Cable Experiences Nationwide Internet Outage 133

Posted by Unknown Lamer
from the left-the-intern-alone-for-30-seconds dept.
Wolfrider (856) writes Reports are coming in from multiple U.S. states that TWC is having a major Internet outage since early this morning. ... TWC's customer service reps are reportedly a bit overwhelmed by call volume at the moment, and no ETA yet. According to CNET, most locations should be more-or-less back online as of 7 a.m. EDT or so (my TWC connection came back around 7:30 a.m. EDT). TWC says it's maintenance gone wrong: In response to a query by CNET, Time Warner Cable issued this statement: "At 430am ET this morning during our routine network maintenance, an issue with our Internet backbone created disruption with our Internet and On Demand services. As of 6am ET services were largely restored as updates continue to bring all customers back online."
This discussion has been archived. No new comments can be posted.

Time Warner Cable Experiences Nationwide Internet Outage

Comments Filter:
  • by alen (225700) on Wednesday August 27, 2014 @08:28AM (#47764433)

    this level of incompetence clearly deserves the penalty of death
    my kids woke up this morning and couldn't watch their Phineas and Ferb cartoons. my oldest kid had to actually read a book to pass the time before camp and a trip to the museum

    • by B33rNinj4 (666756)
      That must have been torture for them. I'm praying for their safe recovery.
    • by Anonymous Coward

      my kids woke up this morning and couldn't watch their Phineas and Ferb cartoons.

      I hope you reminded them that there is only 5 days of summer vacation left before school comes along just to end it. And that the current problem for them will be trying to figure out how to spend it. Without Cable/Internet. ;)

    • by Anubis IV (1279820) on Wednesday August 27, 2014 @09:26AM (#47764847)

      I hope you seized the opportunity to share at least 2 or 3 eyeroll-inducing "back in my day..." stories. Otherwise, I don't know how else you could have possibly salvaged such a fiasco.

    • Re:shoot the admins (Score:5, Interesting)

      by MachineShedFred (621896) on Wednesday August 27, 2014 @10:31AM (#47765457) Journal

      I get the sarcasm, but this outage almost screwed me - I was scheduled to work from home today so I could be here for a service appointment. With no data service, I can't do that.

      There are real uses for home Internet connections besides porn and Twitter, you know. =)

      • Re: (Score:3, Insightful)

        by praxis (19962)

        I get the sarcasm, but this outage almost screwed me - I was scheduled to work from home today so I could be here for a service appointment. With no data service, I can't do that.

        There are real uses for home Internet connections besides porn and Twitter, you know. =)

        If it is important you don't get screwed then a redundant connection at home might be the answer rather than trusting a consumer cable company. Unless you were exaggerating with the "screwed" and could just afford to take the day off.

        • by Calydor (739835)

          Buying a redundant net connection on the off chance that your net is down on the one day you REALLY can't leave home seems a little over the top to me.

          • Last time we had a DSL outage, we had to use our phones and tablets for connectivity. In many ways, I'm really getting to like the 21st Century.

          • by praxis (19962)

            Buying a redundant net connection on the off chance that your net is down on the one day you REALLY can't leave home seems a little over the top to me.

            I agree. If it is CRITICAL, have redundancy. If it is not CRITICAL, you are not screwed if it fails. He stated he was screwed so I assumed it was CRITICAL.

      • by antdude (79039)

        No backup Internet service? I have dial-up for these Internet emergencies. :)

      • " I was scheduled to work from home today so I could be here for a service appointment"

        Was it a service appointment with Time Warner?

      • I was scheduled to work from home today

        Meh. You are just a fraction of a percent. A rounding error. In this new world of global dominance and market share, fractions of a percent do not count. That is what the MBA folks say anyways. :)

    • No need to shoot them. They just need to be replaced. I hear comcast's network is ALWAYS up. I bet if we just got ahold of their admins... maybe through a merger... service would be WAY more reliable. Don't all you customers want more reliable service? Just support the merger!
      • Cable service in my area has been out a lot lately. Usually for only short periods.

        A technician told me they are making network changes "because Comcast bought us". Apparently nobody told him it wasn't a done deal.
    • While for the standard home user, 4:30-7:30 outage, means you have to forgo your morning entertainment. However there are a lot of small/mid sized businesses dependent on TWC for their operations.

      I just wish they would send me an email about the outage so I know not to try to reboot my wireless router then my cable modem. Then plug my PC to the cable modem to see where the connectivity down.

      (BTW I was kidding about the email notification)

    • by antdude (79039)

      TWC just wants your family to buy their TV service to watch cartoons. :P

      • by Touvan (868256)

        TWC just wants you to send them increasing amounts of money. They don't care for what.

  • I was affected (Score:5, Informative)

    by Anonymous Coward on Wednesday August 27, 2014 @08:29AM (#47764447)

    My time Warner went out sometime around 4:30 EST this morning, oddly enough it came back up before I was awake (IRC logged my disconnect) and with IPv6 service finally, it's a good chance they had a hiccup doing a mass switch to IPv6

    • it came back up before I was awake (IRC logged my disconnect) and with IPv6 service finally

      Guess it's time to turn that on on my router...

    • by antdude (79039)

      Same here with my friend and me in Southern CA, USA: ...
      [03:58am] * Ant is away: (Auto-Away after 120 mins) [BX-MsgLog On]
      ï½ï½ï½ Connection closed from irc...: Remote end closed connection
      ï½ï½ï½ BitchX: Servers exhausted. Restarting. ...
      03:58AM ï½ï½ï½ Ant [~ant@...socal.res.rr.com] has joined #linux
      ï½ï½ï½ Topic (#linux): http://preview.tinyurl.com/muy... [tinyurl.com]
      ï½ï½ï½

    • I have TWC and happened to be awake and doing a little Internet surfing around 4AM CST. My connection went down. Now I know what happened.
    • I can't believe how much chat there is about this outage, and so little talk about what diagnostics showed the problem was. It was a DNS issue, NOT a general routing issue.

      It hit us here in Austin TX. It looked like a DNS outage... but I was using Google DNS. Routing was NOT down... I could still access a selection of web servers by direct IP address, and ping and traceroute. Rebooted my modem and router repeatedly. Modem acquired a link quickly, and status page showed it had a valid configuration. Modem'

  • Comcast (Score:5, Funny)

    by bondsbw (888959) on Wednesday August 27, 2014 @08:30AM (#47764455)

    TWC: "Surely only Comcast can save us! Please allow the merger!"
    Government: "Well golly gee, what a great idea!"

    • Re:Comcast (Score:5, Insightful)

      by gstoddart (321705) on Wednesday August 27, 2014 @09:05AM (#47764671) Homepage

      TWC: "Surely only Comcast can save us! Please allow the merger!"
      Government: "Well golly gee, what a great idea!"

      Comcast: Mwuhahahaha ... no data caps(*), bitches [slashdot.org].

      (*) You can buy all the data you want like suckers, and we'll keep getting away with crap like this once we're the only game in town.

      • Re: (Score:3, Insightful)

        by bouldin (828821)

        Don't worry, in 20 years your only choice will be google.

        Think that's hyperbole?

        http://fiber.google.com/about2 [google.com]

        I'm a big fan of google right now, but let's see how long "don't be evil" lasts once Larry and Sergey have moved on, and MBA brain damage is calling the shots.

        • by gstoddart (321705)

          I'm a big fan of google right now, but let's see how long "don't be evil" lasts once Larry and Sergey have moved on, and MBA brain damage is calling the shots.

          Some of us would argue they gave up on "don't be evil" quite some time ago.

          I don't disagree, however, that a future Google will be even more evil.

        • Don't be evil? That packet left the switch awhile back....

        • Yeah, and in that 20 years, all of the jobs will be overseas managed by both people and equipment overseas. America will be the land of the global elite much how London is now as a city. The poor will still be there to take care of local services; poor because too may people competing for too little jobs. A true have / have-not society.

          The future of America in 20 years will be one with a severally if not totally diminished middle class.

        • Well, we saw what happened when the Bell System got too big for its britches. Heavily regulated for fifty years or so and then broken up. Too bad the days of the trust buster and anti-monopolist are gone.

  • by Shadow99_1 (86250) <.moc.liamg. .ta. .99wodahseht.> on Wednesday August 27, 2014 @08:34AM (#47764469)

    I am on Time Warner at home and when I got up this morning I found I couldn't check my email like I do every morning. I spent a bit of time on it trying to figure out what they had done to the connection and found I could not connect to their DNS servers, but I could connect to remote IPs and even trace route them just fine.

    I had to head into work, so I didn't stay to actually call them... But I'm hardly surprised if they have screwed up their DNS. I had changed my DNS settings to use Google's DNS servers for a long time because theirs had issues. More recently I rebuilt my PC and I hadn't changed the settings again so they still use Time Warner's DHCP settings to pull DNS server settings.

    • by Shadow99_1 (86250)

      I'll add that this was the case at around 6:30 AM EST when I was getting ready. The TW comment says "As of 6am ET services were largely restored", so it may be DNS was just the only thing left not working when I was still at home.

  • When I subscribed to Comcast a while back, there was a 4 day outage. By the second day, I found out that it was due to an attack on the DNS servers. So I went into the gateway and changed mine to Verizon's for the time being.
    • by whoever57 (658626)

      When I subscribed to Comcast a while back, there was a 4 day outage. By the second day, I found out that it was due to an attack on the DNS servers.

      Comcast runs DNS servers? Wow, perhaps I did not need to run my own for all these years! On the other hand, I have not had any problems at my home LAN due to DNS going down.

  • If it was just DNS, I'll be a bit dissapointed. With stuff like "Internet Backbone" I'm hoping for some major BGP messup.
  • by i kan reed (749298) on Wednesday August 27, 2014 @08:41AM (#47764513) Homepage Journal

    I mean, a couple hours isn't the worst TWC has ever done. Not even in the past year. This is just a wide problem. No one even bothers to note 12 hour fuckups on a regional scale.

    • by Charliemopps (1157495) on Wednesday August 27, 2014 @09:04AM (#47764669)

      The level of stress and anxiety on the admins during these outages would break most people. I used to do that and literally could not take it, so I switched to a less stressful job. At least with Cable companies it's only TV and Internet, I had to deal with phones as well. We literally had a man DIE during one of our outages because his family couldn't call 911. Even with data lines only all kinds of things break... Police stations data interconnects... hospitals lose access to databases... power companies might not be able to dispatch. Even small outages are a very big deal, and are taken very seriously. The residential side is irritating but not that big of a deal. It's the commercial side that makes it hard to sleep at night sometimes. The guys that do this, do care... if they didn't they sure as hell wouldn't put up with the miserable job.

      • We literally had a man DIE during one of our outages because his family couldn't call 911.

        Once again showing why analog is better for phones than digital. When was the last time you ever heard of an analog phone line not able to transmit a call? Now compare that to how many times you've have dropped calls or, in this case, no ability to make a call, on a digital line.

        But let's keep saying the move to digital is better than what we had, despite the overwhelming evidence to the contrary.
        • by Endlisnis (208453)
          I don't know what you mean by "analog", but unless you are talking about a single, physical wire connecting you with the 911 operator, then calls have been routed and core lines have been oversubscribed for decades (several). Even before rotary dial, you had to ask an operator to connect you. They probably had more outages than any modern system.
        • Or maybe this guy was going to die no matter what. This sort of thing happens. While our modern infrastructure is important and we should angst about major fuckups (especially if they involve TWC and friends), even the old unbreakable Bakelite phone / POTS system screwed up from time to time.

          The other thing to realize about the current 911 system (at least in the US) is that it is a fairly robust, fairly smart system. In the 'old days' when we had to make sure that paths were clear of T. Rex and raptors,

        • Once again showing why analog is better for phones than digital. When was the last time you ever heard of an analog phone line not able to transmit a call? Now compare that to how many times you've have dropped calls or, in this case, no ability to make a call, on a digital line.
          But let's keep saying the move to digital is better than what we had, despite the overwhelming evidence to the contrary.

          It was, in fact, and analog POTS line. POTS lines go down all the time... all over the place. The difference is, IP traffic is like an airliner going down... when it happens, it's big and happens to lots of people, so it gets more attention. POTS outages usually effect only a few hundreds people, but there are lots and lots of them all over the place all day long.

          ALL telephone service, being analog or or IP phone gets MUXed at the DSA and turned digital within 30k feet of your house anyway. The biggest prob

        • by Touvan (868256)

          Here (in upstate NY) Verizon runs the phone system - and they can't keep that thing running for 10 minutes it seems - if you can even get it (my appartment building is not wired for Verizon anything - phone or other). In my old place an hour away I had Verizon phone and DSL (don't get me started), that's where the phone kept going down. Cable is rock solid, and home cell service is a good backup (they even offer a home phone box that has a nice built in battery backup that can run for a few hours).

          On a side

      • I've been in a similar role. Not only do you have the stress from trying to fix the problem, you have idiotic middle management demanding constant status updates as they sit on top of you, watching your every keystroke because they literally have nothing constructive to do in situations that matter.

        I once lost it and snapped at a manager during an outage, in front of many suits in the "war room" and on the conference call. It went something like "I am actually trying TO FIX THE PROBLEM so BACK OFF!" H
  • "TWC's customer service reps are reportedly a bit overwhelmed by call volume at the moment"

    So it's just a normal day then? It seems that whenever I call one of these companies all I get is "due to unusually heavy call volume we are experiencing extended hold times".

    • Try calling their cancellation lines after an unannounced rate hike(like I was even using the TV cable, and I'd want to pay more for it).

  • by Cro Magnon (467622) on Wednesday August 27, 2014 @08:47AM (#47764557) Homepage Journal

    I haven't had TW for years, but when I did, it was newsworthy when it was actually working.

    • by Dins (2538550)

      I've been using them since early April of this year, and so far so good. Aside from this incident, I think there was one other instance where I was down for maybe 15 minutes.

      That said, they're overpriced and I don't trust them at all. If the Comcast thing goes through I'll be looking at my options, but unfortunately I don't think I have many good ones.

    • Exactly. If TW wasn't outright down, they were heavily throttled/over subscribed, or the modem needed to be rebooted, etc. Since switching to Google fiber in February, I haven't had a single outage.
  • Couldn't even ping 8.8.8.8, so I suspect it was more than just a problem turning URLs into numerical addresses.

    It was so weird to have problems and still have all the lights on the modem lit up (except standby).

    Unfortunately there's no novelty factor in having problems with it in general.

    • by thaylin (555395)
      And even then some host names would translate, just not others. Hulu was fine for example.
    • I work for an ISP. Trying to figure this stuff out from the consumer side is kind of pointless. When things break this bad, we're bringing things down and back up again faster than you can blink. Even if the problem is just DNS, we might end up turning all the interfaces on the core routers up and down. So your tests might have just been showing you the recovery efforts. Trust me, these things are never as simple as the right person noticing and flipping a switch. I guarantee you there were 20 or 30 bleary

      • It won't stop a lot of us from trying. Many of us base our careers largely on the ability to find and analyze problems, and it's almost instinctive by now.

    • The lights on the modem only mean you're talking to the CMTS on the other end of the neighborhood wiring, and that you were able to provision through DOCSIS.

      If they fuck the routing over, you'll have all the blinkinlights in the world, but no packets get where they're supposed to.

      • by unitron (5733)

        It's just that all the many many other times I've had problems, I could see that at least one of the lights that were supposed to be on wasn't on. This was the first time nothing "looked wrong".

  • I was forced (by moving) to switch from TWC to Comcast. I can tell you from experience that everything that is bad about TWC is at least 5x worse with Comcast. TWC fixed this in a matter of hours; if it were Comcast they would have billed the customers for the problem and it wouldn't be working until at least Monday.
  • Screw-up? (Score:5, Interesting)

    by geogob (569250) on Wednesday August 27, 2014 @08:54AM (#47764607)

    Did the NSA screw up again while tapping in into the TW infrastructure? I mean, if the manage to knock whole countries of the Internet by doing so, I wouldn't be surprised if they knocked a few ISP offline now and then.

    • Did the NSA screw up again while tapping in into the TW infrastructure?

      Yeah, NSA is busy upgrading all their gear. They took out Charter earlier this week. Today it was Time Warner's turn. They'll get to Comcast in a day or two.

  • by Joe_Dragon (2206452) on Wednesday August 27, 2014 @08:58AM (#47764637)

    Did they jump the gun on the comcast merger?

  • by nimbius (983462) on Wednesday August 27, 2014 @09:17AM (#47764771) Homepage
    Several rumors about this outage are circulating, so being as I work at Time Warner i thought id dispell them.

    1.Your call is important to us: this is a longstanding glitch in our callcenter software. at no point do any of us sincerely have any valued interest in you, or your communication.
    2.We are working to resolve the issue quickly: not likely. Tina started her lunch break at like, 10 am. im posting this from the beerpong tavern up the road. My manager is asleep in the datacenter. nick and rob from networking didnt show up today because they had like, 11 vodkas last night and that means theyll be on from vpn around 6 pm to sign their timesheet.
    3.This was caused by routine maintenance: again, a rumor. Dale and I wanted a kegerator in the NOC but we were already using the fridge in there for bottled beers. Instead we emptied a rack and made space. we're calling it cellar temperature but really its a bit warmer.
    4.You are all valued customers: I dont know where people get this idea. have you tried calling? seriously most of the phone tree is a circular loop. we have maybe 4 phones in this office and its incredibly difficult to order pizza and wings while you "people" keep inundating us with sob stories about your internet.

    anyhow sign up for the bundle, or 3 play or whatever it is the coked up hobo in marketing calls it these days. maybe that will help, or not, i dont know. that food truck with the salvadorian food is outside again and daddy needs a pupusa so, whatever.
  • Now I'm inclined to believe the typical explanation of "A wizard did it". God dammit, Dresden, I thought we told you to stay away from the datacenters.

  • It could also be that you chose to root your device, or to uninstall myriad floarware (bloat). Srsly, thought it was a server bug for TF2. Then for an hour-and-a-half it seemed as though the Android's networking must have gone terminal. Until getting back to the desk to find the pc in the same state. It's return was magical of course.
  • If we didn't hate TWC for other reasons, this would be dismissed as a quickly-corrected outage resulting from human error during some maintenance operation. But since we hate TWC, people make a big deal out of it and declare conspiracy and yell about bad customer service.

    Nothing to see here, people.

  • by Lumpy (12016) on Wednesday August 27, 2014 @10:10AM (#47765233) Homepage

    The fact they have ONE backbone connection is an utter and epic failure. Who designed their network because that person needs to be fired.
    Each major section or city needs it's own backbone connection. At least that is how we did it back in the old @home days, one failure can not bring down all services across the nation. Some cities had multiple backbones going into the area's OTN.

    • The fact they have ONE backbone connection is an utter and epic failure. Who designed their network because that person needs to be fired.

      That would be the MBA who said no we can't afford to buy that much connectivity, cram it all through one connection. 'cause we need more PROFIT! MOAR!

    • The fact they have ONE backbone connection is an utter and epic failure.

      Wait, you're not believing what the PRBot says, are you?

      • by Lumpy (12016)

        then you explain how a single point failure will take out service across the country. There is no way in hell that can happen if the network was designed properly.

        • by msauve (701917)
          "explain how a single point failure will take out service across the country. "

          If the routing protocols (e.g. BGP) get screwed up, it doesn't matter how many peer connections are up, the network doesn't know where to send the traffic.

          You haven't done any serious networking, have you?
          • by Lumpy (12016)

            Epic fail, routing protocols on 170+ POP routers all at the same time? Again, epic fail on design and implementation.

  • This will never happen again once Comcast buys TWC.

  • TWC's customer service reps are reportedly a bit overwhelmed by call volume at the moment

    Hopefully, their remaining customers, who evidently are willing to wait on hold for at least two hours under normal circumstances, will understand and will patiently wait on hold for ten hours for the privilege of being told "We're looking into it but there's nothing we can do right now."

    Luckily, TWC has competition in my area. Every time I see an ad from them that seems cheaper than what I'm paying to their competitor, I remember the horrifying customer service, and the urge to save money goes away.

  • A self-inflicted single point of failure to a mission-critical network? TWC, you are now qualified to be a government contractor!
  • I'm a TW customer, and I didn't notice the difference because TW always sucks (and so does their sole competition in the area. F oligopolies).

Dead? No excuse for laying off work.

Working...