Forgot your password?
typodupeerror
Google Cloud

Google Apps Suffering Partial Outage 150

Posted by Unknown Lamer
from the eternal-recurrence dept.
First time accepted submitter Landy DeField writes "Tried accessing your Gmail today? You may be faced with 'Temporary Error (500)' error message. Tried to get more detailed information by clicking on the 'Show Detailed Technical Info' link which loads a single line... 'Numeric Code: 5.' Clicked on the App status dashboard link. All were green except for the Admin Control Panel / API. Took a glance 2 minutes ago and now, Google mail and Google Drive are orange and Admin Control Panel / API is red. Look forward to the actual ...'Detailed Technical Info' on what is going on." The apps dashboard confirms that there is a partial outage of many Google Apps. The Next Web ran a quick article about this, and in the process discovered there was an outage on the same date last year.
This discussion has been archived. No new comments can be posted.

Google Apps Suffering Partial Outage

Comments Filter:
  • by Quakeulf (2650167) on Wednesday April 17, 2013 @10:18AM (#43472143)
    It's like the start of a new tradition! Yay! :3
    • We'll have to see if the reason for this failure is the same as last years.
    • by Anonymous Coward

      As they teach you in HCI class, consistency is key for an ideal user experience. So this is just another small change by Google to improve usability.

      • by treeves (963993)

        HCl class? You mean chemistry class? That's like calling biology class 'frog class' isn't it?

    • I've said it before and I'll say it again: so far not a single large "cloud" provider has been anywhere near as reliable as either my (relatively small) hosting provider, or my own servers.

      When they can offer uptime comparable to what I already have, I might consider using them for important things. Not until.
      • by lgw (121541)

        If by "reliable" you mean "no failures for any users anywhere" then sure. If your service has 300 million users, you're going to have downtime for some of them more frequently than if you have 300 users. But if by "reliable" you mean "no failures for a randomly chosen user" then they're already quite good (and technology is just starting to mature in this space - it will get better).

        Of course, most of the problems we've seen on /. have been for the free services. I'd hope the paid services are better.

        But

        • "f by "reliable" you mean "no failures for any users anywhere" then sure. If your service has 300 million users, you're going to have downtime for some of them more frequently than if you have 300 users."

          No. By "reliable" I mean "not being down in any given year, for any length of time, for any significant percentage of your user base".

          And EVERY major cloud service so far has failed that standard. Every one that I know of: Microsoft Azure, iCloud (which is based on Azure), AWS, Google Apps, Gmail... the list goes on.

          • by lgw (121541)

            OK, so, what then? I've never worked anywhere that didn't have regular service outages for stuff like mail. The fact that most of the outages were planned doesn't make then any less down. And even the unplanned ones were at least once a year - usually due to being too cheap with the corporate internet connection.

            Reliability is easy at very small scale - just run a single server with some OS that doesn't need regular reboots. But the cost per user is remarkably high, in comparison.

            • "Reliability is easy at very small scale - just run a single server with some OS that doesn't need regular reboots. But the cost per user is remarkably high, in comparison."

              That may be true but it has nothing to do with my point. My point was: until the reliability is up to what I consider to be a reasonable level (and I consider reasonable to be the same as the services that I currently use), then I'm not putting my important business "in the cloud".

              It may be that kind of reliability in "the cloud" is prohibitively expensive or difficult right now. I don't care. Unreliable is unreliable.

      • by exomondo (1725132)

        When they can offer uptime comparable to what I already have, I might consider using them for important things. Not until.

        What do you classify as 'comparable'?

  • by Anonymous Coward

    4/17/13 9:09 AM
    We are continuing to investigate this issue. We will provide an update by 4/17/13 9:55 AM detailing when we expect to resolve the problem.
    This issue is affecting less than 0.007% of the Google Mail user base. The affected users are unable to access Google Mail.

  • I'm sure many companies who switched their productivity to Google apps are panicking... and wondering ...
    • by poetmatt (793785)

      exactly how many are "many", when it's less than 1%?

    • by DrEldarion (114072) on Wednesday April 17, 2013 @11:14AM (#43472745)

      They're probably thinking "wow, this response is much faster and the downtime much shorter than when we ran our own services".

      Nobody has ever sold cloud services with a guarantee of 100% uptime. It is, however, almost certain to be better than the vast majority of companies' homegrown solutions.

      • by joebok (457904)

        I'm sure no cloud contract has a 100% uptime guarantee, but it is "sold" as the perfect solution all the time.

        I try to remind people that the cloud is not filled with magic beans. Sometimes it is just what is needed, sometimes it isn't; it depends!

        • by idontgno (624372)

          I'm sure no cloud contract has a 100% uptime guarantee, but it is "sold" as the perfect solution all the time

          And that, in a nutshell, is the difference between a signed SLA and marketing.

          I'm sure it's inconvenient and unfortunate and maddening to the many affected, but it's not as if your internal fully-owned fully-self-managed infrastructure never have a backhoe network outage or a widely-deployed OS patch-o-death that knocks your operations into a loop for hours or days.

          The cloud is just another variant

      • Hey, that's what I was thinking this morning!

        I've got better things to do than baby servers along, so it sure is nice to have this be someone else's problem. Things were fixed in less than 30 minutes for us. Maybe 6 people at our company (roughly 60 people) even noticed that there was a problem.

        And I got 30 minutes without anyone pestering me by email. Glorious.

  • by ArcadeMan (2766669) on Wednesday April 17, 2013 @10:22AM (#43472185)

    They sent an email explaining the cause of the... oh wait.

    • So what would be wrong with that? An email notice that it is down would be absurd, but an email explaining the cause makes perfect sense, since they will have it working again at some point if it isn't already.
      • If... your e-mail doesn't work... how would you read the... um, said e-mail?
        I tell you, there's MAGIC at work!

        But I give you that: it DOES make some sense, because if your web-based e-mail isn't working, maybe the back-end works; maybe you can read e-mails on your mobile device; maybe POP push works, or IMAP, or whatever.

  • by El Lobo (994537) on Wednesday April 17, 2013 @10:24AM (#43472223)

    In the mean time...I'm working in my desktop machine, saving to my own disk (with automatic backup to my server AND my machine at work) and getting my mail into my own server not depending one ounce on any cloud services. Life is good.

    • by SJHillman (1966756) on Wednesday April 17, 2013 @10:36AM (#43472353)

      Sounds like you have a comparatively high number of potential points of failure compared to the cloud services.

      • by neminem (561346)

        Yeah, but the difference is, if something breaks, you can fix it. (Unless the something is your internet connection, I suppose. But then, if your internet connection died, you wouldn't really be able to use services in the cloud, either.)

        • by bufke (2029164) on Wednesday April 17, 2013 @10:49AM (#43472493)

          This is the real benefit with Google Apps. When Google Apps is down...I don't have to do anything! Life is good.

        • by MBGMorden (803437) on Wednesday April 17, 2013 @11:11AM (#43472711)

          Yeah, but the difference is, if something breaks, you can fix it.

          It might make one feel like they're taking a more "active" role in the problem, but you're likely to spend as much time fixing your homegrown solution as Google is fixing Gmail. With the cloud solution when something goes wrong though SOMEONE ELSE fixes it.

          Besides - Gmail actually has an "offline" mode available for Chrome users. For those really that worried about downtime they can use that.

          • by idontgno (624372)

            Gmail actually has an "offline" mode

            In which you can compose a new email, and look at old email you've received, but exchange no actual email?

            Yeah. I know they didn't invent offline mode. But that doesn't change the fact that offline mode in an email program is like no-engine mode in a car. You can sit on a comfy seat out of the rain and listen to the radio, but you're not doing the one thing people actually want a car for: driving.

        • by MozeeToby (1163751) on Wednesday April 17, 2013 @11:14AM (#43472751)

          Yeah, but the difference is, if something breaks, you can fix it.

          Alternatively, if something breaks in Google's servers, it gets immediate attention by people who know a hell of a lot more than I do about maintaining a server. And things are multiply redundant, making something breaking comparatively unlikely.

          • by someones (2687911)

            my servers at home are more stable and reachable that google ;)

            Its like the 3rd time in 2 years google fails.

            • by AvitarX (172628)

              It's .007% of their users, and last time it was a different small percentage. If you assume all of google is down when 1 in 10000 users are, it sounds bad. The reality of it is that It's not.

            • by kqs (1038910)

              Your servers never need to reboot due to hardware failure or kernel updates? Your internet never goes down? Your power never goes down? Your DNS never gets hosed? You never have storms that knock down power and data lines? Sweet!

              I run servers at my house too (for friends, non-commercially) and I'd guess that 100% of my users lose access to their email and web sites for ~8 hours a year, or about 3 9's of reliability. I bet that heads would roll at Google if they had that much unreliability.

              I switched m

        • by rev0lt (1950662)
          Let me rephrase that:

          Yeah, but the difference is, if something breaks, you need to stop whatever your doing to fix it.

          It is not always an advantage... and I do prefer to work with my own resources than to rely on 3rd party "service" providers, but there are a lot of cases when choosing a provider is the sane approach.

        • by keytoe (91531)

          Yeah, but the difference is, if something breaks, you can fix it.

          I used to feel the same way as you and ran my own mail server. Over time, I realized that I was spending a tremendous amount of time playing email administrator cat and mouse with spammers and script kiddies.

          Since I wasn't being paid for any of this, at some point I realized that paying* Google to manage it for me was actually a good deal. They're significantly more reliable than I ever was, it's some other admin frantically fixing problems a

      • by someones (2687911)

        so in the cloud there is no smtp? no imap? no http? ... do you expect things magically turn into cloud transfer protocol or what?

        In the cloud you just have MORE potential points of failure: namely all that you have locally and all that run the cloud server, the loadbalancers, ... which is much more points of failure you can ever grasp.

        The diffrence is, who gets blamed. But if your boss doesnt see you working while noone in the company gets any work done, dont expect to get a raise too soon.
        On the other hand

        • so in the cloud there is no smtp? no imap? no http? ... do you expect things magically turn into cloud transfer protocol or what?

          You've pretty much summed up the understanding of every PHB who was ever sold on a cloud solution.

        • by CastrTroy (595695)
          The difference is that a well designed cloud solution will have the necessary redundancy built in such that failure is less common and less catastrophic. The mere existence of loadbalancers (plural) shows that they are much better able to deal with a hardware failure than most in-house mail server solutions.
    • Yup, I know the feeling of using my own setup.

      Right now, I'm waiting for a shipment from NewEgg that should see if I can recover everything from my last computer. (They seem to die of jealousy whenever I get a replacement, usually without waiting for me to transfer everything over.)

      Naturally, the important stuff is backed up (to a cloud service), and I have that, but there's a few weeks of minor stuff I'd like to have back.

  • by sandytaru (1158959) on Wednesday April 17, 2013 @10:28AM (#43472275) Journal
    This took down one of our clients who pay for Google apps. So it's not just the freebie users who got affected on this, hence Google's rapid response.
    • yup same here - our organization of +50,000k users was affected. we pay for their suite and just switched over this year.
      • by rnturn (11092)

        ``our organization of +50,000k users was affected''

        So who at your company gets dinged when 50,000 employees are sitting on the hands while Google fixes their problem? Surely some monetary value can be assigned to a loss of productivity this widespread. Or does management take advantage of Google's outage by calling impromptu staff meetings?

        • If things were planned the right way, nobody should get dinged. Of course, things are rarely planned the right way.
          But let's say they were.
          Then there's a set of SLAs which tells the customer "we ensure 99.99% uptime. If we don't, then we deduct X dollars per minute from what we're billing you" or something like that. The customer has their expectations set and when there's an outage, it's accounted for. Furthermore, an SLA involves a theoretical loss of productivity (which is expected).

          This "witch hunt" of

        • by acoustix (123925)

          ``our organization of +50,000k users was affected''

          Or does management take advantage of Google's outage by calling impromptu staff meetings?

          And how would you notify your workers of these meetings?

        • Email is used here as tool and enabler for the actual work in the org. I dont think much productivity was lost, the staff just went back to doing their primary functions. Not to be enigmatic but it is equivalent to say they went back to making widgets. Of the 50k staff, ~40k of them do not sit in front of a PC all day.
    • by dickens (31040) on Wednesday April 17, 2013 @11:30AM (#43472975) Homepage

      Confirmed here ... it was down for about an hour including the admin control panel.

      One nice thing about multi-tennancy is problems get attention immediately.. they simply cannot be ignored.

    • Hey, the FBI needed its live Gmail intercept.

  • 4 it's an unkucky number in Japan. 17 it's also an ulucky number in Western countries. Coincidence? we at TV-Show-On-Whacky-Therories don't think so.
    • by invid (163714)
      17?
    • by alexhs (877055)

      4 it's an unkucky number in Japan. 17 it's also an ulucky number in Western countries.

      Amusingly, it is for the same reason.

      In japanese, 4 is pronounced "shi" in the On reading, which is an homonym of "death".
      In latin numerals 17 = XVII, which is an anagram of VIXI, which is latin for "I lived", implying "I'm dead".

    • by rwise2112 (648849)

      4 it's an unkucky number in Japan. 17 it's also an ulucky number in Western countries. Coincidence? we at TV-Show-On-Whacky-Therories don't think so.

      According to the Wikipedia entry for 17:

      Described at MIT as 'the least random number', according to hackers' lore.

      In Italian culture, the number 17 is considered unlucky.

      When viewed as the Roman numeral, XVII, it is then changed anagrammatically to VIXI, which in the Latin language it translates to "I have lived", the perfect implying "My life is over.

      The fear of the number 17 is called 'heptadecaphobia' or 'heptakaidekaphobia'.

      • by Pope (17780)

        Laffo at 17 being an unlucky number in Italy. You have to rearrange the damn letters first to even make the words that are supposed to be "bad."

        Man, superstitions are the dumbest thing in so many ways. At least the "shi" thing has some sort of explanation that makes a bit of sense.

    • by gman003 (1693318)

      Coincidence? we at TV-Show-On-Whacky-Therories don't think so.

      Oh, you're with the History(tm) Channel?

    • by TimMD909 (260285)
      Four is unlucky in Japan because it can be pronounced "shi". Seven is also unlucky because that can pronounced "shichi". Why is this bad? "Shi" is a homophone for "death". Hence, they "invented" alternate ways to pronounce both. For #4, it's "yon". For #7, it's "nana". Simple as that, the entire Japanese society was able to cheat death. (BTW, why does /. not allow me to add kanji to this post? I feel less otaku...)
  • Was unable to log into Youtube starting around 9pm Pacific. The log in prompt would just redirect in a loop and eventually reload the homepage with out ever giving any login dialog.
  • google gmail is always having outdage on this day, may be they are doing upgrading or Htaccess [htaccessredirect301.com] problem
  • The tiniest open-source violin plays for you. (or at least it would if you had a local copy of tiny_opensource_violin.flac)

  • I felt a great disturbance in the Force, as if millions of chromebooks suddenly cried out in terror and were suddenly silenced.
  • Sorry for advertising my own product, but pretty much on topic here. :) Buy two (cheap) servers from completely different networks / data center providers, and keep them replicated with http://wiki2.dovecot.org/Replication [dovecot.org]. You can set up MX records to both of them, and use DNS to switch between the replicas for IMAP/POP3 as needed. Either one of the data centers can die and your mail won't stop working. Or keep one of the replicas in local network and your mail keeps working even if your internet connectio

Our informal mission is to improve the love life of operators worldwide. -- Peter Behrendt, president of Exabyte

Working...