Gmail Is Broken Right Now, One Day After a Massive Outage (techcrunch.com) 73
An anonymous reader quotes a report from TechCrunch: While it doesn't appear to be completely down like it was yesterday morning, we're hearing many reports from Gmail users that the email service is having major issues right now. Some users are reporting that Gmail is particularly slow, while others are reporting constant error messages. One TechCrunch writer, meanwhile, noticed that emails he was sending to Gmail accounts appeared to immediately bounce, with Gmail's server responding with an error reading "550-5.1.1 The email account that you tried to reach does not exist."
Google confirms the issues on its services dashboard, writing at 1:30 PM Pacific that they're impacting a "significant" number of users: "We're aware of a problem with Gmail affecting a significant subset of users. The affected users are able to access Gmail, but are seeing error messages, high latency, and/or other unexpected behavior." In a second update at 2:30 PM, Google says its teams are "continuing to investigate this issue"; as of 3:30 PM, the company says it expects the issues to be fixed by 4:00 PM while noting that time may change. Google says the problems have been resolved, although encrypted email service ProtonMail tweets that the email bouncing issue is widespread, with many emails sent to Gmail users bouncing permanently.
Google confirms the issues on its services dashboard, writing at 1:30 PM Pacific that they're impacting a "significant" number of users: "We're aware of a problem with Gmail affecting a significant subset of users. The affected users are able to access Gmail, but are seeing error messages, high latency, and/or other unexpected behavior." In a second update at 2:30 PM, Google says its teams are "continuing to investigate this issue"; as of 3:30 PM, the company says it expects the issues to be fixed by 4:00 PM while noting that time may change. Google says the problems have been resolved, although encrypted email service ProtonMail tweets that the email bouncing issue is widespread, with many emails sent to Gmail users bouncing permanently.
Re:Rubber meets the road (Score:5, Funny)
You mean the ones who passed the puzzle test? (Score:3, Interesting)
Being able to explain how many pingpong balls fit in an schoolbus doesn't solve real life problems like this outage.
Those Google job interviews were silly.
Not a single question about one's experience or ideas.
Just solving dumb puzzles about ridiculous theoretical problems.
Re: (Score:3, Interesting)
Re: (Score:2)
Most likely none of the managers had ever driven a forklift for more than
Re:You mean the ones who passed the puzzle test? (Score:5, Interesting)
Being able to explain how many pingpong balls fit in an schoolbus doesn't solve real life problems like this outage.
Those Google job interviews were silly.
Not a single question about one's experience or ideas.
Just solving dumb puzzles about ridiculous theoretical problems.
I found that having experience was actually a hindrance in the interview. My take is they only wanted smart people _without_ experience. Now we see what the results of that are.
Re: (Score:2)
Where all about to find out the real value of the Google engineers behind all of this.
Basically from what I can tell they've hired the most competent engineers around. I mean my ISP's email is down about twice a year by comparison.
Re: (Score:2)
They did not. They did hire smart people with a lack of experience. And it shows.
My online Outlook has been screwy also (Score:1)
I wonder if it's related
Outlook trying to deal with Gmail's outage (Score:2)
It's most likely related.
Maybe the spammers are DDoSing Gmail? (Score:2)
Why?
Hmm... Think fast....
Maybe the google decided to do something to break the spammers' business model?
Naw, that's too crazy a theory. "Live and let spam" is the motto.
Re: (Score:2)
Re:My online Outlook has been screwy also (Score:5, Insightful)
Its "all eggs, one basket" these days.
Re: (Score:2)
Re: (Score:3)
Pretty much. Mono-cultures are always a very bad idea. They are cheap initially though and today's "leaders" only care for short term profits.
Re: (Score:2)
Would be interesting to hear of a large scale email system that proved more reliable than Gmail. What are we talking about here, literally minutes of downtime over how many years?
I've never known a corporate system to be more robust, that's for sure.
Re:My online Outlook has been screwy also (Score:5, Informative)
We are talking 90 minutes complete outage yesterday and continues problems yesterday and today. FYI, for the (old) TelCo standard of 99.999% uptime, those 90 minutes represent the complete outage allowed for 176 years. This is not minor in any way.
Re: (Score:2)
But again, can you point to any email service that is actually more reliable than Gmail?
I doubt any of them get 99.999% uptime. That number also fails to account for the nature of the down-time, if it's "sorry we deleted all your mail but at least the outage only lasted 39 seconds" you might not care about the speed with which the service came back up.
I see to have slept through all this, I didn't notice any email issues at all over the last few days.
Re: (Score:2)
You mean, randomly discarding incoming mail after having accepted it? If you consider a piece of mail to be spam, you need to reject it in the SMTP dialog, not silently hide.
Re: (Score:2)
Re: My online Outlook has been screwy also (Score:2)
Not an email service, but life has existed for some 3.5 billion years on Earth. 100% uptime, so far!
Re: (Score:2)
Yeah, but we came pretty close to an outage 67 million years ago, and 252 million years ago; and maybe 550 million. Where would you be if one of those incidents had been a complete outage?
Re: (Score:2)
quick solution: (Score:2, Offtopic)
Re: (Score:2)
AWS Introduces New Chaos Engineering as a Service (Score:4, Funny)
Yup, emails to customers were bouncing today.. (Score:2)
Automated emails to customers (confirming orders, etc) were bouncing on random Gmail addresses (some fine, some would bounce), oddly they were bouncing almost immediately, rather than there being multiple attempts to deliver (as ought to be standard?) Error messages stated the email account didn't exist.
Re:Yup, emails to customers were bouncing today.. (Score:5, Informative)
oddly they were bouncing almost immediately, rather than there being multiple attempts to deliver (as ought to be standard?) Error messages stated the email account didn't exist.
I'm pretty sure just about all mail agents won't attempt multiple deliveries when an account is reported to not exist - and why would they?
Multiple attempts only happen for more generic and ephemeral delivery errors.
Re: (Score:1)
I'm pretty sure just about all mail agents won't attempt multiple deliveries when an account is reported to not exist - and why would they?
Multiple attempts only happen for more generic and ephemeral delivery errors.
Correct.
Re: (Score:3)
Per the RFCs, yes. In practice, it actually seems to depend more on the specific species of marketing weasel responsible for the sending of the emails. Job agencies seem to be notoriously bad for ignoring 5xx errors, and some of the larger email service providers seem to operate a pretty lax interpretation of the RFCs when it comes to 5xx's as well. Spammers obviously don't give a damn and/or never get to see the status messag
Re: (Score:3)
Works fine for me right now (Score:2)
Re: (Score:3)
I was having the problem today, but it was intermittant. I suspect that there was an SMTP server behind the load balancers that was misconfigured (maybe more than one). Based on the number of test e-mail messages I sent to myself that bounced, I would say that most of the servers were okay. I guess the good news is that some spammers likely removed me from their list :-)
Re: (Score:3)
The perfect storm. (Score:5, Funny)
Re:The perfect storm. (Score:5, Funny)
Why am I on this mailing list? Remove me please.
Re: (Score:2)
Re: (Score:2)
Youtube (Score:2)
Re: (Score:2)
Re: Youtube (Score:2)
I wish sites did this systematically during happy hour. The internet as a whole would be a better place. :)
If slashdot did this I'm certain my karma would be cleaner anyways.
Re: (Score:1)
Neither could Google. They bought it.
(quote refers to gmail)
Wikipedia disagrees with you, it says gmail was developed in-house and used at first for internal emails.
Re: (Score:2)
You're thinking of Hotmail (which for the first several years was hosted on Sun servers because WinNT 3.51 couldn't handle the load).
Re: (Score:2)
I'm fairly certain it was Solaris, since I don't think that Free BSD had a version for SPARC microprocessors yet.
Ah, we're both right.
https://en.wikipedia.org/wiki/... [wikipedia.org]
Hotmail initially ran under Solaris for mail services and Apache on FreeBSD for web services
How could this happen ? (Score:5, Funny)
I thought the Cloud and Kubernetes solved any issues of downtime ?
Re: (Score:1)
I thought the Cloud and Kubernetes solved any issues of downtime ?
It's precisely that thinking which causes downtime.
But before I get my pitchfork out, can you point to a document where Google has said the Cloud and Kubernetes offers 100% uptime? I can't help but compare gmail to my ISP's and my company's email server, both of which seems to be at least an order of magnitude if not more unreliable than gmail...
Re: (Score:2)
K8s makes development and deployment easier than hacking together some ssh/scp and shell scripts (or even Ansible). Obviously if you hardware, network, or product are fucked then you're going to have a bad time.
For what it's worth, Borg worked pretty well when I used it, but it's not a panacea. You still have to have quality software, quality processes, and quality infrastructure if you want to provide excellent service.
Re: (Score:2)
EarthLink too... (Score:2)
... from last week's Monday that lasted almost all day. Sheesh!
SolarWinds (Score:1)
Probably had a SolarWind attack
Re: (Score:1)
Probably had a SolarWind attack
Could be. Think Google would use someone else's monitoring?
If I could bet on this, I'd bet it was an UN-authorized change, or an authorized change that wasn't tested.
My gmail account was doing wonky things about two weeks ago. Wondered if they were updating the VM that my account is on.
Migration from Centos 8 to Centos 8 streams? LOL.
Is Gmail broken? (Score:2)
Intentionally broken 24/7/365 (Score:2)
Re: (Score:1)
It's also always broken. Try to bulk delete emails for example. They want you to buy more space so they crippled the bulk tools with intentional errors. This has been the case for at least several years.
Try connecting via a mail client, like evolution. You shouldn't have a problem deleting from there. In fact I like the whole experience better using Evolution than their web interface. I've never been a web interface type of guy for that stuff.
Gmail is a Juicy Target (Score:2)
Bad guys wish we all used something less effective. They are under constant attacks looking for a way to leak in and the countermeasures surely cause issues.
The root cause of these problems are sociopaths attacking our computer systems, not ineffective engineers.
We need to wake up, pass laws, sign treaties, and fund a transparent agency that locates and provides evidence to local authorities to enforce laws that protect us.
We need to demand that
Does anyone still use GMail? (Score:1)
Slow notifications (Score:2)
I've seen a few posts saying "It hasn't affected me."
Unless you don't use gmail or email anyone who does, how can you be sure?
I sent one email yesterday to two recipients, and got a "does not exist" error for one of them within seconds, and I resent it, and they got it and replied. Almost 11 hours later, I got a notification that the other recipient "didn't exist" either. So it may take quite awhile before you know.
Also, how do you know if you aren't getting emails others sent to you?
Gnail is working 100% with me. (Score:1)
meh (Score:2)
I only use gmail for things that aren't important like the various web site log-ins I have to have to because they don't let you check out as guest anymore. For family/work I have different, more private, email addresses.
Coincidences (Score:2)
Does Google use SolarWinds for monitoring, by chance? Kidding, of course (and it wasn't even a good joke).
No correlation, and no causation, but it would be kind of funny if there was a connection.
AOL (Score:2)
Maybe I'll pull out one of those AOL CDs I got in the snail-mail.
Re: (Score:1)
LOL, Really? Duh duh duh... The number you have reached...
Going to party like it's 1999?