Millions of Smartphones in 11 Countries Were Taken Offline Yesterday by an Expired Certificate (theverge.com) 34
Ericsson has confirmed that a fault with its software was the source of yesterday's massive network outage, which took millions of smartphones offline across the UK and Japan and created issues in almost a dozen countries. From a report: In a statement, Ericsson said that the root cause was an expired certificate, and that "the faulty software that has caused these issues is being decommissioned." The statement notes that network services were restored to most customers on Thursday, while UK operator O2 said that its 4G network was back up as of early Friday morning.
Although much of the focus was paid to outages on O2 in the UK and Softbank in Japan. Ericsson later confirmed to Softbank that issues had simultaneously affected telecom carriers who'd installed Ericsson-made devices across a total of 11 countries. Softbank said that the outage affected its own network for just over four hours.
Although much of the focus was paid to outages on O2 in the UK and Softbank in Japan. Ericsson later confirmed to Softbank that issues had simultaneously affected telecom carriers who'd installed Ericsson-made devices across a total of 11 countries. Softbank said that the outage affected its own network for just over four hours.
Re: (Score:2)
You fucking nerds need to figure out
Having worked at Ericsson for 5 years, I can tell you this: the nerds are not the problem. The problem is that in Ericsson, every engineer is managed by multiple managers. Instead of the normal pyramid where you have let's say 10 engineers and 1 manager, Ericsson has an upside down pyramid. Each engineer has multiple managers that each want their own projects to take priority. Not to mention that the engineer:manager ratio is waaaay off.
Not to mention the fact that promotions take place based on speaking
We need to talk about your TPS reports (Score:2)
We need to talk about your TPS reports
Working as designed. (Score:2)
OTOH these are the people tasks with keeping your phones and conversations "safe." What OTHER minor things have they overlooked? (Everyone, not just them. They're just at the head of the line right now.)
Re: (Score:1)
Stuff like this has me thinking twice ... (Score:2)
... about using a major player smartphone. Eyeing and considering Sailfish and old Blackberry on a regular basis.
You misunderstand. (Score:5, Informative)
This wasn't Ericsson brand cell phones going offline because of this certificate. This was *ENTIRE CELL NETWORKS* going offline because the backend hardware's certificates were being rejected because they expired without replacement certificates in place.
Having a different brand of cell phone doesn't help if your phone is rightly rejecting expired certificates from the cell network, or if the cell network is not authorizing new cellular connections because it can't connect to servers.
This was likely a backend problem, either with authentication servers for the basestation/router licenses, or some centralized bckend service that was actually web based.
Re: (Score:2)
Re: (Score:2)
Things like "Certificate renewal" and "DNS renewal" should have reminders (or errors, or whatever) in your monitoring tool,
Some of my clients have every piece of infrastructure monitored that can possibly go wrong, and some that probably can't.
Meanwhile, our local ILEC will happily tell you that they don't need to monitor anything because customers will call and let them know what's out.
The difference? The ILEC is not subject to competitive pressures; they benefit from a monopoly grant from the State and a
People, rotate your certificates! (Score:2)
It canâ(TM)t be considered critical enough. At work we have three teams that get alarms of expiring certificates, just to make sure it doesnâ(TM)t fall through the cracks. The next phase will be complete automation of the renewal process against the internal CA, with a review before the final deployment of the renewed cert.
Re: (Score:2)
It probably did...
SMS messages were either not arriving at all, or erroneously being declared as such (I pity the poor soul who may have got a bunch of "Is o2 up yet?" messages from me because I was being told they hadn't been delivered)
Voice calls ... not so sure, but the system would have been overloaded due to people resorting to speaking to each other on their handsets
Re: (Score:2)
It only broke data connections, and possibly SMS. So you could still make a call.
Re: (Score:2)
Re: (Score:2)
"Alright Alice, now for the next 200 MB file for payroll, enter the following ..."
Thanks Ericsson (Score:2, Insightful)
Time to switch to Huawei.
Security theater again (Score:1)
Goes to show our security systems are more hazardous than the bad guys.
And we still want self driving cars? (Score:1)
As a thought experiment, what will it look like when this happens to a network of connected self driving cars?
I say "when", not "if". I can't think of a way that this doesn't happen someday.
For starters, emergency vehicles will not be able to get through the resulting traffic jams after a few million cars come to a stop.
On the bright side, you'll probably still be able to read the ads on the entertainment system
When will these amateurs learn? (Score:2)
We need to start doing IT with professionals. You know, with people that actually have a clue what they are doing. Sure, they will cost more individually, but overall the whole thing will get a lot cheaper as such major pathetic fuckups will become very rare.
Total Inability To Support Users Phones (Score:2)
n/t