Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×
The Internet IT Technology

An Amazon Server Outage is Causing Problems for Alexa, Ring, Disney Plus, and Others (theverge.com) 99

Posted by msmash from the closer-look dept.
Problems with some of the Amazon Web Services cloud servers are causing slow loading or failures for significant chunks of the internet. From a report: The company's widespread network of data centers powers many of the things you interact with online, so as we've seen in previous AWS outage incidents, any problem can have massive ripple effects. People started noticing problems at around 10:45AM ET. There are reports of outages for Disney Plus streaming, as well as games like PUBG, League of Legends, and Valorant. We've also noticed some problems accessing Amazon.com, as well as other Amazon products like the Alexa AI assistant, Kindle ebooks, Amazon Music, or Ring security cameras. The DownDetector list of services with spikes in their outage reports runs off nearly any recognizable name: Tinder, Roku, Coinbase, both Cash App and Venmo, and the list goes on.
This discussion has been archived. No new comments can be posted.

An Amazon Server Outage is Causing Problems for Alexa, Ring, Disney Plus, and Others More

An Amazon Server Outage is Causing Problems for Alexa, Ring, Disney Plus, and Others

Comments Filter:

  • Single point of failure..? (Score:5, Insightful)

    by xanadu113 ( 657977 ) on Tuesday December 07, 2021 @12:34PM (#62055865)
    Has Amazon become a single point of failure..?

    • Re: Single point of failure..? (Score:5, Insightful)

      by AutoTrix ( 8918325 ) on Tuesday December 07, 2021 @12:37PM (#62055883)
      For the people that depend on Amazon yes. This is caused by corporate decisions not wanting to spend on robustness. once they transition to cloud infrastructure they assumes it amazon's (or M$, Google, IBM, Oracle, etc) responsibility to make them robust.

      • Re: (Score:2)

        by hey! ( 33014 )

        ... and people who depend on services that depend on Amazon too. These people may not have chosen to be dependent upon Amazon, they may not even be *aware*.

        It's kind of like weather used to be. For most of human history, when you got up in the morning you had no idea if a storm was rolling in that would interrupt or even destroy your work. In a world dependent upon interdependent data services you can never be certain whether your day will be interrupted by an outage that apparently came out of nowhere.

    • Re:Single point of failure..? (Score:5, Insightful)

      by aaarrrgggh ( 9205 ) on Tuesday December 07, 2021 @12:37PM (#62055889)

      Only if you are doing it wrong.

    • Re:Single point of failure..? (Score:4, Funny)

      by olsmeister ( 1488789 ) on Tuesday December 07, 2021 @12:41PM (#62055901)
      Hasn't affected me. Slashdot is still up.

    • How VERY dare you. They're too big to fail!

    • Re: (Score:2)

      by Z80a ( 971949 )

      It's a lesson to be learned that will not be learned

    • Re:Single point of failure..? (Score:5, Insightful)

      by Drew M. ( 5831 ) on Tuesday December 07, 2021 @01:54PM (#62056085) Homepage

      Has Amazon become a single point of failure..?

      AWS has only one availability zone down right now, US-EAST-1. With proper redundant design across multiple availability zones, this outage isn't an issue. All it does it show us the companies who are bad at reliability engineering.

      • Re:Single point of failure..? (Score:4, Informative)

        by outz ( 448278 ) on Tuesday December 07, 2021 @05:46PM (#62056771)

        Has Amazon become a single point of failure..?

        AWS has only one availability zone down right now, US-EAST-1. With proper redundant design across multiple availability zones, this outage isn't an issue. All it does it show us the companies who are bad at reliability engineering.

        US-EAST-1 is a REGION. AWS's global services were down. The global service outage (api/s3/cf/r53 etc) affected all regions.
        Even if you had a hot/hot setup that spanned from EAST-1 to WEST-2 it would have had issues because it's a global service outage. There would be nothing routing your traffic between regions, cause it's a global services outage...

        If it's anyone's fault for not having a proper redundant design it's AWS... they're too dependent on EAST-1.

      • Drew... you may have incorrect/confused understanding between region and avaiability zones. Redundancy design shouldn't be done across availability zones. It's designed across regions. Having service in different AZs are short term fault tolerance only within the same region and can cover issues like building fires, power outage, local weather. When one building goes down, other building in the same region can pickup the slack and with the help of ASG they can scale up their capacity and take over the work

    • A lot of stuff was still up.

      The good thing about these occasional Amazon outages is they remind people cloud services are not invincible, to have backup plans, so that Amazon does not become a single point of failure...

    • There were a couple of legit businesses I was trying to access today and their websites were failing with peculiar generic errors. Still not up.

    • It doesn't matter for like 99.9% of businesses. If Coinbase, Slack, and Amazon.com are seeing issues, none of your customers blame you when foo.com goes down along with them.

  • Why Alexa keeps on having trouble with my timers.

    • The only timer I really use regularly on Alexa is my morning alarm that plays via my Echo Dot on the nightstand.

      The problem I keep having is really odd though. At one point in time, I remember playing with an "action" that would play music for 30 minutes or so after an alarm was silenced. Ever since then, I have this weird glitch where every Thursday, it insists on playing alt. rock every Thursday when I silence the morning wake up alarm. Can't find anyplace where such a thing would be scripted or trigger

      • Try the web interface at: alexa.amazon.com
        I have a great routine setup for mornings, one bedroom lamp turns on then 30 seconds later the other lamp comes on at both a minimum intensity. Then every 5 seconds it increases by 1% up to 10% brightness then it plays my prefered local radio station over every device in the house. (I live alone, so this is not a dick move.) This is a relatively nice way to wake up. At the time I absolutely NEED to jump out of bed another routine kicks in and reads me the local we

        • Re: (Score:2)

          by King_TJ ( 85913 )

          Yeah... nice morning routine there. I'd do something similar except in my case, my house has ceiling fans as the light fixtures in both bedrooms and the living room. So without spending a bundle to replace those with "smart ceiling fans" (which all cost WAY too much for what you get), I'm stuck not really being able to automate those well. (Could do a smart light-switch but then the pull-string on the ceiling fan would be controlling whether or not the fan was coming on with the light and at what speed --

          • Same sort of issue with my bedroom lights. That is why it is the bedside lamps that turn on. My house was built in 1951, so a lot of outlets are poorly placed for modern use, with light switches in inconvenient places, smart bulbs/outlets and Alexa have made my house 100% more liveable.

        • I would rather depend on an old fashioned clock radio with a battery backup for my wake up alarm.

            This is critical if that alarm means the difference between staying hired/getting fired.

          • I would rather depend on an old fashioned clock radio with a battery backup for my wake up alarm.

            This is critical if that alarm means the difference between staying hired/getting fired.

            If your employment is one late day away from ending, I suggest either you are a crap employee, or you work for a crap employer.
            Also my phone has a last second drop everything and run to work alarm.

            • Not everyone has the luxury to not work in such a crap enviroment, at least in the short term.

                When it comes to cannot miss no matter what events, I have at least two different alarm clocks set to go off within a couple minutes of each other.

      • It's truly amazing these things are as reliable as they are, when you really think about it.

          Your strange glitch could've been as simple as a config file becoming corrupted on the server end. Usually glitches like this are very rare but they can happen.

    • Why does Alexa need Internet access to execute timers? Android does that all locally on your phone.

  • I don't consider smart speakers (aka telescreens) being broken to be a problem. Same goes for nodes of a private surveillance network like Ring.

    If I could push a button and melt all of those devices into silicon slag, I would.

    • It's more problematic for me. I use Ring to surreptitiously monitor my elderly mother who has Alzheimer's. Without Ring, it's a lot more difficult to keep her from doing something that could be harmful to herself or others.

      • It's more problematic for me. I use Ring to surreptitiously monitor my elderly mother...

        If the ip cameras don't work without a call to the mothership you bought a shit camera. I have six cams watching parents and property and one gifted "must login" type. Don't depend on broken internet of crap that must have app and remote accounts. If our moms like mine just tell her the cameras are a clock and paint some numbers on it, be creative.

        • Sadly, it's not as simple as that. If your parents suffer from Alzheimer's, then you know how unpredictable it can be. Each person is different, requiring different techniques to deal with them.

          Additionally, I have relatives who, unlike me, haven't been in IT for 40 years. Ring's interface makes it easy for someone else to monitor her if I'm unavailable or I just need a break.

          Could I come up with a home-brew solution? Absolutely. Could I explain to my siblings and descendants? Not in a million years.

          • Could I come up with a home-brew solution? Absolutely. Could I explain to my siblings and descendants? Not in a million years.

            Uh, sending them a url to web page on a camera through a firewall is too hard for them? Most residential ip addys don't change much anymore, you make it one click. Nothings perfect, but Ring and its ilk suck ass. If your lazy just say "I'm lazy"

            • I would caution you against making such remarks while you are ignorant of the specific circumstances. While I'm too old to care, most people would find your remarks insulting.

              • You're really getting to enjoy the "jerks of the internet" in this post, aren't you. I get where you're coming from. Some people just have to throw shade.

        • " I have six cams watching parents and property and one gifted "must login" type. Don't depend on broken internet of crap"

          Except you are still using the internet albiet with as few points of failure as possible. If the internet goes down, what do you do?

  • Internet Snow Day! (Score:3)

    by kiviQr ( 3443687 ) on Tuesday December 07, 2021 @12:55PM (#62055935)
    enjoy

  • We are affected too... (Score:5, Interesting)

    by sentiblue ( 3535839 ) on Tuesday December 07, 2021 @01:00PM (#62055951)
    I'm on a conf call with our war room. It's up to AWS to resolve their networking issue and we're pretty much powerless until they do. But we're still on the call assessing impact and planning course of action once recovery is inbound. Last time they had such an outage wasn't too long ago.

    • Re: (Score:3, Informative)

      by Drew M. ( 5831 )

      I'm on a conf call with our war room. It's up to AWS to resolve their networking issue and we're pretty much powerless until they do. But we're still on the call assessing impact and planning course of action once recovery is inbound. Last time they had such an outage wasn't too long ago.

      It's only one availability zone (US-EAST-1) that's down. Being down right now mostly means that you didn't build your service to be redundant across availability zones. Amazon isn't going to do that for you.

      • Re:We are affected too... (Score:5, Informative)

        by MachineShedFred ( 621896 ) on Tuesday December 07, 2021 @01:51PM (#62056075) Journal

        The only issue with that statement is that any global service is homed in us-east-1, including parts of the management console and APIs. I'm trying to run Terraform against resources in us-west-2 and it's taking an excessively long time to check the status of existing resources, because the APIs are running dog slow.

        When us-east-1 has a problem, AWS has a problem globally.

      • Re:We are affected too... (Score:4, Informative)

        by sentiblue ( 3535839 ) on Tuesday December 07, 2021 @03:35PM (#62056379)
        I understand that only US-east1 is down and we do have service in Sydney, US-east2, US-west* as well. But for us this is having devastating/catastrophic consequences because lots of APIs reside in Virginia. We rely heavily on AWS automations and this is a big problem. As far as AWS concerns, if their US-EAST-1 region is down, it has a global impact regardless if their customers have redundancy.
      • BTW... redundancy is built across regions, not across AZ. The AZ is only to deal with localized issues such as a building on fire, or local weather flooding. Redundancy is always across regions with data replication so that when catastrophes such as earthquakes, massive floods, terrorism happen... we'll switch services to another geographic location. In this case, it's not even a local issue, it's simply a device-based, networking-related issue that AWS has already identified. They just need time to resolve

  • Why is your doorbell in the cloud? (Score:3)

    by Fly Swatter ( 30498 ) on Tuesday December 07, 2021 @01:06PM (#62055963) Homepage
    It should be near the front door.
    • In my case, it's because there's a Cam in my Ring doorbell. I use Ring to surreptitiously monitor my elderly mother who has Alzheimer's. Without Ring, it's a lot more difficult to keep her from doing something that might harm herself and others. I particularly need the exterior door cams. She lives on a busy street and has been known to sit down with her legs in the street to do infinitesimal yard work. She's also gone through neighbors' trash. I use the cams to know when she's exited the house, in th

      • She needs to be in a memory care facility. If you are not close to her physically, all these cameras are doing is giving you a view of your mom becoming a hood ornament.

        • I would caution you against giving medical advice without being aware of the specific circumstances. While I'm too old to care, most people would find such remarks insulting.

          My family and I are not caring for her in a vacuum, but rather with the guidance of medical professionals and specialists. I'll take their advice over a stranger on the Internet.

        • There are some issues with your post.
          "She needs to be in a memory care facility" - you forgot the "IANAL" part - this is just your opinion based on incomplete facts.
          "If you are not close to her physically" - she only needs to escape from under supervision for a couple of minutes to go from - lets say - safely into bed, into the middle of a street.
          "all these cameras are doing is giving you a view of ..." - there might not be enough outside visual coverage for that.

      • Have you considered using local cameras? IP cameras are dirt cheap these days. Put them on their own vlan, poke a hole or two in your router, and with a dynamic IP service and something like IP cam viewer on your phone you've got basically the same functionality with none of the cloud BS.

        • We've considered it. While I could certainly come up with a home-brew solution, my other relatives didn't spend 40 years in IT as I have. I could never explain to them how to access such as system.

          Ring makes it simple for them. It's not perfect, but I can at least take breaks from monitoring without worrying that my other relatives would be utterly confused.

          • I could never explain to them how to access such as system

            My parents are in their 70s and victims of never having to learn any technology due to me living with them until my mid 20s. Even they can figure out how to open ip cam viewer on their phone. It's literally as simple as amcrest camera-->point IP cam viewer at the cameras IP address (you can even configure it once and pass on the configuration via QR code)-->done. Mix in some static DHCP reservations, port forwarding, dynamic DNS services, and non-standard ports to taste if you want it accessible remo

            • My mother is beyond learning any technology. It's why we can get away with surreptitious cameras.

              I have absolutely considered your suggestions prior to settling on Ring. There are numerous benefits that the company and its products bring to the table beyond simple monitoring.

              There are also my own points of failure inasmuch as local Internet goes down far more frequently than AWS.

              This AWS outage as an absolute earthquake to be sure. I have a good friend who's a consultant who has so far spent 20 hours wit

        • Ironically, I bought a system just to avoid cloud-based restrictions. Unfortunately, after investing in a dozen cameras and NVR, the manufacturer decided to make the login process cloud-only and the app cloud-only or same local network. So, despite having a properly configured VPN I am stuck with their cloud BS.

          Fsking Ubiquiti.

  • "Outage is Causing Problems for..." (Score:5, Funny)

    by Cognivore ( 301236 ) on Tuesday December 07, 2021 @01:06PM (#62055965)

    "...Alexa, Ring, Disney Plus..."

    At least it's not affecting anything important.

  • You know you're living in the future when you can't direct your robotic vacuum cleaner to programmed areas due to a "cloud outage". After staring at the blank app for a couple of minutes I realize I'm lucky my brush and dustpan work fine without a needing to contact a service gateway.

    • Re: (Score:2)

      by cusco ( 717999 )

      Indeed, the inconvenience reminds me of how incredible it is that this stuff can be automated. So it's down for a few hours, when I was little this stuff was imagined for 2050. That it's up over 99% of the time was unimaginable, a long distance phone call wasn't even 90% reliable, international communications were probably half that.

  • cloud fall down go boom (Score:2, Insightful)

    by drwho ( 4190 )

    rainy day cloud everywhere, blue skies, no cloud in sight...oh when will the storm roll in.

  • This is why I prefer my digital assistants to do all their processing in-device. If they canâ(TM)t work off grid, then I donâ(TM)t want them for home automation either. Just imagine not being able to turn on your lights because of an internet outage.

    • Based on our SlashIDs, we are approximately the same digital generation, whatever that means. Probably because of that, we were also in complete agreement on this issue until a couple of years ago. Looking carefully over your statement, we are probably still mostly in agreement. However, having developed a web application that was hosted completely in-house, realizing that I would never have a day of freedom from the office while the application remained in-house, re-developing the same web application on A

      • Most applications can be dual-homed in cloud and local in fail-over mode without too much pain. Competing with truly distributed applications that are real-time and load balancing is a bit more work though of course. Key is to understand your needs.

        As for home automation, Proxmox VE and an external NAS for backups makes for a pretty solid environment and if you want more robust the appliance route like Universal Devices ISY994 is nearly bulletproof.

      • I don't mind the cloud being used, unless it is an exclusive requirement. If it is an exclusive requirement, then if the servers go down, the internet connection drops or the business goes out of business, then you are screwed. More so, for the last point. For the last point we have seen Nest devices that stopped working, because of a policy change at Google and anyone who bought the SmartHalo device for the bike found themselves with a paperweight when the company went bust.

        As to "I don't use home automati

  • All it is, is publicly available services running from datacenters. We've always had that, companies have had that for decades. However individual companies were on the hook for data management / retention / redundency like replication to another datacenter in case of failure. Well, data centers offered managed servers with replication.

    Then eventually they just offered their own email server on their own managed servers where they setup redundancy and replication, and called it a cloud service. Like office

  • Push notifications via SNS were broken (Score:3)

    by bustinbrains ( 6800166 ) on Tuesday December 07, 2021 @04:37PM (#62056557)

    If there's one positive thing to come out of this: Amazon SNS is broken when these types of outages occur. That means push notifications for many apps get broken as a result. Quieter phones and computers = happiness for most users.

    Some queued notifications have started showing up, so it's soon back to the useless deluge of push notifications.

  • The "cloud" is just someone else's computer. (Score:3)

    by MacMann ( 7518492 ) on Tuesday December 07, 2021 @05:18PM (#62056689)

    Putting some service on the "cloud" doesn't mean it's not running on a computer. It means it is running on a computer that is in a place out of your direct control, and that someone else is being paid to maintain it. This sill means that failures can happen, and the failures still have an effect on you. This can make you quite powerless in resolving any issues.

    I'm quite annoyed with how many products are dependent on the "cloud" to function. If there's an interruption with my internet service, or a larger issue like this, my stuff stops working. It would not be so bad if the people making stuff let people know that an internet connection is required for certain functions. I get it that a TV needs internet for showing YouTube videos, but it should not need it so I can change the channel from a WiFi device on the same network.

    The "cloud" is just someone else's computer. If something was dependent on my own computer then I can at least fix it. If something is dependent on some computer I don't own, can't touch, and have no option to replace if it's broke, then that something is inherently broken. The only fix is to replace that item with something that isn't dependent on some computer I don't own.

    • Re: (Score:2)

      by cusco ( 717999 )

      This sill means that failures can happen, and the failures still have an effect on you.

      Duh. If you had kept it all in-house you'd still have failures, just of a different type. Do you keep 99+ percent uptime? (Don't lie, you do not.) Do you also have one of the largest IT security organizations on the planet securing your systems? Do you have the ability to back up your own servers internationally? You go to the cloud for abilities you can't provide or can't afford to do on your own.

      • Duh. If you had kept it all in-house you'd still have failures, just of a different type. Do you keep 99+ percent uptime? (Don't lie, you do not.) Do you also have one of the largest IT security organizations on the planet securing your systems? Do you have the ability to back up your own servers internationally? You go to the cloud for abilities you can't provide or can't afford to do on your own.

        Cloud services are great for many many people. Such as people that don't have the resources to hire full time staff for web services. Disney is not one of those people.

        I have two big complaints. First is how often products I buy reliant on computers outside of my control. If I buy a security camera then I want the option to be able to store the video on my own computer, not just to the computers run by the people that sold me the camera. Can I keep 99% uptime on my computers? Not likely, but if the on

  • Interestingly, I discovered 20+ websites that had a Trojan virus communicating to an AWS IP address. I let Amazon know at 9:50 am, and 40 min later their network dropped. Perhaps they tried to shut them down and got more than they expected?
  • AWS is big but apparently not too secure, which is probably one reason the Pentagon gave the Jedi contract to Microsoft instead of AWS. Of course Jeff Bezos sued them without any evidence to prove his allegations that Trump was behind the award to Microsoft and now the Pentagon has to regroup. For our nation's security I hope AWS is left out again.

Slashdot Top Deals

Invest in physics -- own a piece of Dirac!

Close