Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×
Windows Microsoft IT

Global Computer Outage Impact Vastly Underestimated, Microsoft Admits 64

Posted by msmash from the how-about-that dept.
Microsoft has revealed that the global computer outage caused by a faulty CrowdStrike software update, which impacted numerous major corporations, affected far more devices than initially reported, with the tech giant stating that the previously announced figure of 8.5 million affected Windows machines represents only a "subset" of the total impact. Microsoft has refrained from providing a revised estimate of the full scope of the disruption.

The revelation comes as the technology sector continues to grapple with the fallout from the incident, which occurred 10 days ago and led to widespread disruptions across various industries, prompting Microsoft to face criticism despite the root cause being traced back to a third-party cybersecurity provider's error. Microsoft clarified that the initial 8.5 million figure was derived solely from devices with enabled crash reporting features, suggesting that the true extent of the outage could be substantially higher, given that many systems do not have this optional feature activated.

Further reading: Delta Seeks Damages From CrowdStrike, Microsoft After Outage.
This discussion has been archived. No new comments can be posted.

Global Computer Outage Impact Vastly Underestimated, Microsoft Admits More

Global Computer Outage Impact Vastly Underestimated, Microsoft Admits

Comments Filter:

  • We're not faring well with skynet (Score:5, Insightful)

    by Z80a ( 971949 ) on Tuesday July 30, 2024 @03:15PM (#64667724)

    This software that caused this problem is advertised as an AI driven protection tool that just happens to have full access to the ring 0 of your OS. and what we did? we installed it on several very important machines.
    Every computer that crashed 10 days ago is a computer where the owner is basically willing to install a skynet like software on it.

    • Every computer that crashed 10 days ago is a computer where the owner is basically willing to install the real-time protections required today because of how toxic and risky it is to connect any computer to ANY network.

      FTFY. Now before you respond, really sit and think how you are also guilty. AI has fuck-all to do with the real-time requirements and immediate patch schedules required in order to even stand a chance against non-infection. Firewalls, routers, A/V updates, malware updates, spyware updates, the real-time maintenance demand grows every day, as do the zero-day risks and attack vectors.

      Attack those putting all their eggs in one basket all you want. Just don’t be a fucking hypocrite.

      • Re: (Score:2, Interesting)

        by Anonymous Coward

        Just don’t be a fucking hypocrite.

        the irony is rich here.

      • Re: (Score:1)

        by Anonymous Coward

        If CrowdStrike's buggy nonsense is "required", why did most enterprise Windows systems keep working on the 19th?

        • If CrowdStrike's buggy nonsense is "required", why did most enterprise Windows systems keep working on the 19th?

          Todays buggy nonsense brought to you by Crowdstrike.

          Tomorrows buggy nonsense brought to you by [your A/V/malware/real-time protection provider]

          Doesn't really matter the vendor. Shit can Happen to any of them. The actual required buggy nonsense is all the real-time protection shit we have to install on a Windows machine today just to use the damn thing and not get infected faster than a virgin in a whorehouse on nickel beer night.

          With the ever present risk of someone attacking and compromising the ultimate

    • This software that caused this problem is advertised as an AI driven protection tool that just happens to have full access to the ring 0 of your OS. and what we did? we installed it on several very important machines.
      Every computer that crashed 10 days ago is a computer where the owner is basically willing to install a skynet like software on it.

      The "AI" stuff is a rebrand of ML, which was the term crowdstrike used between 2011and ~ 2018. So, not really a skynet.

      The AI/ML stuff happens in Crowdstrike's Central servers (named so from 2011 ~ 2015, now called cloud). So, not on every owner's machine.

      The protection tool need to be at ring 0 to observe suspect behaviour in ring 0 and ring 3

      There is no API to do that safely from ring 0 or 3 because, when microsoft proposed one, the EU said it could stiffle competition if Microsoft had direct access to th

    • Re: (Score:2)

      by AmiMoJo ( 196126 )

      It has nothing to do with AI. The "AI" in these products is just some basic heuristics, the kind of thing that AV software has been doing since the 90s.

      The problem was entirely man-made. There is no similarity to Skynet or allowing AI to control anything.

  • Wrong focus. (Score:5, Insightful)

    by geekmux ( 1040042 ) on Tuesday July 30, 2024 @03:15PM (#64667728)

    Gotta love how Microsoft just keeps that Crowdstrike name rolling off their global-disaster tongue in the middle of debates about over-reliance on one vendor..

    ..as if the single-source OS vendor that corruptly dominates the business desktop space have room to even open their mouth about problems related to putting all your eggs in one basket. Microsoft has brought us far more pain in the aggregate. Corruption at every level has simply trained us to tolerate far more abuse.

    Perhaps this is a good time for business to talk about ALL of their over-reliance on one vendor.

    • They're just happy that this time, it wasn't them that caused the outage!

      • Except that they did. If an application can bring down the OS, then the OS is wrongly designed and has to bear the responsibility.

        If course that won't happen. MS will weasel out of it by blaming the application. In fact, they already have and they are getting away with it.

      • Re: (Score:2)

        by Teun ( 17872 )
        Microsoft DID cause the outage.
        When a widespread used OS needs 3rd party applications to work safely there is a fundamental problem with this OS.

    • Re: (Score:2)

      by AmiMoJo ( 196126 )

      Given that the same Crowdstrike software crashed Linux systems, requiring a boot into single user mode to fix, it doesn't seem like Microsoft/Windows is the issue here.

      The issue is shitty anti-virus software and mandates to install it.

      The companies that are claiming on business continuity insurance now can only do so because they followed the rules set out by the insurers, which state that they need to have AV software installed.

    • Re: (Score:1)

      by xpyr ( 743763 )
      Because this is not Microsoft's fault. CrowdStroke also released updates for Linux previously that caused the Linux Kernel to crash: https://www.theregister.com/20... [theregister.com]

  • Scavanger hunt time (Score:5, Interesting)

    by mcfatboy93 ( 1363705 ) on Tuesday July 30, 2024 @03:38PM (#64667810) Homepage

    Personally, I only had to physically locate ~65 PCs and was done later that afternoon of the incident, several of my colleagues were not as lucky. A large organization with older buildings will be finding these devices for years. An unused classroom, a PC in a closet, a machine in a basement has never seen the light of day, there is a non-zero chance that no one knows where these devices are, that they still effected or that they even exist.

    I agree with the writer, there is absolutely no way the actual number is that low, and we will be finding them for a very long time.

    • Re: (Score:2, Informative)

      by Anonymous Coward

      yup.. initial count was 5,000+ machines at my org... we are still hunting down about 300 that have proven difficult to find/fix..

      no idea what the total is now, but its definitely way over 5,000..

    • Re:Scavanger hunt time (Score:5, Interesting)

      by geekmux ( 1040042 ) on Tuesday July 30, 2024 @04:05PM (#64667906)

      An unused classroom, a PC in a closet, a machine in a basement has never seen the light of day, there is a non-zero chance that no one knows where these devices are

      The forgotten PC in the closet or the abandoned machine in the basement, is more the fault of the business owner/IT department half-assing their inventory documentation. (We’re all guilty of it, but we should be honest as to why.)

      Given the sheer amount of SaaS in use today and licensing costs charged per “active” machine, it floors me that businesses can even afford to be that sloppy with inventory. There’s a damn good chance the machine that hasn’t seen the light of day, is still racking up monthly licensing costs. And that’s just one machine they forgot about. Monthly checks between hardware inventories and what is being reported to AD and/or A/V systems would mitigate that risk, and likely help a lot towards keeping track of machines.

    • If it takes somebody years to locate a device that is no longer working, did it really need to be on in the first place ?

  • Microsoft should be made to replace every affected system with an Apple.

    • Re: (Score:3)

      by jsonn ( 792303 )
      Why do you blame Microsoft? If your org bought ClownStrike, they should sue them for gross neglect.
      • Did you mean -->> gross negligence ?
      • CrowdStrike would not have been able to wreak havoc had Microsoft not allowed them to do so in the first place. Microsoft are the owners of the OS - as such, they are ultimately responsible for whatever is going in kernel space.

        • Re: (Score:2)

          by jsonn ( 792303 )
          Please check your facts. Microsoft does not have that level of control as results of antitrust agreements with the EU.
      • If CrowdStrike could look at someone else's code and then write protections to make it more secure, don't you think the company that wrote the software should have done that in the first place ?

        • Re: (Score:2)

          by jsonn ( 792303 )
          Have you considered that ClownStrike is doing nothing nothing of the kind and that Microsoft actually offers comparable. Seriously, you have no idea what this product even is.

  • I remember reading all the stories / news reports about this for several days, but personally never saw any impact from it. I haven't traveled by air in nearly two decades, and apparently the businesses I interacted with during that time (financial services, mainly), some of which were reported to be experiencing problems, seemed to have things under control. My employer uses a lot of Microsoft systems but apparently doesn't use the affected Crowdstrike software, so it was business as usual for us.

    • Re: (Score:2)

      by kackle ( 910159 )
      My sister was traveling at the time and her hotel had to escort each guest to the room because the computer-controlled door locks didn't work, requiring some sort of master key for entrance. AND, they really couldn't casually leave the room once there because that would require waiting in line again in the lobby for a human escort to take them back to their room!
  • > Microsoft has revealed that the global computer outage caused by a faulty CrowdStrike software update

    CrowdStrike AV software wouldn't be required if the underlying OS wasn't so defective.

    • > Microsoft has revealed that the global computer outage caused by a faulty CrowdStrike software update

      CrowdStrike AV software wouldn't be required if the underlying OS wasn't so defective.

      Try to use a linux machine in a PCI or HIPPA environment (among many others like it) sans security software (antivirus and the like) and see how that goes for you...
      Then try again with a Mac Sans security software in the same environment (PCI, HIPPA, finanacial, legal...).
      Bonus point if you go with another BSD (macOS being BSD-ish)

      Please, do not forget to come back latter and tell us all about it.

      • Try to use a linux machine in a PCI or HIPPA environment (among many others like it) sans security software (antivirus and the like) and see how that goes for you...

        Very well, in my case: I have had Internet-facing Linux systems for decades now without any problems so far, never having needed any commercial packages to keep them so - in particular, no antivirus and the like.

        • Re: (Score:3)

          by gweihir ( 88907 )

          Same here. Linux simply is not anywhere as vulnerable as Windows is, given halfway competent system administration.

          • Same here. Linux simply is not anywhere as vulnerable as Windows is, given halfway competent system administration.

            Again:

            HIPPA? PCI? Financial? Law? Fortune 500 (and therefore, valuable target)?

            Or you mean your lonesome personal webserver and mail?

            • Re: (Score:2)

              by gweihir ( 88907 )

              You actually do not need to have AV, for example, to satisfy a regulator. You just have to do proper risk analysis and then accept the residual risk. On Linux, in most situations, AV _decreases_ system security.

              I do IT and IT security audits in a regulated environment (among other things). I actually have a bit of a clue how that works.

        • Try to use a linux machine in a PCI or HIPPA environment (among many others like it) sans security software (antivirus and the like) and see how that goes for you...

          Very well, in my case: I have had Internet-facing Linux systems for decades now without any problems so far, never having needed any commercial packages to keep them so - in particular, no antivirus and the like.

          HIPPA? PCI? Financial? Law? Fortune 500 (and therefore, valuable target)?

          Or you mean your lonesome personal webserver and mail?

      • Try to use a linux machine in a PCI or HIPPA environment

        Hard to take advice on this topic from someone who can't spell HIPAA correctly.

    • Re:Global outage caused by faulty MICROS~1 Windows (Score:4, Informative)

      by Tony Isaac ( 1301187 ) on Tuesday July 30, 2024 @05:27PM (#64668146) Homepage

      You mean, like Debian? https://www.theregister.com/20... [theregister.com]

      CrowdStrke has crashed Linux too.

    • Re: (Score:2)

      by gweihir ( 88907 )

      Indeed. And Crowdstrike would not have crashed those machines if they did not need a kernel driver because MS does not offer an API for the things the Crowdstrike module does does. Talk about bad architecture. And that bad architecture (and the overall insecure OS) is 100% on Microsoft.

       

  • Comment removed based on user account deletion

    • Does 'Crowdstrike' sound like the codename for some villainous plot to be thwarted by 007?

      Crowdstrike CROWDsources suspected threat behaviour, analyses it using ML (nowadays called AI) and uses that to protect all the machines under its umbrella

      In principle, the faster the samples arrive to Crowdstrike central servers (nowadays called cloud), the faster can they analyze them using ML and develop countermeasures to STRIKE the threaths down.

  • Because every Windows version since Windows 7 (and perhaps the older versions) sends telemetry to Microsoft when your computer is powered on, and online. So, all they had to do was look at their data.

  • The next time the boss tells you to... (Score:3)

    by TomGreenhaw ( 929233 ) on Tuesday July 30, 2024 @05:53PM (#64668220)
    ...promote that untested update before the weekend, just say we cannot afford a CrowdStrike. They obviously did not test that update properly.

    But let's be honest. How many of us have succumbed to pressure to send out something half-baked.

    • Re: (Score:2)

      by gweihir ( 88907 )

      How often does it have to be said? This update was automatic with no way to delay or refuse it.

  • we need building codes for software (Score:5, Insightful)

    by peterww ( 6558522 ) on Tuesday July 30, 2024 @06:17PM (#64668290)

    The building code, and mandatory inspections, exist because we know that leaving "doing the right thing" up to contractors would lead to disaster.

    We need to do the same thing for critical software or this bullshit will keep happening, forever.

    • Re: (Score:2)

      by Tom ( 822 )

      Yes, but hero programmers are everywhere. The type that doesn't let anyone tell them how to do things because they know best.

      And sadly, Open Source does foster such people. With github and co making it trivial to fork a project and make your own version. Which is a good thing but also a curse.

      This shit will continue to happen for a few decades at least, because we're still in the hero phase and software is still innovation driven.

    • Back in the 1980s (90s?), Microsoft software had the disclaimer that the software could not be used for medical or other critical systems. This was to avoid the regulation that you are seeking.

      Since Microsoft software is being used in those settings, now might be the time to propose such legislation again... but don't be surprised when you don't get your wish. Money is too concentrated and will corrupt any legislation that is proposed. Game over bro.

  • You do not own your computer (Score:4, Informative)

    by RUs1729 ( 10049396 ) on Tuesday July 30, 2024 @06:38PM (#64668342)
    At least not when running Microsoft Windows: Microsoft own the OS and everything in it, they can access your computer whenever they want and however they want, and they can do with it as they wish. So shut up, and put up.

    • Re: (Score:2)

      by Seahawk ( 70898 )

      They can, but not legally... (Not that I disagree with the sentiment of not running Windows :))

  • The post seems like a complete misreading of this passage: "We can leverage the unique stack and attributes of this crash to identify the Windows crash reports generated by this specific CrowdStrike programming error. It’s worth noting the number of devices which generated crash reports is a subset of the number of impacted devices previously shared by Microsoft in our blog post, because crash reports are sampled and collected only from customers who choose to upload their crashes to Microsoft." If
  • It's kind of like how emergency vehicles are a small fraction of all motor vehicles.

Slashdot Top Deals

Your password is pitifully obvious.

Close