Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
IT Technology

It's Not Just CrowdStrike - the Cyber Sector is Vulnerable (ft.com) 90

Posted by msmash from the concentration-risk dept.
An anonymous reader shares a report, which expands on the ongoing global outage: The incident will exacerbate concerns about concentration risk in the cyber security industry. Just 15 companies worldwide account for 62 per cent of the market in cyber security products and services, according to SecurityScorecard. In modern endpoint security, the business of securing PCs, laptops and other devices, the problem is worse: three companies, with Microsoft and CrowdStrike by far the largest, controlled half the market last year, according to IDC.

While the US Cyber Safety Review Board dissects large cyber attacks for lessons learned, there is no obvious body charged with analysing these technical failures to improve the resilience of global tech infrastructure, said Ciaran Martin, former head of the UK's National Cyber Security Centre. The current global outage should spur clients -- and perhaps even governments and regulators -- to think more about how to build diversification and redundancy into their systems. Further reading: Without Backup Plans, Global IT Outages Will Happen Again.
This discussion has been archived. No new comments can be posted.

It's Not Just CrowdStrike - the Cyber Sector is Vulnerable More

It's Not Just CrowdStrike - the Cyber Sector is Vulnerable

Comments Filter:

  • Re: (Score:1, Troll)

    by account_deleted ( 4530225 )
    Comment removed based on user account deletion

  • CrowdStrike is a POS. (Score:5, Informative)

    by methano ( 519830 ) on Friday July 19, 2024 @12:01PM (#64638295)
    The university where I work requires me to run Crowdstrike on that platform that wasn't supposedly affected. It's a CPU hog, constantly grinding away in the background. It's its own little denial of service virus to anyone using it. If you haven't typed in the last 5 minutes, it starts up and takes over. As soon as you return, it takes part of a minute before you can get back in control. I've complained but to no avail. Maybe now, they will allow me to take this garbage off my machine and get back to being productive.

    • Are you sure that's Crowdstrike? Sounds like normal Windows 10 behaviour to me (I have not had to use Win 11 up to now).
      Oh, and I have just looked at the NASDAQ 100. Biggest drop of the day is something called Crowdstrike Holdings, down 12.22% as of a couple of minutes ago.

      • Re: (Score:2)

        by methano ( 519830 )
        I "run Crowdstrike on that platform that wasn't supposedly affected". I have an iMac.

    • This brings back terrible memories.

      In a previous job I complained that it was putting too many admin teams on the boxes. I didn't like that because the updates happened whenever.

    • I've worked on CrowdStrike for a decade and never seen that issue. It's one of the easiest agents I've used. It's misconfigured then and not the agent itself. It's your IT department that's the POS.

  • Monopolies in General (Score:5, Insightful)

    by RossCWilliams ( 5513152 ) on Friday July 19, 2024 @12:23PM (#64638387)
    It ought to make people think about the vulnerability of allowing monopolies and other concentrations of any kind. What if everyone wasn't using Microsoft or had three other similar systems to provide resilience when one went down?

    • Simple, you'd have multiple possible sources of error. Just like how RAID1 doubles the chance of hardware failure. The thing about diversifying is that you may half the number of people impacted but you double the chance that either half is affected.

      That's the strange world we live in now. The world shut down for a couple of hours. It made the news. But each individual company shutting down for other unrelated issues happens too, and it doesn't make the news. (Well it does when it's an airline, and that act

  • Phasing out Windows (Score:4, Insightful)

    by RitchCraft ( 6454710 ) on Friday July 19, 2024 @12:28PM (#64638411)

    A good start.

    • And doing what? Run an OS that magically doesn't have any software with administrator privileges? Or do you somehow think that other OSes can't be made to kernel panic when privileged software screws up, in which case you really should never ever give an opinion on OS security again - for other people's sake.

      • And doing what?

        1. Not giving any software permission to connect to the Internet and download updates on its own. That's a recipe for a security disaster like just happened.

        2. Not implementing single points of failure in systems which are supposed to be high-availability. It's not just about hardware. Implementing Crowdstrike on all of a company's machines made it a single point of failure. You have to use enough different pieces of software to maintain operation if it catastrophically breaks.

        That applies to Linux and Wind

        • I can see you've never managed large groups of systems before. If you think anyone is going to use multiple different enterprise security solutions in one organisation you'll be quickly fired for wasting money. Bonus points for thinking that redundant servers can be managed with different unique configurations. I'll bet the biggest risk to an organisation would be you, when you go and do a failover test.
           

          • I worked in the devops group for one of the largest Linux deployments on earth. Literally millions of servers. We didn't use different security solutions but we did not did not did not allow them to update all on the same day.

          • Also, you don't build "failover" systems in a credible high-availability design. You build active/active systems because that's the only way to catch mistakes before the secondary systems are needed.

            If you design a standby system and expect it to be in good working order when the primary system suffers an outage, you've already lost.

      • Hey, I agree, but Windows being closed source and being a massive clusterfuck enables these kinds of problems with ease. It will only get worse as Microsoft continues to migrate everyone to SaaS. I hope this is a massive wake up call for companies and governments alike. MICROSOFT IS NOT YOUR FRIEND.

    • The CrowdStrike issue has nothing to do with Windows specifically. If they had deployed a bad driver on Linux systems, those systems would have been crashing just as much.

      • Re: (Score:1)

        by MeNeXT ( 200840 )

        This is Windows speak for I don't know what I'm talking about. Who is they and how can they deploy a bad driver on a production Linux system? Where did they get that driver? How incompetent is this admin?

        I haven't had anything like this happen to me in the last 30 years on Linux, FreeBSD or Mac.

        • They = CrowdStrike

          The implementation of their (CrowdStrike's) Windows agent was created as a driver. That driver had a bug.

          If CrowdStrike released a Linux agent, there's no reason to suppose that agent couldn't also cause the OS to crash, given enough permissions (as the Windows agent was given).

          "They" (CrowdStrike) wrote "that driver."

          How competent is "the" admin? "The" admin has nothing to do with this. "The" admin was instructed to install CrowdStrike. CrowdStrike's agent requires "God" permissions. "The

        • Re: (Score:2)

          by gweihir ( 88907 )

          Fully agree. There is people that do not understand Windows and the alternatives. These usually think Windows is great and everything else is worse. Then there are people that actually do understand the alternatives. A great success for Microsoft marketing, but a bad outcome for the human race overall.

      • Re: (Score:1)

        by gweihir ( 88907 )

        Au contraire. It has everything to do with Windows. Without Windows such a product would not even exist.

        • If that's true, why does CrowdStrike offer Linux protection? https://www.crowdstrike.com/pa... [crowdstrike.com]

          • Re: (Score:2)

            by gweihir ( 88907 )

            Simple: Because a lot of IT is clueless, and when they have it on Windows, they think they also need it on Linux. But Linux alone would not have created that market.

            • Ah, I see, because obviously only a dumb IT person would ever choose Windows in the first place. Got it.

              Guess what, Linux is no paragon of security. Remember Heartbleed? https://en.wikipedia.org/wiki/... [wikipedia.org] That was on Linux, and it was open source, and the bug was pretty dumb (but also not easy to spot).

              I realize that Heartbleed was an exploit, and this was just a bad deployment. The point is, every complex OS has the potential to go very, very wrong.

      • I just read a comment that says their software uses eBPF on Linux where it is mature, but not on Windows where it is not. If that is true then you are flat wrong. But I haven't bothered to do any research.

        • I don't know how mature eBPF for Windows is, but it certainly exists. https://github.com/microsoft/e... [github.com]

          Whether eBPF is more mature on Linux or not, if you give software enough permissions, it can get itself into trouble, whether it's on Windows or Linux / eBPF. There is no way a framework like eBPF can protect against *all* risks.

  • ...testing on one machine before widespread deployment?

    • ...testing on one machine before widespread deployment?

      I figured my host _was_ the one machine, since it started dropping out before the error's official time. ;-)

    • ...testing on one machine before widespread deployment?

      Not any longer. Now it's simply turn on auto updates because they're highly reliable, and away you go.

      Or, for those old enough to remember, "Just set it and forget it."

    • That, and testing your rollout to progressively larger production instances, rather than all at once!

    • Re: (Score:3)

      by gweihir ( 88907 )

      That would be solid engineering. It is a cost factor, and the usual idiots will buy your stuff anyways because they have no clue. Better invest more effort into marketing. Well, until the house of card collapses.

  • one vendors software can fuckup lots of system with an bad update.

    maybe if there was more compaction then we may have less systems all useing the same software.

  • Of course (Score:2, Offtopic)

    by DrXym ( 126579 )

    The fundamental issue is antivirus software has to write dangerously low level, privileged stuff in dangerous languages like C and C++. If Microsoft has sense they'll look at the calamitous fallout of this issue and figure out ways mitigate it. e.g. by mandating use of safer programming languages like Rust for writing drivers, or mandating their own AV mini driver interface that obviates the need for vendors to write so much code.

    • Thank you, that was the stupidest thing I've read today. Rust offers no more protection against semantic errors than you get in C and C++. Telling the computer to do the wrong thing with valid code isn't impacted by how safe your memory allocation is.

      • Re: (Score:2)

        by DrXym ( 126579 )

        Uhuh.

        Actually Rust stops a heap of issues at compile time that C/C++ allows through to runtime - null / freed pointer references, buffer over / under flows by design. If you don't know that, I suggest you stop writing dumb comments and labeling other people as "stupid". Especially in the face of a major outage that caused billions of dollars in losses and was clearly caused by a very low level and fundamental issue.

        • It's not clear that mandating Rust would have helped avoid this problem.

          • Re: (Score:2)

            by DrXym ( 126579 )

            Never said it would have. I said mitigate, i.e. lessen the risk and said "e.g." before Rust to demonstrate what I meant by that. Mitigate means reducing the chance of it happening again. The reality is that a lot of instability in drivers - perhaps the majority - is from the language they're written in. NULL is considered the billion dollar mistake and C/C++ don't give a damn about it, or many other issues. And then there is the quantity of code required to implement the driver that could be reduced with pa

  • Tech is hard. Pay me to do it. You can trust me. (Score:5, Insightful)

    by Big Hairy Gorilla ( 9839972 ) on Friday July 19, 2024 @01:01PM (#64638535)
    Microsoft has perfected how to promise "solutions" for dummies. Just pay the money and press the button and you'll get what you want. Repeatedly promising people they'll get what they want, at some point in the future, just give me money, is the classic con. Don't worry. We'll take care of EVERYTHING. But it's taking a bit longer and you'll have to pay a bit more. Once you're balls deep, there's no easier or cheaper option. But this is where it ends up. They own you now. woopsie.

    Also, every business and government was balls deep about, what? 20 years ago?
    Approximately the last time anyone thought strategically for themselves.
  • I haven't commented on SlashDot since about 2006. Probably the last time I logged in as well. I don't work in IT anymore and haven't since 2009. I don't miss it. Not even the more palatable environment of the time period. However, like all of us, I have watched the insane gaps that have been *Constructed* between long-standing, de-centralized, redundant, fault-tolerant, and fail-safe systems to cloud computing services. IT departments of all sized have, by and large, dismissed the need for risk assessment a
  • So many dopey CEOs these days. If they aren't navel gazing and feeding you positive psychology bs, they are moving too fast and uncritically. I worked in validating systems in the pharma industry. It was hard, hard work. One aspect is "ruggedness" - will this work on multiple platforms, in multiple countries, with multiple users? It's an expensive approach, but in the end, you get systems that take into account every potential flaw. It's hard work, but the idea is REAL SIMPLE
  • Yes, we've all heard of Solwarinds. Oh, and Microsoft OSes. Oh yeah, and AWS and their constant widespread outages.

  • It's about compliance, not security (Score:5, Insightful)

    by Chelloveck ( 14643 ) on Friday July 19, 2024 @02:37PM (#64638819)
    The problem, IMHO, is that endpoint security software isn't actually designed to secure endpoints. It's designed to tick a checkbox on some compliance form that says endpoint security is being used, so don't sue us when it all goes pear shaped.

    • Re: (Score:2)

      by gweihir ( 88907 )

      Yes, clearly. Most stupid instance I have ever seen was a completely isolated environment, where they had to put in a backdoor because external audit told them they must have AV in there. When we then told them they not did not have an isolated environment anymore, they did not understand what we were saying.

    • Re: (Score:2)

      by kwalker ( 1383 )

      The problem, IMHO, is that endpoint security software isn't actually designed to secure endpoints. It's designed to tick a checkbox on some compliance form that says endpoint security is being used, so don't sue us when it all goes pear shaped.

      I wish it was that simple. I'd love to just install this on a system, have it spawn a little tender app that checks into the mothership and completes that regulatory requirement.

      Don't forget to add a "use an obscene amount of system resources for no goddamn reason" footnote to that checkbox.

      Also "dogpile on busy systems" and "increase your Cloud Spend for more resources". Just use tiny print.

  • This is a wake up reminder that you shouldn’t have an internet connected privileged
    binary running on your production systems. What was a bad update could have easily
    been a massive adversary backdoor. A third party vendor will always be the weakest link.
    Isolate critical systems
    Christopher Stanley
    @cstanley

  • FT wants $1 to access the article which no doubt signs you up for $75/month subscription unless you cancel through some arcane process. Hard NO.

    • Whenever you see these stories submitted by "an anonymous reader" that link to a paywalled web page, you can safely assume the blogger who wrote the paywalled story submitted it.

  • "Products" do not cut it (Score:4, Interesting)

    by gweihir ( 88907 ) on Friday July 19, 2024 @03:50PM (#64639022)

    Products do not make your IT secure (or safe or reliable). For that you need skills and insights. IT products these days are as crappy as the vendor thinks they can get away with.

  • Suggestions (Score:3)

    by jd ( 1658 ) <imipak@yahooMENCKEN.com minus author> on Friday July 19, 2024 @04:41PM (#64639154) Homepage Journal

    1. Require higher standards of kernel code, to reduce the risk of malware being able to compromise security.

    2. Require good instrumentation and security APIs, so that security software can have clearly defined and well-bounded risks that can be clearly understood, and to ensure that modification of the kernel is not required for full security.

    3. Require that commercial product vendors be liable under lemon laws for products that could have never worked as intended.

    These three would go a long way to mitigate the risks involved and reduce the need for intrusive security software.

    As far as Linux and FreeBSD are concerned, it would be financially cheaper for the US government to pay to have both OS' rigorously scrutinised with an eye to fixing security holes and serious defects than it would be to have another day like today.

    And as Linux and FreeBSD are used in critical infrastructure that absolutely must stay working, it would be in line with the government's stated aim of securing that infrastructure.

    • How do you require higher standards of code? That is, how do you measure that the code is good or bad?

      • For code that is free/libre: Defects are the user's problem, but the baseline estimated defect density should be published somewhere.

        For commercial code:

        For regular activity, the estimated average defect density should be 0.5 per kloc.

        For code sold to key industries, this should be 0.4 per kloc.

        For code sold to critical industries, this should be 0.3 per kloc

        For mission-critical systems (as defined by failure causing death or in excess of $1m damage), code amenable to formal verification by HOL of COQ shoul

  • The Dangers of a Software Monoculture [schneier.com]

    “In 2003, a group of security experts—myself included—published a paper saying that 1) software monocultures are dangerous and 2) Microsoft, being the largest creator of monocultures out there, is the most dangerous. Marcus Ranum responded with an essay that basically said we were full of it. Now, eight years later, Marcus and I thought it would be interesting to revisit the debate.”
    • Microsoft basically bought off Federal employees, by giving them almost free software for home personal use. Ten bucks and you get a passcode to unlock it all.... this was a decade plus ago, before everything was online keylogging (spyware! - and this includes other "free" software) - I got off microserfs a while ago, now working on the other creepy stuff. But it's hard to get off this stuff. It's all gamified to keep us trapped. I know, I know, obvious!
    • Thank you for posting this.

      I became aware of the problems with a software monoculture on my own several years ago, and I thought I was rather original in my thinking! This Schneier guy continues to impress me at how far ahead he can see. 21 years is forever in technology. Wow.
      Will keep this link to prove it wasn't my idea.

      I'm curious to see the counterpoint essay, but the link was dead.
      • > I'm curious to see the counterpoint essay, but the link was dead

        Counterpoint Marcus Ranum [bitpipe.com]
        • I read it. Not convincing at all.
          In 2010? Welllll... ok. I think he was defending monopolistic behaviour myself.
          I think the friday outage shows the impact on critical systems a MS monoculture causes.
          I'd call that proof enough that there's no resilience in the system as Ranum argues and
          we are definitely seeing the effects of blind trust in technology. We've definitely arrived in dystopian territory.
          The companies legal departments must be shitting a brick right now.

          Schneier is still way out front on this issu
          • > I read it. Not convincing at all.

            Always interesting in seeing two sides of the argument. Ranum does have some interesting things to say on The Six Dumbest Ideas in Computer Security [clemson.edu]
            • It's a pretty good read. I like to hear POVs on security.
              I wouldn't say I'm a fan of this guy's writing style and analysis, it's a bit folksy and light on substance, imho, however he brings up some good points. .. but in a twist of irony says

              "Around the time I was learning to walk, Donn Parker was researching the behavioral aspects of hacking and
              computer security. He says it better than I ever could:"Remote computing freed criminals from the historic
              requirement of proximity to their crimes. Anonymity and fr
  • The Crowdstrike update didn’t screw MacOS or Linux desktops.

Slashdot Top Deals

"Here's something to think about: How come you never see a headline like `Psychic Wins Lottery.'" -- Comedian Jay Leno

Close