×
Businesses

Who Will Pay For the Costs of Crowdstrike's Outage? (cnn.com) 196

8.5 million Windows devices were ultimately affected by the Crowdstrike outage, according to figures from Microsoft cited by CNN.

And now an anonymous Slashdot reader shares CNN's report on the ramifications: What one cybersecurity expert said appears to be the "largest IT outage in history" led to the cancellation of more than 5,000 commercial airline flights worldwide and disrupted businesses from retail sales to package deliveries to procedures at hospitals, costing revenue and staff time and productivity... While CrowdStrike has apologized, it has not mentioned whether or not it intends to provide compensation to affected customers. And when asked by CNN about whether it plans to provide compensation, its response did not address that question. Experts say they expect that there will be demands for remuneration and very possibly lawsuits.

"If you're a lawyer for CrowdStrike, you're probably not going to enjoy the rest of your summer," said Dan Ives, a tech analyst for Wedbush Securities....

But there could be legal protections for CrowdStrike in its customer contracts to shield it from liability, according to one expert. "I would guess that the contracts protect them," said James Lewis, researcher at the Center for Strategic and International Studies...

It's also not clear how many customers CrowdStrike might lose because of Friday. Wedbush Securities' Ives estimates less than 5% of its customers might go elsewhere. "They're such an entrenched player, to move away from CrowdStrike would be a gamble," he said. It will be difficult, and not without additional costs, for many customers to switch from CrowdStrike to a competitor. But the real hit to CrowdStrike could be reputational damage that will make it difficult to win new customers... [E]ven if customers are understanding, it's likely that CrowdStrike's rivals will be seeking to use Friday's events to try to lure them away.

One final note from CNN. Patrick Anderson, CEO of a Michigan research firm called the Anderson Economic Group, "added that the costs could be particularly significant for airlines, due to lost revenue from canceled flights and excess labor and fuel costs for the planes that did fly but faced significant delays."

See also: Third Day of 1,000+ Cancelled Flights, Just in the US, After Crowdstrike Outage .
Crime

Ransomware Attack Takes Down Computer System for America's Largest Trial Court (apnews.com) 33

A ransomware attack has taken down the computer system of America's largest trial court, reports the Associated Press: The cybersecurity attack began early Friday and is not believed to be related to the faulty CrowdStrike software update that has disrupted airlines, hospitals and governments around the world, officials said in a statement Friday. The court disabled its computer network systems upon discovery of the attack, and it will remain down through at least the weekend.
Friday's statement called it "a serious security event," adding that the court is receiving help from local, state, and federal law enforcement agencies. "At this time, the preliminary investigation shows no evidence of court users' data being compromised." Over the past few years, the Court has invested heavily in its cybersecurity operations, modernizing its cybersecurity infrastructure and making strategic staff investments in the Cybersecurity Division within Court Technology Services. As a result of this investment, the Court was able to quickly detect an intrusion and address it immediately.

Due to the ongoing nature of the investigation, remediation, and recovery, the Court will not comment further until additional information is available for public release.

Sunday the Court posted on X.com that they're "working diligently to get the Court's network systems back up and running...

"When we have a better understanding of the extent to which the Court will be operational tomorrow, July 22, we will provide information and direction to court users and jurors, likely later this evening."
China

One Nation Mostly Unaffected by the Crowdstrike Outage: China (bbc.com) 49

The BBC reports that "while most of the world was grappling with the blue screen of death on Friday," there was one country that managed to escape largely unscathed: China. The reason is actually quite simple: CrowdStrike is hardly used there. Very few organisations will buy software from an American firm that, in the past, has been vocal about the cyber-security threat posed by Beijing. Additionally, China is not as reliant on Microsoft as the rest of the world. Domestic companies such as Alibaba, Tencent and Huawei are the dominant cloud providers.

So reports of outages in China, when they did come, were mainly at foreign firms or organisations. On Chinese social media sites, for example, some users complained they were not able to check into international chain hotels such as Sheraton, Marriott and Hyatt in Chinese cities. Over recent years, government organisations, businesses and infrastructure operators have increasingly been replacing foreign IT systems with domestic ones. Some analysts like to call this parallel network the "splinternet".

"It's a testament to China's strategic handling of foreign tech operations," says Josh Kennedy White, a cybersecurity expert based in Singapore. "Microsoft operates in China through a local partner, 21Vianet, which manages its services independently of its global infrastructure. This setup insulates China's essential services — like banking and aviation — from global disruptions."

"Beijing sees avoiding reliance on foreign systems as a way of shoring up national security."

Thanks to long-time Slashdot reader hackingbear for sharing the article.
Microsoft

Microsoft Releases Recovery Tool for Windows Machines Hit By Crowdstrike Issue (theverge.com) 115

The Verge reports that for machines that aren't automatically receiving Crowdstrike's newly-released software fix, Microsoft has released a recovery tool that creates a bootable USB drive. Some IT admins have reported rebooting PCs multiple times will get the necessary update, but for others the only route is having to manually boot into Safe Mode and deleting the problematic CrowdStrike update file. Microsoft's recovery tool now makes this recovery process less manual, by booting into its Windows PE environment via USB, accessing the disk of the affected machine, and automatically deleting the problematic CrowdStrike file to allow the machine to boot properly. This avoids having to boot into Safe Mode or a requirement of admin rights on the machine, because the tool is simply accessing the disk without booting into the local copy of Windows. If a disk is protected by BitLocker encryption, the tool will prompt for the BitLocker recovery key and then continue to fix the CrowdStrike update.
Windows

Who Wrote the Code for Windows' 'Blue Screen of Death'? (sfgate.com) 40

Who wrote the code for Windows' notorious "Blue Screen of Death? It's "been a source of some contention," writes SFGate: A Microsoft developer blog post from Raymond Chen in 2014 said that former Microsoft CEO Steve Ballmer wrote the text for the Ctrl+Alt+Del dialog in Windows 3.1. That very benign post led to countless stories from tech media claiming Ballmer was the inventor of the "Blue Screen of Death." That, in turn, prompted a follow-up developer blog post from Chen titled "Steve Ballmer did not write the text for the blue screen of death...."

Chen then later tried to claim he was responsible for the "Blue Screen of Death," saying he coded it into Windows 95. Problem is, it already existed in previous iterations of Windows, and 95 simply removed it. Chen added it back in, which he sort of cops to, saying: "And I'm the one who wrote it. Or at least modified it last." No one challenged Chen's 2014 self-attribution, until 2021, when former Microsoft developer Dave Plummer stepped in. According to Plummer, the "Blue Screen of Death" was actually the work of Microsoft developer John Vert, whom logs revealed to be the father of the modern Windows blue screen way back in version 3.1.

Plummer spoke directly with Vert, according to Vert, who'd remembered that he got the idea because there was already a blue screen with white text in both his machine at the time (a MIPS RISC box) and this text editor (SlickEdit)...
Businesses

CrowdStrike Stock Tanks 15%, Set For Worst Day Since 2022 (forbes.com) 81

Shares of cybersecurity company CrowdStrike Holdings dropped 15% on Friday after the company's software update resulted in what may turn out to be the largest IT outage ever. CrowdStrike stock "is on pace for its steepest daily loss since November 2022 and its $290 low share price is the lowest intraday mark since April 25," reports Forbes. "CrowdStrike is on track for the third-worst day in its five-year history as a publicly traded company." From the report: Microsoft, which was swept up in the outage as the downed systems are those running CrowdStrike's cybersecurity applications and Microsoft's Windows software, also slumped, with its shares down about 1% to the $3.2 trillion behemoth's lowest share price since June 11. CrowdStrike competitor Palo Alto Networks enjoyed a 4% rally Friday, while the tech-heavy Nasdaq Composite stock index gained about 0.2%, held up by the likes of Microsoft rival Apple's 1% stock gain and a 1% rise for shares of Alphabet, which is reportedly in talks to buy cybersecurity firm Wiz for $23 billion.

The CrowdStrike selloff is "an overreaction to a temporary setback," Rosenblatt analyst Catharine Trebnick wrote in a note to clients Friday. It's a "compelling buying opportunity" as it "creates a window for investors to buy into a high-quality, growth-oriented cybersecurity company at a discounted valuation," Trebnick continued. To her point, CrowdStrike stock's relative valuation, according to its price-to-earnings ratio (P/E), which compares its market value to its projected profits over the next four quarters, fell Friday to its lowest number since April. Still, CrowdStrike's P/E of about 70 is very high for a company of its size, meaning investors will need to express significant confidence in the business' ability to grow earnings, a challenge if Friday's incident were to impact CrowdStrike's client base.

The Courts

OpenAI Dropped From First Ever AI Programming Copyright Lawsuit 8

OpenAI escaped a copyright lawsuit from a group of open-source programmers after they voluntarily dismissed their case against the company in federal court. From a report: The programmers, who allege the generative AI programming tool Copilot was trained on their code without proper attribution, filed their notice of voluntary dismissal Thursday, but will still have their case against GitHub and parent company Microsoft, which collaborated with OpenAI in developing the tool. The proposed class action filed in 2022 in the US District Court for the Northern District of California was the first major copyright case against OpenAI, which has since been hit with numerous lawsuits from authors and news organizations including the New York Times.
XBox (Games)

FTC Attacks Microsoft's Post-Merger Game Pass Price Increases (arstechnica.com) 15

The FTC says the across-the-board price increases that Microsoft recently announced for its Xbox Game Pass subscription service tiers represent "exactly the sort of consumer harm from the merger the FTC has alleged" when it sought to block Microsoft's merger with Activision. From a report: In a letter to the court posted as part of an ongoing appeal by the FTC in the case, the federal regulator alleges Microsoft's moves are a clear example of "product degradation" brought about by "a firm exercising market power post-merger." The letter's primary focus is on the soon-to-be-discontinued $10.99/month Console Game Pass tier. That's being replaced with a $14.99/month Game Pass Standard tier (a 36 percent price increase) that no longer includes "day one" access to all of Microsoft's first-party titles. To maintain that key benefit, "Console" subscribers will have to spend 81 percent more for the $19.99 Game Pass Ultimate tier, which also includes a number of additional benefits over the current $10.99/month option.

The FTC notes that these changes "coincide with adding Call of Duty to Game Pass's most expensive tier." Previously, Microsoft publicly promised that this Game Pass access to Activision's ultra-popular shooter would come "with no price increase for the service based on the acquisition." It's that "based on the acquisition" clause that's likely to give Microsoft some wiggle room in arguing for its planned pricing changes. Inflation is also a sufficient explanation for a large portion of the price increase in nominal terms -- the $14.99 Microsoft charged for a month of Game Pass Ultimate when it launched in 2019 is the equivalent of $18.39 today, according to the BLS CPI calculator.

Microsoft

Sanctioned Russia Emerges Unscathed in Global IT Outage (yahoo.com) 110

Russian officials boasted on Friday that Moscow was spared the impact of the global IT systems outage because of its increased self-sufficiency after years of Western sanctions, though some experts said Russian systems could still be vulnerable. From a report: Microsoft and other IT firms have suspended sales of new products in Russia and have been scaling down their operations in line with sanctions imposed over Russia's war in Ukraine, which Moscow describes as a special military operation. The Kremlin, along with companies from state nuclear giant Rosatom, which operates all of Russia's nuclear plants, to major lenders and airlines, reported no glitches amid the outage that affected international companies across the globe. "The situation once again highlights the significance of foreign software substitution," Russia's digital development ministry said. Russian financial and currency markets also ran smoothly.
IT

It's Not Just CrowdStrike - the Cyber Sector is Vulnerable (ft.com) 90

An anonymous reader shares a report, which expands on the ongoing global outage: The incident will exacerbate concerns about concentration risk in the cyber security industry. Just 15 companies worldwide account for 62 per cent of the market in cyber security products and services, according to SecurityScorecard. In modern endpoint security, the business of securing PCs, laptops and other devices, the problem is worse: three companies, with Microsoft and CrowdStrike by far the largest, controlled half the market last year, according to IDC.

While the US Cyber Safety Review Board dissects large cyber attacks for lessons learned, there is no obvious body charged with analysing these technical failures to improve the resilience of global tech infrastructure, said Ciaran Martin, former head of the UK's National Cyber Security Centre. The current global outage should spur clients -- and perhaps even governments and regulators -- to think more about how to build diversification and redundancy into their systems.
Further reading: Without Backup Plans, Global IT Outages Will Happen Again.
Microsoft

To Fix CrowdStrike Blue Screen of Death Simply Reboot 15 Straight Times, Microsoft Says (404media.co) 173

Microsoft has a suggested solution for individual customers affected by what may turn out to be the largest IT outage that has ever happened: Just reboot it a lot. From a report: Customers can delete a specific file called C00000291*.sys, which is seemingly tied to the bug, Microsoft said in a status update published Friday. But in some cases, people can't even get to a spot where they can delete that file. In an update posted Friday morning, Microsoft told users that they should simply reboot Virtual Machines (VMs) experiencing a BSoD over and over again until they can fix the issue.

[...] "We have received reports of successful recovery from some customers attempting multiple Virtual Machine restart operations on affected Virtual Machines," Microsoft told users. "We have received feedback from customers that several reboots (as many as 15 have been reported) may be required, but overall feedback is that reboots are an effective troubleshooting step at this stage."

Microsoft

Global IT Outage Linked To CrowdStrike Update Disrupts Businesses (techcrunch.com) 274

A widespread IT outage, caused by a defective software update from cybersecurity firm CrowdStrike, is affecting businesses worldwide, causing significant disruptions across various sectors. The issue has primarily impacted computers running Windows, resulting in system crashes and "blue screen of death" errors. The travel industry appears to be among the hardest hit, with airlines and airports in multiple countries reporting problems with check-in and ticketing systems, leading to flight delays. Other affected sectors include banking, retail, and healthcare.

CrowdStrike CEO George Kurtz confirmed the outage was due to a "defect" in a content update for Windows hosts, ruling out a cyberattack. The company is working on a fix. CrowdStrike said the crash reports were "related to the Falcon Sensor" -- its cloud-based security service that it describes as "real-time threat detection, simplified management, and proactive threat hunting."

A Microsoft spokesperson told TechCrunch that the previous Microsoft 365 service disruption overnight July 18-19 was unrelated to the widespread outage triggered by the CrowdStrike update.

Editor's note: The story has been updated throughout the day and moved higher on the front page.
Microsoft

Microsoft Outage Hits Users Worldwide, Leading To Canceled Flights (wsj.com) 168

Microsoft grappled with a major service outage, leaving users across the world unable to access its cloud computing platforms and causing airlines to cancel flights. From a report: Thousands of users across the world reported problems with Microsoft 365 apps and services to Downdetector.com, a website that tracks service disruptions. "We're investigating an issue impacting users' ability to access various Microsoft 365 apps and services," Microsoft 365 Status said on X early Friday. On its status page for Azure, Microsoft's cloud computing platform, the company said the issue began just before 10 p.m. ET Thursday, affecting systems across the central U.S. In an update, Microsoft said it had determined the cause and was working to restore access to its users.
Businesses

Valve Runs Its Massive PC Gaming Ecosystem With Only About 350 Employees (arstechnica.com) 83

Valve had its employee and payroll data leaked through a poorly redacted document in an antitrust lawsuit in May, offering a rare glimpse into the company's small but impactful workforce over the years. As first noticed by SteamDB's Pavel Djundik, Valve's significant influence in PC gaming transactions has been maintained by just a few hundred employees. Kyle Orland reports via Ars Technica: It's striking to consider just how small Valve is compared to other major players in the game industry. In 2021, Microsoft estimated Valve's annual revenue at $6.5 billion, roughly on the same scale as EA's $7.5 billion in 2024 revenue. But Steam achieved those numbers with around 350 employees, compared to well over 13,000 people employed by EA. The disparity highlights just how much money Valve brings in with a relatively small workforce. And a lot of that is thanks to the chunk of revenue Valve takes from every sale on Steam. The dominant PC gaming marketplace has seen a massive increase in the number of annual game releases since 2012 or so, thanks to initiatives like Steam Greenlight and Steam Direct.

Yet, surprisingly, the size of the "Steam" department inside Valve has shrunk in recent years, from a peak of 142 employees in 2015 down to just 79 in 2021. From the outside, having just 79 employees keeping track of more than 11,000 Steam releases in 2021 is a pretty incredible ratio. Some readers may also be surprised that Valve's "Games" department has represented a majority of the company's headcount since 2003. That has remained true (though to a lesser extent) even in more recent years, as Valve's output of new games has become much more occasional. It seems likely a large number of those Games department employees are devoted to ultra-popular Valve games like Dota 2 and Counter-Strike 2, which enjoy tens of millions of players and need significant support work.

The leaked data also shows the slow rise of Valve's small Hardware department, which started with just three employees in 2011 as the company began work on its doomed Steam Machines initiative. Transitioning into the Valve Index era in the late 2010s, the hardware department still represented just a few dozen people and a paltry 3 to 4 percent of the company's annual payroll. By the time we hit 2021 and the run-up to the Steam Deck, the Hardware division still makes up just 12 percent of Valve's small total headcount. Looking back, it's impressive that such a small team was able to create a portable gaming device that quickly spawned a whole micro-industry of imitators. We can only hope the Hardware team got a little more employee support in the wake of the Steam Deck's market success.

Google

Google's $500 Million Effort To Wreck Microsoft EU Cloud Deal Failed, Report Says (arstechnica.com) 9

Ashley Belanger reports via Ars Technica: Google tried to derail a Microsoft antitrust settlement over anticompetitive software licensing in the European Union by offering a $500 million alternative deal to the group of cloud providers behind the EU complaint, Bloomberg reported. According to Bloomberg, Google's offer to the Cloud Infrastructure Services Providers in Europe (CISPE) required that the group maintain its EU antitrust complaint. It came "just days" before CISPE settled with Microsoft, and it was apparently not compelling enough to stop CISPE from inking a deal with the software giant that TechCrunch noted forced CISPE to accept several compromises.

Bloomberg uncovered Google's attempted counteroffer after reviewing confidential documents and speaking to "people familiar with the matter." Apparently, Google sought to sway CISPE with a package worth nearly $500 million for more than five years of software licenses and about $15 million in cash. But CISPE did not take the bait, announcing last week that an agreement was reached with Microsoft, seemingly frustrating Google. CISPE initially raised its complaint in 2022, alleging that Microsoft was "irreparably damaging the European cloud ecosystem and depriving European customers of choice in their cloud deployments" by spiking costs to run Microsoft's software on rival cloud services. In February, CISPE said that "any remedies and resolution must apply across the sector and to be accessible to all cloud customers in Europe." They also promised that "any agreements will be made public."

But the settlement reached last week excluded major rivals, including Amazon, which is a CISPE member, and Google, which is not. And despite CISPE's promise, the terms of the deal were not published, apart from a CISPE blog roughly outlining central features that it claimed resolved the group's concerns over Microsoft's allegedly anticompetitive behaviors. What is clear is that CISPE agreed to drop their complaint by taking the deal, but no one knows exactly how much Microsoft paid in a "lump sum" to cover CISPE legal fees for three years, TechCrunch noted. However, "two people with direct knowledge of the matter" told Reuters that Microsoft offered about $22 million.

AI

Microsoft Investigated by UK Over Ex-Inflection Staff Hires (bloomberg.com) 3

Microsoft's investment into Inflection AI will get a full-blown UK antitrust probe, after the watchdog said it needed to take a closer look at the hiring of former employees from the artificial intelligence startup. From a report: The Competition and Markets Authority said Tuesday it was opening the formal phase one merger probe into the partnership, setting a Sept. 11 deadline on whether to escalate it to an in-depth investigation. The agency has been swift to act against big tech's AI startup investments after it found a pattern of large tech firms piling money into start ups.
AI

Microsoft Unveils a Large Language Model That Excels At Encoding Spreadsheets 38

Microsoft has quietly announced the first details of its new "SpreadsheetLLM," claiming it has the "potential to transform spreadsheet data management and analysis, paving the way for more intelligent and efficient user interactions." You can read more details about the model in a pre-print paper available here. Jasper Hamill reports via The Stack: One of the problems with using LLMs in spreadsheets is that they get bogged down by too many tokens (basic units of information the model processes). To tackle this, Microsoft developed SheetCompressor, an "innovative encoding framework that compresses spreadsheets effectively for LLMs." "It significantly improves performance in spreadsheet table detection tasks, outperforming the vanilla approach by 25.6% in GPT4's in-context learning setting," Microsoft added. The model is made of three modules: structural-anchor-based compression, inverse index translation, and data-format-aware aggregation.

The first of these modules involves placing "structural anchors" throughout the spreadsheet to help the LLM understand what's going on better. It then removes "distant, homogeneous rows and columns" to produce a condensed "skeleton" version of the table. Index translation addresses the challenge caused by spreadsheets with numerous empty cells and repetitive values, which use up too many tokens. "To improve efficiency, we depart from traditional row-by-row and column-by-column serialization and employ a lossless inverted index translation in JSON format," Microsoft wrote. "This method creates a dictionary that indexes non-empty cell texts and merges addresses with identical text, optimizing token usage while preserving data integrity." [...]

After conducting a "comprehensive evaluation of our method on a variety of LLMs" Microsoft found that SheetCompressor significantly reduces token usage for spreadsheet encoding by 96%. Moreover, SpreadsheetLLM shows "exceptional performance in spreadsheet table detection," which is the "foundational task of spreadsheet understanding." The new LLM builds on the Chain of Thought methodology to introduce a framework called "Chain of Spreadsheet" (CoS), which can "decompose" spreadsheet reasoning into a table detection-match-reasoning pipeline.
AI

Microsoft CTO Kevin Scott Thinks LLM 'Scaling Laws' Will Hold Despite Criticism 18

An anonymous reader quotes a report from Ars Technica: During an interview with Sequoia Capital's Training Data podcast published last Tuesday, Microsoft CTO Kevin Scott doubled down on his belief that so-called large language model (LLM) "scaling laws" will continue to drive AI progress, despite some skepticism in the field that progress has leveled out. Scott played a key role in forging a $13 billion technology-sharing deal between Microsoft and OpenAI. "Despite what other people think, we're not at diminishing marginal returns on scale-up," Scott said. "And I try to help people understand there is an exponential here, and the unfortunate thing is you only get to sample it every couple of years because it just takes a while to build supercomputers and then train models on top of them."

LLM scaling laws refer to patterns explored by OpenAI researchers in 2020 showing that the performance of language models tends to improve predictably as the models get larger (more parameters), are trained on more data, and have access to more computational power (compute). The laws suggest that simply scaling up model size and training data can lead to significant improvements in AI capabilities without necessarily requiring fundamental algorithmic breakthroughs. Since then, other researchers have challenged the idea of persisting scaling laws over time, but the concept is still a cornerstone of OpenAI's AI development philosophy.
Scott's comments can be found around the 46-minute mark.
AI

Microsoft CTO Says AI Progress Not Slowing Down, It's Just Warming Up (arstechnica.com) 28

An anonymous reader shares a report: During an interview with Sequoia Capital's Training Data podcast published last Tuesday, Microsoft CTO Kevin Scott doubled down on his belief that so-called large language model (LLM) "scaling laws" will continue to drive AI progress, despite some skepticism in the field that progress has leveled out. Scott played a key role in forging a $13 billion technology-sharing deal between Microsoft and OpenAI. "Despite what other people think, we're not at diminishing marginal returns on scale-up," Scott said. "And I try to help people understand there is an exponential here, and the unfortunate thing is you only get to sample it every couple of years because it just takes a while to build supercomputers and then train models on top of them."

LLM scaling laws refer to patterns explored by OpenAI researchers in 2020 showing that the performance of language models tends to improve predictably as the models get larger (more parameters), are trained on more data, and have access to more computational power (compute). The laws suggest that simply scaling up model size and training data can lead to significant improvements in AI capabilities without necessarily requiring fundamental algorithmic breakthroughs. Since then, other researchers have challenged the idea of persisting scaling laws over time, but the concept is still a cornerstone of OpenAI's AI development philosophy.

Facebook

Facebook Ads For Windows Desktop Themes Push Info-Stealing Malware (bleepingcomputer.com) 28

Cybercriminals are using Facebook business pages and advertisements to promote fake Windows themes that infect unsuspecting users with the SYS01 password-stealing malware. From a report: Trustwave researchers who observed the campaigns said the threat actors also promote fake downloads for pirated games and software, Sora AI, 3D image creator, and One Click Active. While using Facebook advertisements to push information-stealing malware is not new, the social media platform's massive reach makes these campaigns a significant threat.

The threat actors take out advertisements that promote Windows themes, free game downloads, and software activation cracks for popular applications, like Photoshop, Microsoft Office, and Windows. These advertisements are promoted through newly created Facebook business pages or by hijacking existing ones. When using hijacked Facebook pages, the threat actors rename them to suit the theme of their advertisement and to promote the downloads to the existing page members.

Slashdot Top Deals