Microsoft Confirms Windows Server Security Update Caused Memory Leak, 'Unscheduled' Reboots (bleepingcomputer.com) 35
"Microsoft confirmed that a memory leak introduced with the March 2024 Windows Server security updates is behind a widespread issue causing Windows domain controllers to crash," BleepingComputer reported Thursday.
Friday Microsoft wrote that the issue "was resolved in the out-of-band update KB5037422," only available via the Microsoft Update Catalog. (The update "is not available from Windows Update and will not install automatically.")
BleepingComputer reported the leak only affected "enterprise systems using the impacted Windows Server platform," and home users were not affected. But Microsoft confirmed it impacted all domain controller servers with the latest Windows Server 2012 R2, 2016, 2019, and 2022 updates: As BleepingComputer first reported on Wednesday and as many admins have warned over the last week, affected servers are freezing and restarting unexpectedly due to a Local Security Authority Subsystem Service (LSASS) process memory leak introduced with this month's cumulative updates.
"Since installation of the March updates (Exchange as well as regular Windows Server updates) most of our DCs show constantly increasing lsass memory usage (until they die)," one admin said.
"Our symptoms were ballooning memory usage on the lsass.exe process after installing KB5035855 (Server 2016) and KB5035857 (Server 2022) to the point that all physical and virtual memory was consumed and the machine hung," another Windows admin told BleepingComputer.
The leak "is observed when on-premises and cloud-based Active Directory Domain Controllers service Kerberos authentication requests," Microsoft wrote. "Extreme memory leaks may cause LSASS to crash, which triggers an unscheduled reboot of underlying domain controllers..."
"We strongly recommend you do not apply the March 2024 security update on DCs and install KB5037422 instead..."
Friday Microsoft wrote that the issue "was resolved in the out-of-band update KB5037422," only available via the Microsoft Update Catalog. (The update "is not available from Windows Update and will not install automatically.")
BleepingComputer reported the leak only affected "enterprise systems using the impacted Windows Server platform," and home users were not affected. But Microsoft confirmed it impacted all domain controller servers with the latest Windows Server 2012 R2, 2016, 2019, and 2022 updates: As BleepingComputer first reported on Wednesday and as many admins have warned over the last week, affected servers are freezing and restarting unexpectedly due to a Local Security Authority Subsystem Service (LSASS) process memory leak introduced with this month's cumulative updates.
"Since installation of the March updates (Exchange as well as regular Windows Server updates) most of our DCs show constantly increasing lsass memory usage (until they die)," one admin said.
"Our symptoms were ballooning memory usage on the lsass.exe process after installing KB5035855 (Server 2016) and KB5035857 (Server 2022) to the point that all physical and virtual memory was consumed and the machine hung," another Windows admin told BleepingComputer.
The leak "is observed when on-premises and cloud-based Active Directory Domain Controllers service Kerberos authentication requests," Microsoft wrote. "Extreme memory leaks may cause LSASS to crash, which triggers an unscheduled reboot of underlying domain controllers..."
"We strongly recommend you do not apply the March 2024 security update on DCs and install KB5037422 instead..."
wat (Score:4, Insightful)
(The update "is not available from Windows Update and will not install automatically.")
They'll cause the problem automatically, but won't fix it automatically? They think their customers want Windows to be crashing. Perhaps they are right, they are a defense contractor, and this could be part of an operation.
Re:wat (Score:4, Funny)
They think their customers want Windows to be crashing.
Do you not? It's literally the only reason to run Windows. Here, let me get you in on a little secret. Nobody got fired for buying a Microsoft product because it's guaranteed billable hours. When you need shit to work like on your home computer after you're done thinking about work for the day, you run Debian or some other Linux...something actually Good (so no RPM based or distros or Gentoo). You leave Windows for the bloody morons who pay you.
Re: (Score:2, Troll)
Well, I do IT security, but yes, MS is a rather big reason my specialty is very much in demand at this time. I still find this thate of affairs highly disgraceful not only for MS, but also for the whole "wannabe" section of the IT field that uses their crap in anything even remotely "professional" computing.
This stuff is not professional in any way. It is the cheap, plastic, Made-in-China fake of a real product.
Re: (Score:2)
This stuff is not professional in any way. It is the cheap, plastic, Made-in-China fake of a real product.
If that's the description of Windows, ServiceNow is nothing other than a way to cripple a country's business environment.
Hans Kristian Graebener = StoneToss
Re: (Score:2)
This stuff is not professional in any way. It is the cheap, plastic, Made-in-China fake of a real product.
If that's the description of Windows, ServiceNow is nothing other than a way to cripple a country's business environment.
It probably is.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Sure. But depending on your personal ethics (or absence thereof), it may still be an unacceptable situation. I find that for me, it is unacceptable.
Re: (Score:2)
Re: (Score:2)
They think their customers want Windows to be crashing.
D Nobody got fired for buying a Microsoft product because it's guaranteed billable hours.
Exactly. Considering my unit had an entire division of people who kept the Windows machines running, and one person - me - to keep the Macs working, and I wasn't even an IT person, I pissed them off a lot by telling them that using Windows was a job security program, so keep buying the Microsoft Job creation devices.
Re: (Score:2)
Re: (Score:2)
I run Devuan at home, and I am pretty happy with it, although I have installed enough stuff from backports to make it slightly spicy. But at work we use Windows whether I like it or not (guess which) and I would like it to work because otherwise I can't work.
Re: (Score:2)
Re: (Score:2)
patching mixed (Score:4, Insightful)
I try to wait a couple of weeks on Windows Updates just for this reason. I've been bitten a couple of times. Security patches really should not feel like a roll of the dice, or one step forward two steps back.
Re:patching mixed (Score:5, Insightful)
In a sane world, doing things this unprofessional for an expensive "server-grade" product would have massive punitive and liability cost for the vendor. As it is, they will just continue as before. Pathetic.
Re: (Score:2)
Re: (Score:2)
Sad but true. And the world burns. Well, smoulders somewhat now, but this will only get worse.
MS confirms MS OSes are crap... (Score:2, Interesting)
That is basically what this boils down to. Memory leaks are _very_ easy to identify in any reasonable testing. Apparently MS just pushes out not reasonably tested (or not systematically tested at all?) updates, and they do so for the figging, expensive _server_ version of their crappy excuse for an OS.
Time to stop using anything Microsoft makes. They are worse than Boeing at this time.
Re: (Score:3)
Not just MS any more - check out the clusterfuck that is the current 14.4 release of MacOS. They've broken printers, java and some USB. Seems to me that testing is becoming an afterthought these days and that "Agile" development has embrassed the customer - toss a release over the fence and let the suckers find the bugs.
Re: (Score:2)
check out the clusterfuck that is the current 14.4 release of MacOS. They've broken printers, java and some USB
And don't forget cURL [slashdot.org].
Hans Kristian Graebener = StoneToss
Re: (Score:2)
That one is special. Basically actively sabotaging if a user knows a bit more and is trying to be careful. I would classify this one under criminal malfeasance.
Re: (Score:2)
Quite possibly, yes. Kind of like if electricians would accept 1/10 houses they wire simply burning down. At some point, society cannot tolerate these shoddy, dishonorable practices anymore.
Wrong link (Score:2)
"was resolved in the out-of-band update KB5037422,"
The link for KB5037422 in the summary points to KB5035857, the update it supercedes. You know...the update that introduced the memory leak?
smdh.
Re: (Score:3)
https://www.catalog.update.microsoft.com/Search.aspx?q=KB5037422
This shit keeps happening... (Score:2)
...MS keeps being pretty blasé about it.
And yet, everyone and their dog keeps yammering on about how everything must run in Azure yesterday.
I am utterly incompatible with humanity sometimes :D.
Re: (Score:2)
I support 'cloud' systems for a living; I hate the fucking cloud for fundamental technological reasons but I like my paycheque more. I try to focus on the non-cloud portion of the work so I can be happy solving those problems.
Using a cloud provider means giving up control. Computers are inexpensive enough you can have your own on-prem VM hosts and machines. Maybe use a cloud backup solution, sure. But if you are able, you should self-host everything else. You don't even have to be big enough to have a
Re: (Score:2)
Bare metal hosting where you install your own OS from scratch on bare metal servers in many data centers works fine as well. No need to host *everything* in-house IMHO.
You have to reboot every day anyways (Score:3)
Isn't that normal for a Windows server?
Re: (Score:2)
Isn't that normal for a Windows server?
No. And it hasn't been in thirty years.
Par for the course (Score:2)
Patching cycle (Score:2)
One benefit to being almost negligently behind on patching is that this won't hurt at all.
Very strange (Score:2)
Microsoft's own development tools have memory checkers that will bark at you if even a single byte is not freed. How does something that happens under normal use slip by basic testing?
Home users not affected (Score:2)
So, if I run a domain controller at home, I'm immune?