6791200
story
Comments: 201 +- Technology: The Machine SID Duplication Myth on Tuesday November 03, @09:20PM
Posted
by
kdawson
on Tuesday November 03, @09:20PM
from the no-harm-in-seeing-double dept.
from the no-harm-in-seeing-double dept.
background: url(//a.fsdn.com/sd/topics/topicwindows.gif); width:49px; height:65px;
windows
background: url(//a.fsdn.com/sd/topics/topicms.gif); width:75px; height:55px;
microsoft
background: url(//a.fsdn.com/sd/topics/topicsoftware.gif); width:65px; height:65px;
software
background: url(//a.fsdn.com/sd/topics/topicit.gif); width:75px; height:61px;
it
toppings writes "Microsoft Technical fellow Mark Russinovich explains why he is now retiring NewSID, which has been used by IT departments for years when deploying Windows to new systems from customized clone images. Russinovich writes: 'The reason that I began considering NewSID for retirement is that, although people generally reported success with it on Windows Vista, I hadn't fully tested it myself and I got occasional reports that some Windows component would fail after NewSID was used. When I set out to look into the reports I took a step back to understand how duplicate SIDs could cause problems, a belief that I had taken on faith like everyone else. The more I thought about it, the more I became convinced that machine SID duplication — having multiple computers with the same machine SID — doesn't pose any problem, security or otherwise. I took my conclusion to the Windows security and deployment teams and no one could come up with a scenario where two systems with the same machine SID, whether in a Workgroup or a Domain, would cause an issue. At that point the decision to retire NewSID became obvious.' He concludes: 'It's a little surprising that the SID duplication issue has gone unquestioned for so long, but everyone has assumed that someone else knew exactly why it was a problem. To my chagrin, NewSID has never really done anything useful and there's no reason to miss it now that it's retired. Microsoft's official policy on SID duplication will also now change and look for Sysprep to be updated in the future to skip SID generation.'"
Related Stories
"The following is not for the weak of heart or Fundamentalists."
-- Dave Barry
All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2009 Geeknet, Inc.
fp (Score:4, Funny)
Except for Domain Controllers.. (Score:2, Informative)
Re: (Score:3, Informative)
Agreed...when I was reading up for one of the Server 2008 AD MCTS exams, I cloned a base VM image of Server 2008 to simulate two DCs, a file server, an IIS/application server, etc. I had to download and run NewSID because every server I joined to the domain (i.e. the "primary" DC) had problems getting joined correctly. I don't recall the specifics but Server 2008 did throw a hissy fit and I had to run NewSID on each VM prior to joining before I could do anything else.
Re:Except for Domain Controllers.. (Score:5, Informative)
It's not for domain controllers in general it's for the very first domain controller used to initialize a brand new domain. You want to never create a new server with that same SID again. The first domain controller's SID is special, it will be used to generate the domain SID. From then on, all subsequent domain controllers promoted in the domain will have the same machine SID.
So you're good if you create the very first DC with a unique install, and clone all your other servers from an image.
As I said earlier, there’s one exception to rule, and that’s DCs themselves. Every Domain has a unique Domain SID that’s randomly generated by Domain setup, and all machine SIDs for the Domain’s DCs match the Domain SID. So in some sense, that’s a case where machine SIDs do get referenced by other computers. That means that Domain member computers cannot have the same machine SID as that of the DCs and therefore Domain. However, like member computers, each DC also has a computer account in the Domain, and that’s the identity they have when they authenticate to remote systems. All accounts in a Domain, including computers, users and security groups, have SIDs that are based on the Domain SID in the same way local account SIDs are based on the machine SID, but the two are unrelated.
issue is if a distributed application used machine SIDs to uniquely identify computers. No Microsoft software does so and using the machine SID in that way doesn’t work just for the fact that all DC’s have the same machine SID.
Parent
Well... it WAS a problem... (Score:2, Insightful)
I thought that the problem was defined as being based around locking a specific machine down with Group Policy... when two machines have the same
Re:Well... it WAS a problem... (Score:4, Interesting)
There are several other software packages with a similar problem. Microsoft SMS is a big one, as well as most McAfee Enterprise Virus scan products.
I think Mark's saying this to conveniently avoid updating his software to work with Windows Vista/Windows 7 =)
Parent
Re: (Score:3, Funny)
Re:Well... it WAS a problem... (Score:5, Informative)
Did you mean the SusClientId? AFAIK this is the only identifier WSUS uses to distinguish between computers (they also don't have to be on the same domain).
On new clones you only need to delete the SusClientId key under HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate; the update service will take care of assigning the machine a new ID.
Parent
Re:Well... it WAS a problem... (Score:5, Informative)
I ran into this same issue. I've now got a batch script that runs at first logon (post-reimaging) that resets the client ID. Probably overkill at this point (the bad image that was causing this has long since been redone), but it ensure that every machine checks in with a fresh key.
net stop wuauserv /v SusClientId /f /resetauthorization /detectnow
reg delete "HKEY_LOCALMACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Windowsupdate"
net start wuauserv
wuauclt.exe
That does require a more windows update client (the old wuauclt only accepted a couple of options, including detectnow, and ignored the reset. If you typed wuauclt.exe /gogetmeabeerandasteak it wouldn't throw an error, it just looked like it ran).
There's a distinct difference between the SID and the SusClientID.
Parent
Re: (Score:2, Informative)
It is a common misconception that duplicate SIDs create the issue where multiple PCs check in as the same PC (with a rolling name) in WSUS. The WSUS ID is in fact stored here: [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate]
as the SusClientId and SusClientIdValidation keys.
It can and should be reset independently of SIDs to have PCs correctly check-in to WSUS
Re:Well... it WAS a problem... (Score:4, Interesting)
I know for a fact that WSUS (Windows Server Update Services... basically a centralized patch server) would do "weird, interesting" things when two machines tried to check into WSUS with the same SID.
I don't even work with Windows servers and I happen to know this from engineering some network infrastructure (load balancing) for the folks in our organization who do manage WSUS. Long story short, what they thought was problematic load balancing across WSUS servers was actually the same SID being used from 1,000+ cloned VMs. WSUS thought they were one machine.
Parent
Re:Well... it WAS a problem... (Score:5, Informative)
This is absolutely correct.
Identical machine SIDs and WSUS identifiers (stored in the registry) don't stop the updates from being applied...they just cause the WSUS reports to show only the details for the last cloned machine that connected.
Parent
Re: (Score:3)
Oh good. So if Machine A fails to apply a patch for whatever reason and machine B comes along 5 minutes later with exactly the same SID but gets on fine.... you'll never know about machine A?
No. If Machine A fails to apply a patch for whatever reason and machine B comes along 5 minutes later with exactly the same WSUS ID but gets on fine you'll never know about machine A. WSUS ID and SID are not the same thing. Failure to properly sysprep your image (or at least manually delete the key) is what causes the issues people are describing, nothing to do with the SID whatsoever.
Re: (Score:3, Informative)
Go Figure (Score:5, Insightful)
This is coming from the same company that billed my employer to the tune of $250,000 USD in order to create a utility that would move a user profile from the old location to the new one after the user account had been moved to a new NT domain.
And then we found the moveuser.exe utility on the server resource kit and asked them what the $250,000 was for. Not that anyone who pays two hundred and fifty thousand dollars for a few lines of vbscript is smart (the phbs wanted something bonafide), but I'm just sayin'...
Man with a hammer (Score:3, Funny)
And then we found the moveuser.exe utility on the server resource kit and asked them what the $250,000 was for. Not that anyone who pays two hundred and fifty thousand dollars for a few lines of vbscript is smart (the phbs wanted something bonafide), but I'm just sayin'...
A company was having a problem with one of their machines, so they called in this specialist. The specialist came in, examined the machine, pulled out a hammer and tapped the machine. The specialist then produced a bill for $1,000. When asked why he was charging $1000 for just tapping he machine with a hammer, the specialist replied, "You're paying for me to know where to tap the machine with the hammer."
The bill was paid.
Re: (Score:3, Funny)
I believe the anecdote is this:
There was an engineer who had an exceptional gift for fixing all things mechanical. After serving his company loyally for over 30 years, he happily retired. Several years later the company contacted him regarding a seemingly impossible problem they were having with one of their multimillion-dollar machines. They had tried everything and everyone else to get the machine to work but to no avail. In desperation, they called on the retired engineer who had solved so many of their
42 (Score:2, Insightful)
So if SIDs are mostly irrelevant, why bother with them at all? Why not just always have them the same number (e.g., 42)?
ID (Score:2)
How is it an ID if reuse in the same context has no ill effects? What does it mean to identify something if all things can have the same ID?
Something is missing here.
-Peter
Re: (Score:3, Funny)
I had to take Golf Ball Inflation six times before I passed.
Duplicate UIDs (Score:3)
Re:Duplicate UIDs (Score:4, Insightful)
The "subtlety" here is that Windows is extremely complex. I don't think anybody knows exactly how it works. Given that, it is hard to determine conclusively whether something can cause problems or not. Without that knowledge, it is best to err on the safe side.
Parent
It's the usual story (Score:5, Funny)
If it's even there at all it's needlessly complex and frilly, looks good without actually covering much and is far too easy to get around or remove completely.
The excessive complexity for no good reason of the SID and the way UIDs are implemented on that array of platforms are a good example of this.
Re:It's the usual story (Score:5, Funny)
Parent
Re: (Score:3, Funny)
Thanks for a good laugh, Sir! But at least in Britney's underwear, it covers something useful.
Re:It's the usual story (Score:4, Funny)
Or, its covering of something is useful.
Parent
Google "COMMANDO" (Score:5, Funny)
GOOGLE IMAGES: britney spears commando [google.com]
Parent
Hmm... Pretty sure I ran into an issue somewhere (Score:2)
Re:Hmm... Pretty sure I ran into an issue somewher (Score:4, Informative)
You should sysprep the machines to reset their state before joining the machines. Basically, you should create a stock VM that is your disk image right after a "sysprep" and then NEVER EVER do anything with that. Clone it, complete the setup process, and join that cloned machine to the domain.
So in your case, you should have installed each VM from the ISO/CD and joined the domain, or used a first sysprepped disk image, cloned that twice, and used the two clones to join the domain.
The reason is that sysprep does the necessary work to separate two machine's identities in a more significant way than just the SID.
Microsoft's policy is you should never clone a disk image in a domain environment without first running sysprep. NewSID was just a way of doing "sysprep lite."
Parent
With an important caveat! (Score:5, Informative)
"As I said earlier, there’s one exception to rule, and that’s DCs themselves. Every Domain has a unique Domain SID that’s randomly generated by Domain setup, and all machine SIDs for the Domain’s DCs match the Domain SID. So in some sense, that’s a case where machine SIDs do get referenced by other computers. That means that Domain member computers cannot have the same machine SID as that of the DCs and therefore Domain. However, like member computers, each DC also has a computer account in the Domain, and that’s the identity they have when they authenticate to remote systems. All accounts in a Domain, including computers, users and security groups, have SIDs that are based on the Domain SID in the same way local account SIDs are based on the machine SID, but the two are unrelated."
The low ramifications of this as mentioned above may have changed post Win2K and XP. This particular caveat governed our processes as system deployment specialists for Microsoft corporate events. We had to make sure that any potential DC had a unique SID even before the machines were promoted to DC, otherwise we saw (verifiably!) many issues with Workstations failing to join the domain. I seem to recall other more esoteric issues with older Microsoft server products, but that may be delusions based on the mass hysteria we had about unique SIDs at the time.
Re:With an important caveat! (Score:5, Interesting)
I think there's an elegant, simple solution to this.
Microsoft should incorporate NewSID into the DCPROMO utility, and force generation of a new SID as part of the process of initializing a new domain (even if it means that another reboot will be required).
Since it's the only case where a DC needs to have a unique SID.
And domain creation is certainly an extra special case. Most potential DCs won't ever be used to perform the initial creation of a windows domain: in general, only 1 DC per domain is supposed to ever have that privilege over the entire lifetime of the Windows-based LAN, which usually means only 1 server per organization will actually ever need to have had a unique SID.
Parent
Re: (Score:2, Interesting)
Re:With an important caveat! (Score:5, Informative)
Here's what happens when a DC and member server are both cloned from the same base image with identical SIDs:
Event Type: Error
Event Source: NETLOGON
Event Category: None
Event ID: 5516
Date: 04/11/2009
Time: 08:52:35
User: N/A
Computer: SERVER01
Description:
The computer or domain SERVER01 trusts domain TESTDOMAIN. (This may be an indirect trust.) However, SERVER01 and TESTDOMAIN have the same machine security identifier (SID). NT should be re-installed on either SERVER01 or TESTDOMAIN.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp [microsoft.com].
Parent
Re:With an important caveat! (Score:4, Informative)
NewSID changes the machine SID
Unjoining and rejoining changes the domain SID
They aren't the same thing and MS support should have told you that.
Parent
Oh, right... that. (Score:4, Interesting)
For what it's worth, using NewSID (or some other technique to accomplish the same thing) was too much trouble to do the first time when push came to deadline and I had to crank out a few hundred WinXP workstations for the college labs. I didn't have any problems. Never gave it another thought.
It is no myth (Score:3, Insightful)
I've had problems (Score:2)
I ran into problems in the past.
When windows 2000 was first released, at my old job we did a complete deployment of Win200 on an NT4 server domain not knowing anything about sysprep or SID's. Every once in awhile we noticed that machines would randomly freeze for no reason. Looking on the net we found other people running into the same issue and found that resetting the SID's would fix the issue. After running sysprep on all of the PC's in the labs, the freezing stopped completely. We then just used sysprep
Re: (Score:2)
In other words... (Score:5, Insightful)
Microsoft is now my employer, and I have no reason to cater to the needs of the user community anymore.
Finally validation! (Score:3, Funny)
Great. (Score:5, Insightful)
Doesn't it bother anyone else that even Microsoft doesn't have a clue how the OS they developed works anymore? That something like this is even an issue?
Re:Great. (Score:5, Insightful)
But not every product is equally complex. I can't think of a feature that's critical to the proper basic administration of a Unix network that's equally poorly understood, to the point that it's considered news when someone figures it out after 10 years.
The feeling I often get when developing for Microsoft's platform is that it is gratuitously complex. Complex APIs are routinely replaced with new, more complex ones. API calls that take a dozen or so arguments, with some of them pointing to structures containing dozens of members, return error codes that complain of a bad argument - good luck finding out which one of the 30 or so the system found to be offensive. Bugs go unfixed for years. It's all rather unpleasant, really.
Parent
Duplicate SIDs are a huge problem with KMS (Score:3, Interesting)
As a student, I worked for the CS department. It was just me and my boss, and we both had extremely limited hours. Thus, we didn't have a whole lot of time or opportunity to figure out how to do things 'the right way' whenever that would change, and just kept doing things as we had been.
This was a problem when Vista was deployed. Once we got out image to where we wanted, we would ghost it and deploy to about 60 machines. For Vista, we used a KMS (Key Management Server) which is one of the options you have for licensing large numbers of machines. In a nutshell, each machine contacts the KMS and gets a license for itself.
This was supposed to be strictly limited to volume licensing; thus, the KMS would not activate any machines until it had at least 25 different machines registered to it.
Now, ideally what would happen is that before you make your image you'd basically set Windows into a 'deployment mode' (not the technical term) where, the next time it's booted, it would go through and reinitialize everything for the machine it's on, and part of this involves generating a unique SID.
We toyed with this a bit with the time we had, but couldn't get it to a place where we were happy with the results. In particular, we had some issues with networking, IIRC, that means we would have had to go and manually setup every machine for our network.
TL;DR: All of our machines had the same SID, the KMS only say 1 unique installation even though 60 machines were connecting to it, and Vista wouldn't activate. In order to fix it, we had to change the SIDs for each machine.
So to say that duplicate SIDs are not a problem is erroneous indeed.
Really? (Score:5, Interesting)
This surprises me. I'm not going to say he's wrong, after all the man literally wrote the book on Windows (Windows Internals from Microsoft Press, great book) but it just seems odd. We seem to have problems at work if a system is Ghosted, but not SID walked. It'll join the domain, but exhibit weird problems, like users not able to log in and such. Now maybe GhostWalk does other things too that are what really needs to be done, but it seems to just be a SID change tool.
Personally I'll keep using GhostWalk until Symantec removes it.
Ignorant and inconsiderate (Score:4, Informative)
Not so much of Mark, if he doesn't want to maintain it, thats fine, it was free, I get it.
However ... this is typical of MS.
They tell us (developers) that the sid will be unique. We write software that expects this and uses the sid as a unique ID.
Now they come along and say 'naaa, its not important to be unique, use the same sid all you want, no one will notice!'
And then I have to say ... thank god for real OSes where backwards compatibility is a rule for a reason, not just because they need it to maintain compatibility. They throw corner cases to the wind and go back on something they've said for years, completely ignoring the fact that people have built things based on something they said was a requirement.
This is the forth change that will break (or potentially in this case) software I have to maintain. Two patches that remove existing functionality in the name of security with the argument that 'no one uses it that way', to which Google can clearly show to be wrong. Even better is that one of them, a change to the DHTML control breaks some of their own apps, OWA for instance.
Its fucked up when you have to find a hack via Google to fix a bug in MS software that they say doesn't effect anyone ... except everyone that uses one of their more popular clients. Their response is 'patch exchange' which breaks OTHER things.
STOP
CHANGING
BINARY
COMPATIBILITY
you worthless fucks. Yes, I'm annoyed.
Re: (Score:3, Insightful)
NewSID allows for activation reset? (Score:5, Interesting)
From the article:
This is called generalizing the image, because when you boot an image created using this process, Sysprep specializes the installation by generating a new machine SID, triggering plug-and-play hardware detection, resetting the product activation clock, and setting other configuration data like the new computer name.
Is the product activation clock reset because of Sysprep, or because the SID is changed?
In other words, could NewSID be used to keep unactivated windows installations running indefinately?
<conspiracy_theory> Would that be the real reason for the NewSID retirement? What's the rush of removing the download instead of leaving it unsupported? </conspiracy_theory>
I'll miss NewSID (Score:5, Insightful)
Not that I ever used it to generate a completely new SID, but what I did find it invaluable for was to set a machine's SID back to its old value after a re-install. This did away with the need to change the ownership on all of the user's files still on the hard drive and meant that most of the time their user profile would just keep on working as if nothing had changed.
Re: (Score:2)
Re: (Score:3, Funny)