First Look At VMware's vSphere "Cloud OS" 86
snydeq writes "InfoWorld's Paul Venezia takes VMware's purported 'cloud OS,' vSphere 4, for a test drive. The bottom line: 'VMware vSphere 4.0 touches on almost every aspect of managing a virtual infrastructure, from ESX host provisioning to virtual network management to backup and recovery of virtual machines. Time will tell whether these features are as solid as they need to be in this release, but their presence is a substantial step forward for virtual environments.' Among the features Venezia finds particularly worthwhile is vSphere's Fault Tolerance: 'In a nutshell, this allows you to run the same VM in tandem across two hardware nodes, but with only one instance actually visible to the network. You can think of it as OS-agnostic clustering. Should a hardware failure take out the primary instance, the secondary instance will assume normal operations instantly, without requiring a VMotion.'"
Instantly? (Score:4, Insightful)
Re:Instantly? (Score:4, Funny)
Re: (Score:3, Funny)
Quantum processing,... hm, that means it will first happen when you look. That is definitely not a good idea. It should just work and I should not be required to walk down to the cellar to find the damn hardware box I am using. Next I will be required to locate the processor before Heisenberg is satisfied.
Re: (Score:2)
yes it is instant - i'm not sure exactly how they are doing it at the moment but basicly the boxes work in tandom to sync move on a per cycle basis.. meaning not a single cpu cycle is lost..
Re: (Score:1)
There probably are scenarios where there is a delay while it tries to figure out if indeed the other participant is down. However the OP's question -- instant -- was best responded to by the quantum processing response, because "instant" is in the mind of the assessor, and one woman's instant is another man's forever.
Whi
Re: (Score:1, Funny)
and one woman's instant is another man's forever...
I see that you too have been forced to wait for "an instant" while your girlfriend/wife does some "quick" shopping.
Re: (Score:2, Insightful)
Basically the two nodes will be in constant communication with each other. All data sent to the primary node is also sent to the secondary node, and the primary and secondary have a constant link between each other. Both nodes will perform the same computations on the data, but only the primary will reply to the user.
If the secondary node notices that the primary is not responding it will immediately send what the primary was supposed to have been sending back to the user.
Re: (Score:1)
Re: (Score:2)
Yep, if an operation takes out the system then the mirrored system will also die. They are in total lockstep so there is no protection of the application. It does give you near instant protection should the hardware or the hypervisor fails, but not if your app or OS should fail.
Re:Instantly? (Score:5, Informative)
Don't ask, just look. (Score:5, Informative)
The first hit on google images [ntpro.nl] should give you a good idea.
In practice, I don't know... I imagine that the secondary instance will still receive network traffic bound for the cluster, so it'd probably be perceived as a hiccup when the primary one goes down, which is good enough for most services.
Re: (Score:1, Interesting)
Re: (Score:1)
Instant eh? Define instant in this context.
To be honest I have found vMotion to be pretty much "instant". Is this thing *more* instant and if so is it more instant enough to justify its existence over vMotion?
I present my opinions as a non AC - if you are afraid of speaking freely then you work for the wrong firm.
Re: (Score:3, Informative)
It keeps a running copy on the failover host, reading from the same storage as the active host. It's as if the server were about to complete VMotion without having just done the final step. outage time is a small hiccough, less than a second. Current running sessions just carry on. If its uploading a file to someone, it just carries on. The outage is well withing the tollerance of typical TCP sessions.
Re: (Score:3, Informative)
It's close enough. I played with this feature at VMworld last year, and when running SQL transactions along with a ping, we dropped one packet but the SQL transactions didn't miss a beat.
It's impressive enough... the two systems are working in lockstep, such that even memory is duplicated between the two systems. It's an extension on the existing VMotion function in VMware today. However, bear in mind it has some limitations; only one CPU is possible at the moment and you still have the overhead of really t
Re:FT (Score:5, Informative)
FT only supports a single vCPU from my understanding... Not too many people running single CPU VM's, at least in my experience...
You should be running single vCPU machines by default and only scale CPU's if absolutely necessary and if the app is fully SMP aware and functional.
Re: (Score:2, Insightful)
If your apps are not SMP aware, WTF at you using?
Multiple CPUs has been the standard for servers for at least a decade in x86 gear.
Re:FT (Score:5, Insightful)
It makes sense. If you have 32 workloads and 16 cores, don't add the overhead of making 64 virtual vCPUs, 32 will use the host resources more efficiently as long as one app doesn't need the power of more than one core. If it does, give it to the guest only when it needs it.
Re: (Score:2)
I just am shocked to hear of any real server apps that are not smp aware.
If you have any boxes with one vcpu then it can't take another cpu when it needs it. Great, now you have 16 boxes and all have one cpu, too bad that none of them can soak all 16 when the others are totally idle. What a wonderful way to waste hardware.
This is why guest priorities are useful. Give them all a number of cpus that they actually needand let the priority sort it out. You will find that is what the individual OSes are doing an
Re:FT (Score:4, Insightful)
Re: (Score:2)
SQL server is a great example. Most of the time on dedicated hardware you might have one SQL server with several Databases and maybe even SQL instances. These will happly share the multiple CPUs torun seperate apps concurrently, but most queries in SQL execute in a single thread.
With VMs, as you dont have to pay $$$ for more hardware, only software licences, we have seen more customers simply provision another single CPU SQL server in VM for general duty work. These would generally have no tuning in the app
Re: (Score:2)
You obviously don't deal with many vertical apps. Most of them are not SMP aware, or tend to sit there hogging a single CPU/Core pretty much all the time. At least, this is true in the Windows world... UNIX is quite different, but there's a dearth of the applications my users want to run.
No, just 'cos I do it for a living doesn't mean I have to like it :)
Re: (Score:2)
Multiple CPUs has been the standard for servers for at least a decade in x86 gear.
Seriously, even Windows NT 4 had SMP support in 1997.
I don't know what year Linux first had support for SMP, but the 2.0 kernel supports SMP, apparently even on a 486. [tldp.org] Just imagine a Beowulf made of 486 class SMP machines!
Re: (Score:2)
Seriously, even Windows NT 4 had SMP support in 1997.
I don't know what year Linux first had support for SMP, but the 2.0 kernel supports SMP, apparently even on a 486. [tldp.org] Just imagine a Beowulf made of 486 class SMP machines!
Making good use of multiple CPUs requires more than just OS support.
Re: (Score:2)
Basically all server apps are SMP aware. Do you think IIS or Apache only have one worker process?
Perhaps that MSSQL and Postgresql only use one thread?
Re: (Score:2)
It wasn't them that I was thinking of specifically.
There are plenty of applications of databases (particularly MIS-type things) which don't really lend themselves to multithreading that well in the first place.
(BTW, Postgres only uses one thread per query and changing that, the last time I checked, was part of some significant work which is only just taking place right now)
Re: (Score:3, Insightful)
Try supporting synchronization of two virtual machines over a network/SAN when you also have to deal with SMP. Gets hard.
Re:FT (Score:5, Informative)
If your apps are not SMP aware, WTF at you using? Multiple CPUs has been the standard for servers for at least a decade in x86 gear.
Yes, but it doesn't work in VMs the same way, at least not in VMWare. On a loaded system you often find a single-vCPU VM will out perform one with more than one vCPU, in fact if you can spread your app over multiple machines you are generally better off running two single CPU VMs instead of one dual-CPU one. This is true no matter how many physical CPUs/cores you have available.
Why is this? Because a single CPU VM can be scheduled when-ever there are time-slices available on any physical CPU/core (though a good hypervisor will try not bounce VMs between cores too often, as this reduces the potential gains from using the core's L1 cache and (on architectures where L2 cache isn't shared) L2 cache too), but a dual vCPU VM will have to wait until the hypervisor can give it timeslices on two CPUs at the same time. If this is the only VM that is actively doing anything on the physical machine (and the host OS is otherwise quiet too) this makes little difference aside from a small overhead on top of the normal hits for not running on bare metal, but as soon as that VM is competing with other processes for CPU resource it can have a massive negative effect on scheduling latency.
Re: (Score:2)
, but a dual vCPU VM will have to wait until the hypervisor can give it timeslices on two CPUs at the same time.
Then their hypervisor is broken.
It should be possible for A dual vCPU machine to have vCPU1 and vCPU2 be two timeslices on the same real cpu if need be.
Re: (Score:2, Informative)
, but a dual vCPU VM will have to wait until the hypervisor can give it timeslices on two CPUs at the same time.
Then their hypervisor is broken. It should be possible for A dual vCPU machine to have vCPU1 and vCPU2 be two timeslices on the same real cpu if need be.
Which would kill any benefit of running SMP in the VM anyway, if it were possible.
My understanding, which may be out of date, is that this is not considered a good idea as timing issues between threads on the two vCPUs if scheduled one after the other on the same core could potentially cause race conditions. And if not that serious, the threads on the vCPU that gets the first slice of the real core could be paused waiting for locks to be released by threads the guest OS has lined up to runon the other vCPU.
Re: (Score:2)
And yet most processing workloads in apps I work with are single-threaded and not terribly scalable. The reality is that >1 CPU is exponentially more difficult to code for (at least so the excuses I hear from developers goes).
However, if your application can sit behind a load balancer and runs a single thread, why NOT have n*single CPU servers? Of course, that sort of takes away from the instant failover advantage. Ho hum :P
Xen did it first (Score:3, Informative)
Re: (Score:1, Informative)
VMware FT is not based on continuous memory snapshotting, it uses deterministic record/replay to do simultaneous record and replay. You could find a overview of this technology at http://www.vmware.com/products/fault-tolerance [vmware.com]
Also VMware demonstrated a working prototype as early as in 2007
http://www.vmworld.com/community/conferences/2007/agenda/ [vmworld.com]
w.r.t to Xen, doing a proof of concept is one thing but implementing it and supporting it in production quality with sufficient performance is another thing.
Re: (Score:2)
Re: (Score:2, Informative)
Marathon has had it working on Xen for quite a while.
Re: (Score:3, Informative)
We have both vMotion and XEN.
vMotion is very noticeable. Some things fail when it happens. Zenworks 6.5 is an example.
With Xen, we setup a VNC mirror. EG the guest was VNC Viewing itself. We were moving a window around and then we moved the guest from Xen server 1 to 2 (we have iSCSI BTW.) There was a noticeable affect that lasted for less than a second, but then we were on XEN #2.
It's nice to see VMWare getting this feature right with vSphere.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Is this a general-purpose Cloud OS? (Score:1)
Re: (Score:1)
and then if we could get these on to peoples computers with out letting them know it. Maybe with an E-mail or a web page...
I bet we could come up with a network of these robotic slave CPUs....
{insert sky-net reference here}
Re: (Score:2)
This is not a terribly new idea -- it's been around ever since Sun coined the phrase "The network is the computer."
The biggest problem with it is, of course, that I don't trust you, and you don't trust me. Why should I trust your computer to run my VM?
It only works for things like SETI because the data is not private, and the results can be verified, both by having multiple nodes run the same workload (and comparing the results), and by re-running the results yourself if you see something that looks like a
How does it detect a 'failure'? (Score:5, Informative)
How many hardware failures are actually characterized by a complete 100% loss of communication (as you'd get by pulling the plug)?
Don't CPU and Memory failures tend to make the computer somewhat unstable before completely bringing it down? How would vSphere handle (or even notice) that?
Even hard disk failures can take a small amount of time before the OS notices anything is awry (although you're an idiot if you care enough about redundancy to worry about this sort of thing, but don't have your disks in a RAID array)
Re: (Score:2)
Don't CPU and Memory failures tend to make the computer somewhat unstable before completely bringing it down?
Yes, and this is one of the key dividing lines between true HA mainframes, and every software implementation of HA services. The latter are what 99% of people seeking HA want (for cost reasons), but the former has been great business for IBM and Sun.
Re: (Score:3, Interesting)
Several Brand new servers with VI3 installed 2 weeks ago, left to run to burn in, first production guests moved onto them on Friday, Saturday sees CPU voltage regulator in one go pop, dead server. It would have been nice to just have the the Exchange server keep on rocking until Monday when we could replace the hardware, but no, now I've spent my Saturday morning going into work and fixing it.
However thanks to VM, the HighAvailbility service did restart the guests automatically, but I did have to repair a
Re: (Score:3, Interesting)
This is one reason I run Exchange 2007 with a clustered PHYSICAL mailbox server, and all the CAS and HT roles I run on virtual machines. I don't run database type apps on VMware for exactly these reasons... I am a big VMware supporter, but I also specify for our big apps that we use big SQL and Exchange clusters for HA... not VMware. Yes, it's a bit more expensive that way, but our Exchange cluster now hasn't been "down" in over a year, despite the fact that each node gets patched once a month and rebooted.
It works as advertized (Score:5, Informative)
Like everyone else pointed out, it's a VM in lockstep with a 'shadow' VM. This is not just 'continuous VMotion'.
If something happens to the VM, the shadow VM goes live instantly (you don't notice a thing if you're doing something on the VM).
Right after that, the system starts bringing up another shadow VM on another host to regain full FT protection.
This can be network intensive, depending on the VM load, and currently only works with 1 vCPU per VM. Think 1-2 FT VMs per ESX host + shadow VMs.
You'll need recent CPUs that support FT and have an VMware HA / DRS Cluster set up.
So if you've got it, use it wisely. It's very cool.
Re: (Score:1)
So, if the software running in the primary VM has a problem causing it to go down pretty hard (think near or BSOD class), and the lockstep mechanism is keeping things synchronized really well to the shadow VM(s), how many microseconds after the shadow VM comes up, does it go the way of the primary VM, as in totally tits up?
Or is the definition of FT (fault tolerance) "when some marketing droid pulls out a network cable or shuts down a server during a demonstration while trying to sell tens of thousands of d
Re: (Score:1)
Please familiarize yourself with the difference between hardware fault tolerance and software fault tolerance.
Re: (Score:1)
So, if the hardware is failing, corrupting memory or data on external storage, and the lockstep mechanism is keeping things synchronized really well to the shadow VM(s), how many microseconds after the shadow VM comes up, does it go the way of the primary VM, as in totally tits up?
Re: (Score:2)
Small correction... (Score:1)
The article mentions an inability (for the "pre-released" version) to PXE boot. If he's talking about booting for installation, then he's 100% wrong. The ESX beta/RC (build 140815) will, indeed, boot and install over a network. It's different from 3.5 so you'll have to adjust your commandline and/or kickstart. They use "weasel" instead of "anaconda" and that runs inside the service console. Short answer... "method=" becomes "url=" -- with a properly formated URL, eg. url=nfs://server/path/to/esx/. It'
Re: (Score:2)
The ESX beta/RC (build 140815) will, indeed, boot and install over a network
Has anyone done a PXE boot of the ESX OS itself yet, though?
AFAIK, the only "diskless" ESX deployments rely on flash storage.
Re: (Score:2)
Yes. It's quite trivial, actually and I seem to recall there's a VMware whitepaper on it.
OK, it's ESXi, not ESX... but the difference is small enough to make no odds. Oh, and on the flip side of that, I do find it easier to have ESXi on an internal flash in case my PXE server is down. I would host it virtually and on HA if it weren't for the fact that I have that whole "chicken and egg" problem :D
Re: (Score:2)
It's of sincere interest to me because we're turning some whitebox servers whose raid controllers aren't on the ESX HCL into hosts. I read that VMWare is moving ESXi as their premeir hypervisor to replace ESX, so this kind of setup would be interesting to explore, though i imagine that a flash based local datastore would be more... robust.
Re: (Score:2)
You know, that's an excellent idea if your SAN is capable... not all are. Our current production SAN is an HP EVA 4200... if we had a 4400 we could do it, but with the older 4200 it doesn't even have a direct network connection of its own; instead it has a dedicated storage server for managing the SAN (actually a DL380 running Windows Storage Server).
The ESXi HCL is a lot tighter than ESX, but I've had few problems so long as I stick with the "common" solutions. I buy almost exclusively HP servers for virtu
Re: (Score:2)
Re: (Score:2)
LOL... it happens to us all eventually ;) Next thing you know it'll be adult diapers and yelling at "those darned kids" :)
Seriously... this should help; http://www.chriswolf.com/?p=182
And I wish you all the best with your solution. Yes, I agree HP is often annoying, but their support for our solution has been great and easy to work with. Dell, well a few years ago I wouldn't touch a Dell server with a ten foot pole but I've had better luck recently with our European offices that insist on Dell (at least unt
Re: (Score:1)
ESX isn't designed to be run "diskless". It has to have somewhere to put it's VMFS -- which in 4.0 also contains swap and a few other things.
(That doesn't mean one cannot bend it into a shape that will run diskless.)
Re: (Score:2)
It has to have somewhere to put it's VMFS
Well, not in the sense of having no datastore, i simply mean without rotating hard disks present in the server. Flash storage accomplishes that, but you could use gPXE [etherboot.org] to connect it to an iSCSI target and remotely access its VMFS datastore from there.... if it were possible to do so with ESX, of course. You might not want to swap to it, and it'd really demand another NIC and so on, but that's what I really meant.
OS Agnostic? (Score:1)
FT CPU requirements (Score:1)
VMware collaborated with AMD and Intel in providing an efficient VMware Fault Tolerance (FT) capability on modern x86 processors. The collaboration required changes in both the performance counter architecture and virtualization hardware assists from processor vendors. These changes could only be included in recent