Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
×
Bug IT Technology

Massive VMware Bug Shuts Systems Down 410

mattmarlowe writes "Imagine if Red Hat released a version of Linux, and after it was deployed, customers noticed that any processes with a start date of today would refuse to run? Well, that's what happened to VMware — a company that wants nearly all server applications running in virtual machines within a matter of years." Supposedly a fix will be available ... in 36 hours.
This discussion has been archived. No new comments can be posted.

Massive VMware Bug Shuts Systems Down

Comments Filter:
  • Re:Ummm... How? (Score:5, Informative)

    by cduffy ( 652 ) <charles+slashdot@dyfis.net> on Tuesday August 12, 2008 @09:54AM (#24567631)

    If you read the article, you'd know it's the license-management code. Licenses expire.

  • Workaround available (Score:4, Informative)

    by fredr1k ( 946815 ) on Tuesday August 12, 2008 @09:54AM (#24567633) Homepage
    A workaround is possible Turn off NTP time on the host. And manually (using the VIC) change that date to one week backwards in time. Voila all set to work.
  • by d_ron_218 ( 1343245 ) on Tuesday August 12, 2008 @10:13AM (#24567961)
    The only way to run a Windows domain controller in VMware is to tie its clock to the physical host's clock. And lots of things break if your domain controllers have the wrong time (Kerberos authentication, NTP across the Windows network, etc, etc). So changing the host clock would generally be a bad idea.
  • by ray-auch ( 454705 ) on Tuesday August 12, 2008 @10:19AM (#24568039)

    USB license dongle for the application software running on the VM.

    Seriously. Last week.

  • by Cyberax ( 705495 ) on Tuesday August 12, 2008 @10:21AM (#24568079)

    LARGE corporations usually just buy site licenses. It's easier for them this way.

  • Re:Patch Tuesday (Score:5, Informative)

    by prandal ( 87280 ) on Tuesday August 12, 2008 @10:23AM (#24568111)

    Rebooting a host doesn't power down the VM.

    The licence checking is done at VM power up, apparently.

  • by Anonymous Coward on Tuesday August 12, 2008 @10:23AM (#24568131)

    They're working on it. Apparently there is a major release due in a few weeks. Not sure when, but maybe before October?

    http://forums.virtualbox.org/viewtopic.php?t=8528

  • by Cyberax ( 705495 ) on Tuesday August 12, 2008 @10:28AM (#24568201)

    Isolation and easy management.

    Isolation of applications in OSGi containers is leaky, one bad-behaving application can bring down the whole containers.

    Lightweight containers (OpenVZ, Virtuozzo) have almost no overhead and allow cool features like load-balancing of ALL applications between cluster nodes. However, all lightweight containers use the same kernel, and one kernel bug can bring down all virtual nodes.

    XEN/KVM have a bit more overhead but with even more isolation (each node has its own kernel).

  • by Comatose51 ( 687974 ) on Tuesday August 12, 2008 @10:29AM (#24568213) Homepage
    Unless something has changed dramatically, an expired license won't bring down any already deployed VMs. It simply won't allow you to deploy undeployed ones. It doesn't shut down the VMs as the headline makes it sound nor is it a bug in the hypervisor. Yes it's embarrassing that this got out but can we have a less sensationalist headline and summary?
  • it has USB support (Score:2, Informative)

    by reaktor ( 949798 ) on Tuesday August 12, 2008 @10:29AM (#24568219)
    Virtualbox has USB support...
  • by Anonymous Coward on Tuesday August 12, 2008 @10:32AM (#24568299)

    I'm not trolling, I'm honestly curious. What USB hardware do you need in your VMs?

    Bluetooth

    My PDA's dock

    A canon SLR camera, the remote capture software for which doesn't run on x64

    my wacom tablet

  • misleading headline (Score:1, Informative)

    by andrew.hill ( 530727 ) on Tuesday August 12, 2008 @10:42AM (#24568493)
    Nothing gets "Shut Down". You can't power on VMs, use vmotion, or DRS.
  • by Slashcrap ( 869349 ) on Tuesday August 12, 2008 @10:58AM (#24568741)

    Unless something has changed dramatically, an expired license won't bring down any already deployed VMs. It simply won't allow you to deploy undeployed ones. It doesn't shut down the VMs as the headline makes it sound nor is it a bug in the hypervisor. Yes it's embarrassing that this got out but can we have a less sensationalist headline and summary?

    No it just makes it impossible to start up VMs, restart VMs or VMotion them. I can't imagine why everyone's getting upset.

    Yes, there's a workaround - you just put back the date on the server. Unless you're in a business where randomly changing the dates on servers is frowned upon for compliance reasons.

  • by Anonymous Coward on Tuesday August 12, 2008 @10:59AM (#24568775)

    Simple...power. Right now our datacenter is strapped for power, and power isn't cheap. Neither is cooling. For 10U and 8000 watts I can install a fully loaded blade chassis with 128 CPU cores and 1 Terabyte of RAM, attach it to a SAN and run 150 VMs in it. Or I can install 150 rack and stack servers at taking up 4 racks and 75000 watts. Let me think here...

    And while I'm thinking about it, let's also remember that using VMWare gives you options like DRS and VMotion that you don't get with physical hardware. Or you can replicate your SAN to another SAN at your DR site and have a VMWare cluster waiting there for recovery. Then instead of having to do a bunch of restores to bare metal hardware, you could potentially get your servers back up and running in minutes instead of hours.

    There are many, many benefits to virtualization. If there weren't then people wouldn't have been using for decades in one form or another.

  • by exi1ed0ne ( 647852 ) <exile.pessimists@net> on Tuesday August 12, 2008 @11:04AM (#24568849) Homepage

    1) edit the vmware config file to include the line:

    host.cpukHz = XXXXX

    Where XXXXXis your CPU in kilohertz

    2) enable time sync via vmware tools

    3) modify Type in HKLM\System\CurrentControlSet\Services\W32Time\Parameters from "Nt5DS" to "NoSync"

    That's always taken care of clock skew issues on DCs for me.

  • by aix tom ( 902140 ) on Tuesday August 12, 2008 @11:05AM (#24568867)

    I would count those software using "license servers" that check on startup and then deny startup of the application into the "license enforcement" category, not in the "license management" category.

    On the other hand, we have one special software that doesn't enforce any license checks during runtime, but offers a "license audit" tool that outputs your concurrent users, maximum users, etc.. during a specific time period. That way you can check easily if you have enough licenses every now and then. And there is a condition in the license agreement that you have to check at least once a year. That is acceptable in my opinion.

    Then as a bad example we have this other software that is a pain in the ass to get to run because they needs a hardware ID to get it to run. Thankfully we were able to fake that hardware ID in VMWare. Because it wouldn't fit in our disaster recovery otherwise.

  • by Animats ( 122034 ) on Tuesday August 12, 2008 @11:07AM (#24568907) Homepage

    VMware is suggesting setting the system time backwards to work around their license manager problem. That's a desperation move. Not only will it mess up everything from Kerberos to CVS to "make", if you're running certain licensed software, in particular software licensed via FlexLM, that software will stop working. FlexLM will disable your licenses if the clock goes backwards by more than 24 hours. Now your expensive high-end software protected by FlexLM (Rational, Avid, Matlab, National Instruments, ANSYS, Cisco Unity, Clearcase, Nokia network management, etc.) will stop working. Setting the clock forward again may not re-enable it, either; there's tamper detection.

    Also, if you have server/client licensing with FlexLM, or multiple license servers, and the clocks disagree significantly, FlexLM gets suspicious and turns licenses off.

  • by alohatiger ( 313873 ) on Tuesday August 12, 2008 @11:21AM (#24569131) Homepage

    What is a "disadvantaged business" anyway, and why would someone actually use that as a sales point?

    Government work. Some government contracts require a percentage of the work to be done by minority/women/veteran/disadvantaged owned businesses.

  • by Len ( 89493 ) on Tuesday August 12, 2008 @11:29AM (#24569289)

    In a major corporation, fear of massive fines and prosecution is enough to stop them from pirating your software.

    Sadly, not true in the real world, as my company has discovered on more than one occasion.

  • by DrYak ( 748999 ) on Tuesday August 12, 2008 @11:38AM (#24569435) Homepage

    Support for USB, iSCSI and RDP (along with USB-over-RDP) are only available in the closed source variants of VirtualBox.
    The opensource edition of Virtual Box doesn't have them.

    Also the USB support may lock the system when in fast emulation/patching/ring-2 mode, and only works flawlessly when using the slower mode with virtualisation CPU extensions (my brother tried using it to get old USB hardware accessible when moving to Vista 64 but since then he ended up buying newer hardware)

  • by Skjellifetti ( 561341 ) on Tuesday August 12, 2008 @11:40AM (#24569465) Journal
    In a major corporation, fear of massive fines and prosecution is enough to stop them from pirating your software.

    Not true. I worked for a smallish software companies that had their software replicated in at least one large customer installation well beyond the number of seats that were actually paid for. When confronted, the reaction was "so sue us..." We eventually settled for about 1/10 of what we would have made if they had obeyed the license terms because the cost of litigation coupled with the delay tactics they could have used would have meant that we would be out of business long before the court case was over. Size just means that they have more resources to defend their slimy actions.
  • by DrYak ( 748999 ) on Tuesday August 12, 2008 @11:44AM (#24569567) Homepage

    If a virtual machine would support something like DirectX or OpenGL so that I could have the kids running their games in a virtual machine (and being able to install them, etc.) I would have them set up with a locked down OS with a virtual system for their games. {...} But I'm sure the technology is getting closer.

    Yup. Indeed. /. mentioned recently "VMGL [toronto.edu]".
    The extension is open source but currently only works for X11 OSes at both end.
    But as you said, a working acceleration layer is bound to be developed in the near future for Windows too.

  • by tsstahl ( 812393 ) on Tuesday August 12, 2008 @11:45AM (#24569575)

    I'm looking at you North Winds Software. I'll BUY a support contract! If you offered such a thing. If you answered the phone.

    Um, isn't North Winds the name of the company that comes with the sample Access database? They're not real, you know... ;)

  • by nurb432 ( 527695 ) on Tuesday August 12, 2008 @11:57AM (#24569797) Homepage Journal

    Ya, its free, and virtual box is no enterprise solution either. So we are comparing apples and oranges here..

    Sure, its cool on a persons desktop to run non critical stuff on, but its no where close to being a product you would run off and virtualize 1000+ production servers with, which is the target market for ESX

  • by initdeep ( 1073290 ) on Tuesday August 12, 2008 @12:26PM (#24570333)

    you've never been to a heart surgeon have you?

    Many of them are minorities.

    in fact, in my experience (having a parent formerly work for one, and the other now seeing one regularly), the people considered minorities in society make up the majority of the heart surgeons.

  • by Anonymous Coward on Tuesday August 12, 2008 @12:29PM (#24570365)

    Use USBAnywhere. It sucks they don't support native USB passthru yet, but this works in the meantime.

  • by operagost ( 62405 ) on Tuesday August 12, 2008 @12:41PM (#24570549) Homepage Journal
    Isn't the sort of thing that the BSA (no, not Boy Scouts) is useful for? Maybe you should join?
  • by tlacuache ( 768218 ) on Tuesday August 12, 2008 @02:14PM (#24571843)
    As far as I understood it, VirtualBox does support USB in the binary distribution you can download here [virtualbox.org]. VirtualBox's "Open Source Edition (OSE)" doesn't support USB, see here [virtualbox.org]. But if you're running VMWare Player (a closed-source product) anyway, the non-Free/Open aspect of it must not be a hangup for you (nor is it for me). So what's your holdup for running the "closed" distribution of VirtualBox. You'd have USB and, unlike VMware Player, could actually create new virtual machines.
  • by Sits ( 117492 ) on Tuesday August 12, 2008 @02:32PM (#24572255) Homepage Journal

    How can you reference Ken Thompson's "Reflections on Trusting Trust" [bell-labs.com] (HTML/non-PDF version) without also mentioning David A. Wheeler's "Countering Trusting Trust" [dwheeler.com] (as found via Bruce Schneier's blog [schneier.com])? So to answer your question:

    What if you can't even trust your compiler?

    Well so long as I have another set of compilers AND at least one is trustworthy then there is process I can follow to build a compiler I can trust. After spotting differences in the resulting binary I would also need to (ah-ha) examine the source code of the used compilers and find out which one is mis-generating the binary and fix it.

    At some point I need to be able to understand binary and read the source of the compiler that generated that binary to ensure that someone else is not jacking me.

  • by matuscak ( 523184 ) on Tuesday August 12, 2008 @02:33PM (#24572277)
    FWIW, ESX does in fact use FlexLM. In this case, it's not the license server that has gone down, but the code in ESX that is interperting the what the license server is returning that is broke.
  • "VMWare licenses for ESX server cost something like $5k apiece."

    That's an exaggeration by a factor of five. Admittedly it ain't cheap, but one can get three dual-processor (unlimited core) ESX licenses and a management software license for $2700, or just ESX server for $1000.

    Of course, today it doesn't look real attractive...

  • by Arterion ( 941661 ) on Tuesday August 12, 2008 @03:55PM (#24573881)

    Don't you love the way that everyone in the world gets more respect than the local IT department? Anything a vendor, friend, or the internet says is completely valid and true, but if it comes out of the mouth of the IT department, it must be wrong.

  • by mrpolyrhythm ( 802924 ) on Tuesday August 12, 2008 @04:43PM (#24574703)
    The article also says that he'd recommend disabling DRS because that would remove resource pools, and goes on to say set the sensitivity to 5. What would be the more correct course of action, would be to set your DRS Cluster to Manual, which is indicating no automation, DRS will not place or move VMs.

I've noticed several design suggestions in your code.

Working...