Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
×
Bug IT Technology

Massive VMware Bug Shuts Systems Down 410

mattmarlowe writes "Imagine if Red Hat released a version of Linux, and after it was deployed, customers noticed that any processes with a start date of today would refuse to run? Well, that's what happened to VMware — a company that wants nearly all server applications running in virtual machines within a matter of years." Supposedly a fix will be available ... in 36 hours.
This discussion has been archived. No new comments can be posted.

Massive VMware Bug Shuts Systems Down

Comments Filter:
  • by db32 ( 862117 ) on Tuesday August 12, 2008 @09:55AM (#24567659) Journal
    Exactly. It is a tremendous pain in the ass to track all the stupid license keys and crap in use. Departments frequently need software specific to only their department and outside the scope of normal IT support stuff. Phone numbers, licenses, etc. God forbid any of those companies get purchased or go under, then you are stuck with expensive software that you cannot recover.

    The call home variety is extremely infuriating. On top of whatever nonsense key/activation crap you have to go through, you have to put up with it trying to call home or deactivating itself. MS isn't the only guilty party in this, but those bastards certainly made the situation much worse.
  • by Anonymous Coward on Tuesday August 12, 2008 @10:00AM (#24567721)

    Then give me USB support in VirtualBox. Cause I kinda need that the most.

  • Patch Tuesday (Score:5, Interesting)

    by Thelasko ( 1196535 ) on Tuesday August 12, 2008 @10:19AM (#24568049) Journal
    FTFA:

    VC will continue to show the hosts as licensed and no errors will appear in vmkernel log file until you try to start up a new vm, reboot a vm, or reboot the host.

    Um, isn't today Patch Tuesday? [wikipedia.org] This could be worse than we thought.

  • by _merlin ( 160982 ) on Tuesday August 12, 2008 @10:21AM (#24568075) Homepage Journal

    Having administered ESX, I can say the license management is useful for one thing: it helps you ensure you aren't exceeding what you're licensed for. For example, if you aren't licensed for multi-processor boxes, it will complain until you get a valid license. If nothing else, it gives you some confidence that you will pass an audit.

    License management is also useful for things like MATLAB and OPNET that are licensed per concurrent user: you can install on as many machines as you like, but they need to be able to talk to your license server (not that this is _your_ license server on your network - it isn't "calling home") to ensure that the number of concurrent users is below the maximum allowed. That way, if say, everyone needs to be able to run OPNET occasionally, but not very often, everyone can install it, but you only need to pay for a few licenses. You know you aren't exceeding your licenses because it won't let you launch more instances than you're allowed simultaneously. If your users regularly complain that they can't fire up OPNET due to lack of licenses, you pay for a few more seats.

    On the other hand, I can't stand software that calls home to ensure that it's "genuine" a la Windows Vista, or those stupid CD copy protection schemes. That's bullshit. Things like that make more work for a sysadmin, not less. I only like license management when it helps me, the admin; I don't care what it does or doesn't do for the software vendor. I'm a selfish pig, I know.

    Another thing I can't stand is things like Rational Purify where they attempt to count your "activations" at their end: when you install Purify, it increases the installed count in IBM's system, and decreases it when you uninstall. If the IBM server thinks you're using all your licenses, you can't install. Too bad people always forget to uninstall Purify before wiping their computers for a clean OS install (or scrapping the computers)! And don't get me started on how bad it is to deal with IBM's phone support. This is one copy protection scheme that I do bypass: I install Purify in a VMware virtual machine, snapshot it, uninstall Purify, and roll the virtual machine back to the snapshot. That way, Purify will work in the virtual machine, but IBM's servers will think I haven't used any of my licenses. Also, I can make copies of the virtual machine for multiple people to use. It's easier for me to track the licences than put up with a crap license management scheme.

  • by Sobrique ( 543255 ) on Tuesday August 12, 2008 @10:23AM (#24568125) Homepage
    I've had oodles of grief from VMs running as DCs for exactly this reason - they pick up clock skew as they're not running _quite_ in real time. And so they drift, and as soon as they hit the ... is it 5 minute? Kerberos window, your whole domain goes nuts.

    Troubleshooting that one was fun.

  • by Bert64 ( 520050 ) <bert AT slashdot DOT firenzee DOT com> on Tuesday August 12, 2008 @10:27AM (#24568193) Homepage

    It's a typical case of companies shooting themselves in the foot.
    Freely available software is already compelling enough and gradually taking over many markets, adding additional artificial costs just serves to make the free/oss option even more attractive.

  • by kungfugleek ( 1314949 ) on Tuesday August 12, 2008 @10:29AM (#24568231)
    Maybe it's easier to get away with dumping chemicals and defrauding investors because of the numbers and motives of the people involved:
    • Defrauding investors only involves the highest level executives, and they keep that kind of thing pretty secret.
    • Dumping chemicals isn't watched as carefully as Windows licenses (for an example) and I doubt the ones who order it or the ones doing it are motivated to talk about it.

    In the case of pirated software, especially something widely used in the company, there would be a lot of eyes (the software vendor watching like a hawk), and a lot of support calls attracting attention from said vendor. Piracy probably happens more on small scales though, where, like you said, you can manage the exposure.

  • KVM and XEN (Score:5, Interesting)

    by kenp2002 ( 545495 ) on Tuesday August 12, 2008 @10:32AM (#24568291) Homepage Journal

    The Open Source Model gets a leg up again after this nonsense. A client of mine just ported all their VMs and said good bye to VMware. That's 280 VMs by the way. Thank God we had a contingency plan for switching VM providers for a DR exercise a year ago and here we go.

    Management is pretty upset and I doubt we will be switching back any time soon to VMWare products after this.

    On a side note this scenario did prove one thing:

    Having a VM-agnostic storage makes migration easy. We changed a mount point, powered on the alternate VM host and we were off and running just that quick. We lost the ability to do live migrations for now but beyond that is was a good opporunity to see just how important an VM-agnostic disk storage array is. (I'm not the admin of those machines but I believe we are using iSCSI).

    On my side though I had about 50 scripts tapping VMWare via PERL but I guess I can start building workarounds now... No more batch submission and dynamic routing for a week or two... The part I hate the most was I had a nice script to take a batch submission and if necessary migrate a utility node to bigger hardware to accomidate the batch... pisses me off but what can I do, thank you Vmware, that aquisition seems to be improving your product as much as when Symantec aquired Ghost Corp!

  • by Bert64 ( 520050 ) <bert AT slashdot DOT firenzee DOT com> on Tuesday August 12, 2008 @10:34AM (#24568339) Homepage

    But then if your license server is down the software won't run, creating an artificial and unnecessary dependency. Similarly if people leave it running they can denial of service other users.

    Plus you have the additional unnecessary cost of the license server, the hardware it runs on, the os it runs on (assuming its not free), the power it consumes and the time required to keep it running and updated.

    License management doesn't help you, it hinders you... If you use software where the license says you can install it on as many machines as you want and use as many instances as you like, you don't need to worry... You can install it on as many systems as you want, with as many processors as you want, without any artificial restrictions and without having to do any nasty hacks.

    The same can be said of pirate software, it typically has all these onerous schemes hacked out, making it a better proposition than the original.

  • by laffer1 ( 701823 ) <luke&foolishgames,com> on Tuesday August 12, 2008 @10:37AM (#24568379) Homepage Journal

    Simple, the industry goes through cycles. Virtualization is hot and some people love it. They want to run it even if there isn't a good reason for it. Some people mistakenly believe it improves security.

    Virtualization is good for testing software and a few other cases where you need to run a different OS but don't want to deal with dedicated hardware or dual booting. I don't see any use in server environments except possibly web hosting.

  • by drachenstern ( 160456 ) <drachenstern@gmail.com> on Tuesday August 12, 2008 @10:59AM (#24568763) Journal

    The more important variant of that question is does the parent want to share enough of the details of operation (clean room style) to get someone to want to write an OS equiv.

    Don't misunderstand me, I like to write code, but if I don't know what the hole looks like, I can't carve a peg to fit it...

  • by Cormacus ( 976625 ) on Tuesday August 12, 2008 @12:05PM (#24569977) Homepage
    Well, that is the way pretty much all software companies seem to be going. The longer they can keep a particular application in service (ie, being sold to their customers) the higher their return on investment

    ( [# copies sold * cost/copy] / [# man hrs in development * salary/hr] == return on investment)

    But they have to be able to keep selling compiled copies of that particular codebase if they want to follow that method. The problem comes in when a piece of software is *good enough* and the customers don't want to buy the next version.

    (Case in point - my copy of MatLAB 5.0 that I got as a college freshman still works just FINE. It doesn't have all the fancy shmancy features that v7.6 has, but I DONT USE those features. Why should I continue to support Mathworks & Co by purchasing another license? I won't . . . unless my license expires . . .)
  • by Anonymous Coward on Tuesday August 12, 2008 @12:13PM (#24570113)

    Free software can get companies shut down, and corporate officers put in prison, due to Sarbanes-Oxley or HIPAA law violations.

    One example. I have two operating systems. One is a F/OSS distribution of Linux. The other is Windows or a FIPS certified Linux distribution like SUSE or RedHat. This is a company that is publically traded, so falls under Sarbox.

    Someone penetrates the machine via a bug in the OS and causes damage, or obtains info. With Windows, or a certified OS, I can tell the auditors that the operating system was certified by the US government with FIPS and/or Common Criteria, and I have used due diligence.

    With a non certified OS, I do not have this protection. I can be held culpable because I did not follow due diligence. If the shareholders are pissed enough, someone is going to prison, and the company may be shut down by the SEC if the breach was big enough.

    Free software is unusable by a lot of businesses for this, and contract reasons. I know a lot of businesses who pay Symantec and Mcafee tens of thousands of dollars a year for antivirus products installed on Solaris and AIX machines. Not because they will ever see a virus, but to check off a contract clause.

  • by adisakp ( 705706 ) on Tuesday August 12, 2008 @12:19PM (#24570207) Journal
    There's an Ask Slashdot for you. Is there something out there that can replace this magic bit of software? Is anyone interested in writing an Open-Source equivalent?

    No there aren't any. Question answered, no need for an "Ask Slashdot"

    Slashdot geeks get excited about writing OSS to be used by first of all themselves, then other geeks, then artistic or creative types.

    Writing free software primarily to be used by what the original poster said is (scum-sucking implied) lawyers at his multimillion-dollar lawfirm is probably near the bottom of their charitable use of their free time in OSS development.
  • Comment removed (Score:2, Interesting)

    by account_deleted ( 4530225 ) on Tuesday August 12, 2008 @01:13PM (#24570963)
    Comment removed based on user account deletion
  • I Am So Dead (Score:4, Interesting)

    by Hasai ( 131313 ) on Tuesday August 12, 2008 @01:18PM (#24571031)

    *sigh*

    Well, it's for real. I've confirmed it here, and my whole data center is affected.

    It's time like this when I wish I hadn't left the Army; at least there, you can shoot back.

    This is going to be one hell of a long night. :(

  • by Anonymous Coward on Tuesday August 12, 2008 @01:21PM (#24571055)

    I'm on the other side of the divide---a tiny company that's not too much more than a guy in his garage (just a few of us), and frankly, I agree with you. I'm astonished at the way we do things, even though we sell to huge firms (including big law firms, like yours). Part of it is just size---we don't have the people or skills to do all the safety, security and support steps a big corporation would. Still, freaks me out that the crap I wrote is out there being used to do important things by important people who don't realize how dumb the guy who wrote their software actually is.

  • Sounds Like It.... (Score:3, Interesting)

    by maz2331 ( 1104901 ) on Tuesday August 12, 2008 @04:00PM (#24574013)

    but...

    If you actually know what you are doing, Access is actually a pretty good development platform. It really is what VB should have been all along. Doing it correctly isn't for the faint of heart nor the inexperienced "guy who knows computers in the department" developer though. It's a LOT of work.

    The biggest issue is that MS markets it as a database app, not a dev platform.

    But there are some caveats to its use.

    1. Never bind controls that can be edited to any datasource. Sorry, but you really need to write code to fill them in, check them, then write them to the back end.

    2. Never store any data in an MDB file. Always use a real backend server such as MSSQL, Oracle, or even Postgres or MySQL.

    3. Once it works, create an MDE file and only run MDEs on clients, never the "source" MDB.

    4. You are checking your db schema revision and comparing it to allowed client app revisions, right?

    Still, there are newer platforms available, and quite often a web-based app will be easier to build and maintain.

  • by dotgain ( 630123 ) on Tuesday August 12, 2008 @05:23PM (#24575253) Homepage Journal
    That's how it works in theory. In reality, the impact of snapshotting & replicating our VMs has been worse than simply shutting them down and copying the images - and we don't seem to be alone, google "esx snapshot hang". It turns out merging a 4 gig "delta" back into a Virtual machine can take hours and hours, with the VM seeming to be hung most of that time.

    Implementing VMWare ESX went from one of the most exciting to the most annoying and disappointing project I've ever witnessed. As someone else has said, fortunatetly the hypervisor itself is quite stable, but most of the support apps are horrendous. VMWare Infrastructure Client is the slowest and most unreliable app I use. We've already lost one 200GiB virtual disk - the file was there but it refused to honour it as a "Virtual Disk" rather than "File". Good thing it was only a test server, but it's surely only a matter of time before we lose a production disk.

    My prediction: In two years we'll look back on VMWare ESX, cringe at all the data-eating server-downing bugs we've found, wonder what all the fuss was about, and go back to *shock horror* running multiple services on single machines again, using operating systems capable of protecting one process from another.

    Maybe I'm bitter and our project itself hasn't gone well, but I know I'm not alone. Not alone in wondering why my Linux PC at home with two SATA disks pisses all over our main fileserver who looks down a 4Gb FC at ten-disk 15kRPM Fibre Channel Stripe. Yes, it's an EMC SAN for those of you wondering quietly. One day, someone's going to cluster a bunch of old PowerMacs with USB/Firewire drives, software RAID, and show those EMC fuckers up.

"Gravitation cannot be held responsible for people falling in love." -- Albert Einstein

Working...