Researchers Sour on Vista Service Pack 1 Performance 339
Stony Stevenson passed us a link indicating that a group of researchers has described Microsoft's upcoming Windows Vista Service Pack 1 as basically a performance dud. Researchers from the Devil Mountain Software group is claiming that a series of in-house benchmark tests showed that users hoping to receive a speed boost from the update will be disappointed. "Devil Mountain ran its DMS Clarity Studio framework on a laptop Barth described as a "barn burner" -- dual-core processor, dedicated graphics, and either 1GB or 2GB of memory -- to compare performance of the SP1 release candidate that Microsoft released last week with the RTM version that hit general distribution last January. The Vista RTM was not updated with any of the bug fixes, patches or performance packs that Microsoft has pushed through Windows Update since the operating system's debut. 'One gigabyte, 2GB [of memory], it didn't make a difference,' said [CTO Craig] Barth. 'SP1 was never more than 1% or 2% faster.'"
Are we shocked? (Score:3, Interesting)
Has it ever improved efficiency? (Score:5, Interesting)
Without wishing to troll, when has a Window service pack ever improved the speed of a Windows OS?
In fact, and I'm sure someone on Slashdot has raw data on this (that perhaps even shows I'm wrong), Apple are the only company who has ever achieved this on a regular basis.
I've found in my rather short development career is something scarily similar to the first law of thermodynamics: "Bad code once created can never be destroyed." In most commercial situations, the risk of breaking a routine far outweighs the benefit the change brings.
We've built an entire area of study, refactoring, on trying to sell the importance of keeping code clean. I'm still not 100% convinced that the case for refactoring has been made. If you spend three months refactoring, is the simpler overall structure really going to speed up development sufficiently to justify the capital outlay? In all but the very worst code-bases, the answer is unclear.Bear in mind, refactoring my cause you to notice bugs that you can't fix because it would break an interface. Now your code has to be badly structured to support this bad business logic. This can be enough to render the effort useless.
This is why service packs rarely improve functionality or performance. Windows XP SP2 is a notable exception. The risk is simply too great.
Simon
Why I even care one bit (Score:5, Interesting)
So I notice Crysis has a "Very High" setting that's disabled for me in XP. Ok, I think, the first half or so of the game runs ok with High settings, so maybe it might just barely be playable on Very High. Just to be able to see what it looks like.
I boot into Vista and install the game there. Lo and behold, it runs at almost exactly half the FPS on High compared to in XP. Had to drop it to Medium to be even remotely playable. Needless to say, Very High is what I'd need to be to enjoy it with everything at max.
Is the culprit crap drivers for my hardware, general performance drain by Vista, or DRM using everything it can to make sure I'm actually allowed to use the computer today? I don't know, but I do know Vista has made me seriously try a Linux on a desktop for the first time (only used it for servers until now). If only more games supported it, or ran under Wine, I'd be happy as can be.
Didn't we have a similiar story five days ago? (Score:2, Interesting)
Only, that one was from PC World Canada.
AND... they at least listed the RC's version (0.275) and explained the tests (well, kinda...),the difference in performance AND the hardware used. http://www.pcworld.ca/news/column/3eef651f0a010408008b33e8065121c5/pg1.htm [pcworld.ca]
WTF is a "barn burner"?
Also, saying "Office-based test script was "statistically insignificant,"...while a multitasking test panel produced results for SP1 less than 1% faster than RTM." doesn't really say much.
Adding to that the first (T)FA actually bothered to mention WHAT was the RC about...
In fact... first thing that comes to my mind after reading TFA (the "Barth said"-part) is Samuel L. Jackson in Pulp Fiction:
"Check out the big brain on Barth!"
Re:Optimization (Score:5, Interesting)
The hard part is usually not the optimisation, it's working out where the optimisations need to go. This typically involves wading through huge amounts of data from profiling runs.
Re:Are we shocked? (Score:5, Interesting)
The slow file copy isn't a joke. We're talking 1hr+ to copy 2.5GB to a FW hard drive from internal SATA. That's about 25MB/min, 120 times slower than the peak speed of FW. I think you can get more out of a parallel port.
There are some nice additions. But it's not worth the trouble, as some of the flaws totally override those.
Re:Are we shocked? (Score:3, Interesting)
So after you are faced with buying new hardware AND new software, it makes you feel a lot less locked-in to MS and compelled to get a Windows machine. I think this is part of why Mac is seeing its fortunes reversed a bit - for the few applications that you can't find for MacOS (or Linux), you can just boot into your old copy of Windows.
Vista's not slow (Score:2, Interesting)
So why is your machine so slow?
Re:Vista's not slow (Score:2, Interesting)
Not only are we happy to push our machines well beyond reasonable, we also tend to take hardware that was never meant to approach even their own specs -- consumer hardware specs are maximums, not typicals.
For my own Vista machines, I didn't go with the minimum ram -- and 2GB is minimum for Vista. I didn't go for half of Vista, I meant Business or Ultimate. I didn't go for the cheap hard drive, the centrino processor, or the slow ram.
The other issue with using power users as any sort of benchmark is that our needs and desires and playfulness grow much faster than the industry -- in part because we are driving the industry. A new super car is invented, and some car guy sees how fast it can go before it breaks. A new plane, and a test pilot takes it out. We get a new OS, and we see how many things it can handle concurrently.
Game developers do the same thing. They build games with adjustable detail. We see that as a way to dial down the graphics for lower machines. We forget, or ignore, the fact that the developers build the very same detail adjustment to dial it up for future machines. But we want to play every new game at maximum detail, which effectively means that the game is old the moment we play it, and that it wasn't developed to take advantage of future hardware. Obviously game developers build for future hardware. We power users push to play that higher level now, and complain when we can't see the best possible version the instant the game is released. Maybe those game developers should hide the higher detail levels from us until six months after release.
Ultimately (heh) I don't see Vista as anything more than a much better OS than XP (I mean in the network, business, and other internals, not aeroglass) that, as with every other software development, will take advantage of hardware improvements over the next two years. We'd be upset if it didn't.
The question I pose to you is not whether Vista now is faster than XP now. I pose to you is Vista faster now than when XP was new? And even more appropriately: will Vista be faster at the end of its life-time than XP is now, at the end of its own.
Re:Why I even care one bit (Score:3, Interesting)
Firstly, I can confirm for you, yep Vista sucked for me too, same driver versions, fully patched machines and the Vista install has several bullshit disk thrash services disabled, it still ran at 34 FPS avg in the benchmark at X settings.
XP ran at 45 FPS avg, same system, same benchmark and settings.
Also the "DX10" features in Vista ARE available in XP with some ini hacking, do a google on it, I think DIGG covered it.
Vista, more like shitsta.
Re:Straw Man? (Score:4, Interesting)
It's not just Vista [blogspot.com] though. Microsoft Office 2007 on Windows Vista consumes over 12x as much memory and nearly 3x as much processing power as Office 2000.
The missed point... (Score:3, Interesting)
1) Some of the performance updates scheduled for SP1 were already released as Updates.
2) Performance on a System of 1GB (the sweet spot) will see virtually no improvement, and they are reviewing systems with 1GB and 2GB or more. If you baseline the performance difference on a 512mb system the performance difference is more dramatic.
3) There are also a few optimization that don't affect most users. Readyboost got a significant jump in how it improves performance, and there has been refining of Superfetch as well. This includes not only USB flash, but Solid State and hybrid Drives will see significant boosts.
4) File copying in RTM did have some performance problems but the majority of the problem was the screen not accurately reporting it was already copying files when it said 'calculating time', so SP1 gets about a 10% boost, but the dialog reports the process more accurately as well.
If Windows Update wasn't doing its job and the updates hadn't already been being released, SP1 would be more of a one time dramatic increase. Also they need to be looking at lower end system when testing if they want to see more SP1 improvements.
Finally, older and pre-Vista designed system configurations see more of a bump as well. If you test SP1 on a system that has the specific chipsets and HD Audio, etc that is designed for Vista, SP1 won't add a lot, as the system components were already designed and optimized for Vista.