More 'Application-Specific' Optimizations in NVidia Drivers 361
EconolineCrush writes "Futuremark and NVIDIA have been embroiled in a spat over various cheat/optimizations in 3DMark03 for several weeks now. Last week, the soap opera appeared to be over; Futuremark and NVIDIA released a joint statement in which Futuremark clarified that NVIDIA was optimizing its drivers for 3DMark03 rather than cheating. This story, however, appears to be far from over. Tech Report has uncovered a new series of optimizations in NVIDIA's Detonator FX drivers that affect image quality in even Futuremark's latest 3DMark03 build. What's more, if you rename the 3DMark03 executable, the optimizations disappear."
and they all told me I was crazy.... (Score:5, Funny)
Mike
Re:and they all told me I was crazy.... (Score:3, Funny)
Re:and they all told me I was crazy.... (Score:5, Funny)
Geez! On a side note, I had heard of something like that where some site recommended that renaming your executable to some "name" would give better results.
I think someone should run "strings" on the nvidia driver to see what other names work...
Re:and they all told me I was crazy..Suspend Them! (Score:3, Funny)
So suspend them for 8 games, unless they are a big superstar and appeal.
And go back in the archives to see if they ever said, "You can test me for steriods any time -- except right now, of course."
Re:and they all told me I was crazy..Suspend Them! (Score:3, Funny)
I can see in now...
--headline--
Microsoft admits to executable doping, anonymous sources heard talking about how they "knew" their executables loaded too quickly. Microsoft spokesperson read a public apology to all executables that had ben resource starved because of the malaction of their executables.
So What? Who Cares? (Score:4, Insightful)
Aren't we smart enough not to be pulled in my marketing hyperzor?
Tom
Re:So What? Who Cares? (Score:5, Insightful)
Sad, but true.
Most people will shop around, to make sure the features they're looking for are simply there and work. Beyond that, they don't do the research to understand which version is better unless they're forced to.
Re:So What? Who Cares? (Score:4, Insightful)
Re:So What? Who Cares? (Score:2, Insightful)
Pretty much any graphics card has drivers for XFree86, the rest should work with the vesa driver.
Re:So What? Who Cares? (Score:2)
Re:So What? Who Cares? (Score:5, Insightful)
You are... I am... And a bunch of /. readers are too. But the millions of teenagers that grew up without technical skills but love games and subscribe to gaming magazines are looking at the benchmarks to decide what to ask for X-Mas and what to beg Mom to buy at CompUSA. I've seen it happen in person with a girlfriends younger brother. The difference between Quake II scores of 110 FPS to 112 FPS was a world of difference. Of course, the card with another 2 FPS was bought! (Not actual numbers, but I remember the difference was in fact 2 FPS!)
That said, what NVidia is doing is cheating, plain and simple. No laywer or press release can spin it otherwise. Well, they try, but the truth hurts in its simplicity. Change the .exe name and the cheats dissapear. And they are not "optimizations" because when the cheats are working they reduce the quality of the rendered image.
Re:So What? Who Cares? (Score:3, Insightful)
Re:So What? Who Cares? (Score:3, Insightful)
Also, compiling for athlons isn't cheating, as long as 2+2 is still equal to 4, and not 3.96. The new set of issues (hell, the original set as well) involve the program not doing what it's supposed to do: Produce the best quality images. The first set found involve
Re:So What? Who Cares? (Score:5, Insightful)
Re:So What? Who Cares? (Score:2)
Re:So What? Who Cares? (Score:2)
I don't know, I think it's more widespread than that. It's the same kind of fervor that surrounds arguments about the # of instruction units on the Pentium 4 vs. the Opteron. They're people with some technical knowledge--maybe even a goo
Re:So What? Who Cares? (Score:2)
I do not expect outside firms to lie to me, ala Arthur Andersen. If Futuremark can not eliminate this, they will be useless, as the plainly see.
I can't imagine the relationship between FutureMark and any company who makes video cards to be peachy after the auditing scandals of late.
I think NVIDIA and ATI expect to be allowed a little play. But thats not in FutureMark's best inter
It's the reviewers' fault (Score:5, Insightful)
The only reliable way to test is by testing it withthe applications it's used for. Get some actual games, and see what the frame rate is. If they optimise for those tests then it doesn't matter! It means they're oiptmised for real world situations.
Re:It's the reviewers' fault (Score:5, Insightful)
Sure 3dMark is OK for a rough idea of your graphics throughput, but like you said some take it wayyyyy to seriously. Unfortunately, the testing methods you propose (albeit rational) aren't quite sexy enough to sell cards. And certainly unquotable for the magazine ads
"My solitaire looks so colorful." - Mavis Jones AARP
I know you meant 3D apps, but these cards are used for 2D as well and seems to get overlooked in that arena at times.
Re:It's the reviewers' fault (Score:5, Insightful)
As to the "Just use games to benchmark!". It's not that easy. 3DMark is meant to test vid-cards on demos that use future technologies. Games obviously can't do that, since in order to have reliable benchmarks with them, the games need to be released first. And fact is that games are lagging when it comes to implementing new tech. that's why we need benchmarks like 3DMark, that test those features that are not yet used in games.
you and your like say "Who cares? It's the games that matter". But I think that cheating (no matter what's the app) tells quite alot of the company in question. The fact that NV has been found to cheat (repeatedly) tells me that they are scum
Re:It's the reviewers' fault (Score:2, Interesting)
Yes, Dell shouldn't rely on these benchmarks either. Anyone who makes buying decisions based solely on generic benchmarks is a fool.
As to the "Just use games to benchmark!". It's not that easy. 3DMark is meant to test vid-cards on demos that use future technologies
Like what? The graphics card industry is a mature market now. Features aren't changing. We're just seeing more speed. About the only new feature recently has been programmable s
Re:It's the reviewers' fault (Score:5, Insightful)
But this is not optimising, it's deliberately cheating. That's what it's called when you ignore the settings you were told to use and substitute in your own faster ones, simply because you know you're running a benchmark programme used by consumers and reviewers to determine performance. Cheating may not a strong enough word - this almost amounts to fraud, claiming one thing and delivering another.
Re:It's the reviewers' fault (Score:2)
Testing sites should make it standard practice to rename the EXE to some random name before ANY testing is done, and declare such up front on every test just so the manufacturers know it.
It won't prevent
Re:It's the reviewers' fault (Score:2)
You'd end up in an arms race, the likes of which anti-virus and anti-spyware find themselves in today. You put in one measure and the opposition counters with something different.
The only solution is for the
Re:It's the reviewers' fault (Score:3, Interesting)
the lengths people will go to... (Score:2, Interesting)
Re:the lengths people will go to... (Score:4, Funny)
Re:the lengths people will go to... (Score:2)
I agree.
This current time is looking alot like the late 19th and early 20th centuries. This is when the divide between rich and poor was about as extreme as today. 99% of the worlds money is owned by 1% of the people! Ronald Reagan started this craziness and after a taste of greed companies and the wealthy can not have enough. Before he took office the rich were taxed close to 90%! Today my taxes have gone up because I had to take delivery d
Cheaters never win! (Score:2, Funny)
Oh...wait...
Never mind...
High or low level strategy? (Score:5, Interesting)
a) Is this indicative of a high level strategy by NVidia's management, who's marketing department is pressuring them to have higher 3DMark2003 scores than ATI?
OR
b) Has some low level device driver programmer (intern?) looked to get some easy brownie points by "optimising" the drivers for 3dMark2003 in a slightly clunky way?
Either is quite interesting :) I've been a victim/perpetrator of both in the past.
Re:High or low level strategy? (Score:5, Informative)
Re:High or low level strategy? (Score:5, Interesting)
Of course it is. Fudging [bapco.com] the drivers for a synthetic benchmark are a time honored way to make crappy hardware [intel.com] look good.
Make NVIDIA drivers Open Source! (Score:5, Interesting)
Benchmarks would reflect the actual performance of the card instead of skewing the results in order to garner favorable reviews.
Only when we allow Nvidia to see the benefits of Open Source can we free the graphic benchmark software from the clutches of Matrox.
Re:Make NVIDIA drivers Open Source! (Score:5, Informative)
In practice, nVidia have made it painfully aware on numerous occasions that they CANNOT do this. Its not just them being nasty closed-source meanies. The driver binaries contain licensed tech from numerous third parties that their license doesn't let them reveal the source to.
Re:Make NVIDIA drivers Open Source! (Score:4, Insightful)
That's right, so Nvidia should instead release specs (not under an NDA) on their chips, just like almost every other IC manufacturer in the world.
Re:Make NVIDIA drivers Open Source! (Score:2)
Hello?
We don't want their stinking source code. We want the specs. Even if their binaries are encumbered, the specs aren't.
Re:Make NVIDIA drivers Open Source! (Score:2)
Re:Make NVIDIA drivers Open Source! (Score:3, Funny)
Re:Make NVIDIA drivers Open Source! (Score:2)
Renaming (Score:5, Funny)
Re:Renaming (Score:5, Funny)
It also sent an e-mail with "I Love You" in the subject to all of your contacts. I wonder if the reason for delays on the FX card was that they'd renamed the burn-in application DukeNukemForeever.exe?
Thank you, thank you. I'll be here all week.
Re:Renaming (Score:5, Funny)
Damn, NVidia's going to be pissed. They delayed the launch of the FX cards and misspelled the file name.
Re:Renaming (Score:5, Interesting)
Actually, that is very interesting. Any bets you've stunbled onto an entirely different cheat?
Let's say Microsoft wanted Outlook to have some special capabilities in the operating system. So the OS recognizes it and gives it special treatment. Another app comes along with the same name and triggers the "special treatment" but can't handle it. Ka-Boom.
This also brings up the possibility of really screwing with these drivers. Go get another game program (QUAKE.EXE or whatever) and rename it to the name of the benchmark. What does the driver do to it?
Re:Renaming (Score:3, Informative)
I will take a wait and see attitude. (Score:5, Insightful)
It doesn't make sense to buy a card to run Doom 3 when the game isn't out. Here is a clue, when Doom 3 does come out I will be able to buy something as powerful as the FX 5900 for $150.
I'm going to go into an offtopic rant now. It is sad that we have huge displays and crazy-go-nuts graphics processors on computers, but consoles will probably always beat PCs for game size. Game makers are too scared to release a DVD only game, so our games are limited to 700MB by disk, and don't even get me started on controllers.
Re:I will take a wait and see attitude. (Score:3, Insightful)
console controllers (Score:2)
Re:I will take a wait and see attitude. (Score:2)
Whatever happened to Origin? I can remember three distinct occasions where I upgraded my PC just so I would be able to play an Origin game (Wing Commander III, IV, and WC Prophecy). Point is, if you release a good enough game, people will upgrade their PCs to get it.
Maybe we'll see Doom3 on DVD? It's a thought.
Re:I will take a wait and see attitude. (Score:2)
Re:I will take a wait and see attitude. (Score:3, Interesting)
http://www.origin.ea.com/
Technically, they're still around. However, after being purchased by EA (IIRC), they lost (EA fired) the majority of their creative and technical talent. The result? Origin exists only in the capacity of endless UO expansions.
> Point is, if you release a good enough game, people will upgrade their PCs to get it.
This is no different on the PC than it is for anything else. Houses, cars, console systems, PC, guns, ceiling fans... when one doesn'
Re:I will take a wait and see attitude. (Score:2, Interesting)
I'm not trolling or anything I'm just tired of people complaining about the game industry not releasing on dvds, its always the same people who also complain about them pushing newer hardware.
Re:I will take a wait and see attitude. (Score:2)
Just to clarify I love PC gaming, but I just believe it has a couple weaknesses. This particular weakness could be addressed with a $30 dvd drive. When most gamers spend $100 for a soundcard and even more for a videocard it makes
Re:I will take a wait and see attitude. (Score:2)
Game movies always look better on a TV screen. The nice low resolution display gives it a little bit of poor man's interpolation. Meanwhile, the crisp hi-res display of the PC means you can see every artifact in its ugly glory.
Re:I will take a wait and see attitude. (Score:3, Interesting)
Re:I will take a wait and see attitude. (Score:5, Interesting)
Today, your 120GB disk would require over 170 CDs (at 700MB each) to fill up (leaving compression aside, this is just a back-of-the-envelope estimate). By contrast, a 4GB DVD brings the ratio to 30 disks. Roughly comparable to the numbers 18 years ago. Of course, we are moving on to larger disks; 250GB is now the top end.
Due to the antics of copyright holders and media/drive maker "consortiums", the rate of expansion of distribution media has fallen behind that of fixed storage media. The DVD has still not become the accepted distribution format, despite the fact that DVD drives are nearly as cheap as CD drives, and DVD writers are becoming affordable.
It's time for the computer and software manufacturers to get their act together and move to the next logical step for distribution.
Re:I will take a wait and see attitude. (Score:2, Insightful)
For first-person shooters, maybe. (and some RTS games, of which very few make it to consoles)
The DualShock2 is the best controller out there for just about everything else. It's comfortable, it has intelligently placed buttons, it has enough buttons, but not *too* many, and for me, the most important part is that I forget I'm holding a controller when I'm using it. It helps me 'just play' and not be distracted. I agree that there are a f
ATI Drivers SUCK!?? Oh wait.. (Score:2)
I'm finding that the 4x.xx drivers don't work that well with my GForce 440MX the TV-Out is screwy, the best drives for me are 30.82 I wish they would fix that instead of optimizing for pointless benchmarks! I'd rather have a driver that works than one that gives me a bigger 3dMark score!
Re:ATI Drivers SUCK!?? Oh wait.. (Score:2)
Who cares about benchmark software? (Score:5, Insightful)
So throw the benchmark software out, fire up Q3 or whatever, and let us know how the card really performs.
Re: (Score:3, Informative)
Re:Who cares about benchmark software? (Score:5, Interesting)
Re:Who cares about benchmark software? (Score:2)
Re:Who cares about benchmark software? (Score:5, Informative)
and the fps thing.. well, you're just outright wrong on that, I can easily see a difference between 30fps and 120fps
Re:Who cares about benchmark software? (Score:5, Informative)
43 76 & 125 all produce similar results.
http://ucguides.savagehelp.com/Quake3/FAQFPSJum
Re:Who cares about benchmark software? (Score:3, Interesting)
I KNOW 125fps is way above the human limit. I tend to disregard any comments people make about being able to tell the difference between frame rates higher than 60. While we are technically able to tell the difference between 60 and 70, it's too damn hard
So? What others are optimised? (Score:3, Interesting)
Shades of 3DFX in nVidia (Score:5, Interesting)
nVidia is walking a tight rope and for the first time in six years I'm actually going to consider buying an ATI. Come September 30th there is 90% chance that I'll have an ATI card on my machine.
Re:Shades of 3DFX in nVidia (Score:2)
Re:Shades of 3DFX in nVidia (Score:2)
Lies, damn lies, and benchmarks (Score:3, Insightful)
Why isn't the benchmark a supervisor that renames the real benchmark to some random name, then runs it.
Seems to me the trick is to stay one step ahead of the marketers.
Re:Lies, damn lies, and benchmarks (Score:2)
Insightful? (Score:5, Insightful)
People are crying about this because they rely upon benchmarks as a gague of how powerful a card is, and make purchasing decisions around such knowledge. Sure, some of them forget that a %5 difference is meaningless in real-world performance, but that doesn't mean that the overall scores are meaningless. nVidia's last round of cheats pretended that the card was 25% faster overall than it actually was. This particular cheat ads between 8 and 18 percent to the total, with the largest false total going to the most expensive card.
In other words, if you bought a $300 nVidia card on the strength of this benchmark, you bought a card that is %40 slower than it should be because nVidia went out of its way to lie about the speed.
ATI optimized for the test, they re-ordered the way in which the card handled executions similar to the way someone might re-order their day for maximum efficiency. It was a cheat, but a minor one that only added 2% to the score. nVidia's cheat involved dropping instructions entirely, equivalent to doing more in a day by checking things off your list without actually doing them, letting the food rot in the kitchen and the dirty laundry pile up.
The sad fact of the matter is that nVidia now has a big problem, in that their fastest card which managed to eeek out ATI's fastest cards can no longer claim that crown, and yet it is ATI's turn next to introduce faster cards. Their technology can compete, but can't demand the premium that graphics card developers rely upon to survive. Furthermore, this cheat comes after nVidia promised to clean up their act and remove all cheats from their driver. Not only did they cheat, they promised to clean up their act and yet cheated again in the very driver that is supposed to be clean. Their public image is bloody shot, significantly worse than ATI's was over their Quake 3 debackle. ATI fell back on their technology and released superior cards, but nVidia doesn't seem to be able to head down that road.
To get back to the poster's original position, the benchmark should try to outsmart the developers, though being a test for future games they have to stay abreast of display technology more than cheating techniques. But blaming the benchmark not the nVidia for cheating is like discovering that during a Car-and-Driver top-test the Ford team took a shortcut shaving %40 off of the race course and congratulating them on their ingenuity.
What they did was indefensible. They knew it, they appologized, and they did it again.
new name (Score:2)
For a good explanation... (Score:5, Informative)
later,
Re:For a good explanation... (Score:3)
No Graphics Icon? (Score:2)
ATI/Nvidia cheat, FutureMark is spineless (Score:5, Interesting)
Once again, and this can't be stated strongly enough - synthetic benchmarks really don't tell you what you think you're hearing. Indicative? Yes. Conclusive? Absolutely not. Don't listen too deeply to them.
When this much money is at stake, don't expect to hear the truth from any angle associated with these companies. Remember, we're dealing with marketers and lawyers here...
Checks and balances (Score:2)
At least someone is checking.
Doesn't really matter... (Score:5, Insightful)
Most major vendors have been "optimizing" for Quake III ever since it became the informal benchmarking standard... I think Futuremark has blown the issue up a little since they weren't on really good terms with nVidia before this started.
But really, 3dmark has always been a "gee-whiz" pretty demo of current graphic card abilities, but never a reliable benchmark. In fact, no one program/game can be a reliable benchmark, since performance must be judged on a variety of applications. Only then do you get some kind of idea of where the "real world" performance lays.
The competition between ATI and nVidia is good for us customers; they both have excellent cards now. ATI has the fastest, while nVidia's drivers (yeah linux support is flakey I know) seem a bit more stable than ATIâ(TM)s.
Really, between the two companies, it is hard to make a really "wrong" choice.
So yeah, everyone, these aren't the droids you are looking for, you can go about your business...
Re:Doesn't really matter... (Score:2)
Be thankful that there is Linux support at all. With any other company, you have vendor support, drivers written by a volunteer third party who most likely had to reverse-engineer everything, etc. You will NEVER get TV-out working on an ATI card under Linux for instance, and TV-in isn't terribly functional either.
That would be true if the hardware was the only issue. Since the software
Please! (Score:2)
My Gforce4 is enough to run any games i want, and any future games i cant run then i upgrade to something the developer recommends...
Re:Please! (Score:2)
Also read some other comments, it says they improve quality in the benchmark? I'm not listening to the market spin, i just think Nvida could help tweak applications at the driver level if they just put some effort in, and give some positive gains that people will see.
Card change (Score:2, Funny)
This is about trust and users (Score:5, Insightful)
It would not surprise me to see that much of this is an attempt by NVIDIA to marginalize the value of FutureMark 3d 2003. If a benchmark isn't favorable to a piece of hardware, then make the benchmark a 3-ring circus with these antics - then nobody trusts the benchmark at all.
A sad way to do business and I can't say when my GF3 Ti 200 will be replaced, but it when it is I will not be using NVIDIA. Apparently they don't trust users to make a decision based on an honest assessment of facts.
Image Quality (Score:3, Insightful)
I opened the optimized screen [tech-report.com] and the renamed .exe screen [tech-report.com] in two tabs and have been scrolling to each corner flipping back and forth between them, and I've got to say I actually think the image quality is higher on the optimized one. If you look at the book in the background, you get a hint of text on it in the optimized version, where it's blank in the renamed one. And the bevels on the edges of the desks are a lot clearer in the optimized one. And there's a really jagged edge on the carpet under the left desk in the renamed one that gets fixed in the optimized one. It's not all good, of course. There are some textures on the left wall that are brighter in the renamed one, but it's hard to tell which one would be better without seeing it in action. (My system gets about 10fps on that test, not really enough.) And finally the optimized one has one thing that looks obviously worse, and that's the cross pieces on the rear window, they're a little strange.
So, anyways, even though it's bad that they change the settings to get higher scores on the benchmark, I'd like to know how to change those settings myself, if it improves performance that much and looks (arguably) no worse, or even better.
Re:Image Quality (Score:2)
So, to make you happy, I will build a videocard that, when it detects a certain filename, instead of rendering the picture, it will play a video of a high quality rendering of the picture... That way, it looks great, seems fast, and doesn't need to bother with doing the actual WORK.
More seriously though, the unoptomized one is much sharper. You might think it looks better because it blurs some of what would be jagged edges, but for that,
Re:Image Quality (MOD PARENT UP) (Score:2, Informative)
How about this (Score:3, Funny)
Or is that just being silly?
Then can somebody explain... (Score:2)
Benchmarks are useless (Score:5, Insightful)
When I want to test-drive a new car, do I build a driving simulator based on the car and load in my route to work so I can see how it would handle during a typical commute? Of course not, I just take the car for a test drive, because sometimes the best way to test something is just to use it.
Benchmarks are inheriently flawed because a benchmark is not what your buy the card for. Card manufacturers are always going to "teach to the test" by optimizing their cards and drivers for whatever the reviewers are using to review the cards. So why not take advantage of this and use popular applications to benchmark? The venerable Quake3 framerate test is one example of this, but I would arge that all benchmarks should be numerical data taken from the performance of real world applications. That way, at least those apps will perform as advertised, and any other apps that take advantage of the same features on the card will benefit.
why 3dmark is irrelevant for many gamers (Score:2)
I could see people making arguments that DX9 tests are good because DX9 games will prolly sta
pfth give me a tseng et2000 card (Score:2)
Why not allow the user to choose? (Score:2, Interesting)
Just add a new page in the driver settings where you can add an exe-file and then allow the user to activate the different "tweaks/optimizations". It would be more honest and people who want more speed than looks would be happy as a clam.
"But won't the user be confused by all the options?" I hear you ask. Maybe, bu
cheats vs. optimizations (Score:2, Insightful)
On the other hand when you enable the 8x aniso fi
Re:cheats vs. optimizations (Score:2, Insightful)
You're quite missing the point. nvidia decided to o
Renaming my q3.exe (Score:2, Funny)
Ok ok, I shut up....
Benchmark-program names (Score:2)
Changing the executable name should be standard procedure when running a benchmark for public consumption. Ideally, benchmark developers would go even further and actively try to cloak their identity from the drivers, databases, or whatever software the benchmark tests. You might think this could lead to an "arms race" between vendors and benchmarkers, but I don't think so. To get away with cheating, the vendors' detection of the benchmark would have to be totally foolproof. It would have to guess right
Cheating 101 (Score:3, Insightful)
"When cheating benchmarks, DO NOT use the name of the benchmark app as an indicator"
Sweet Irony (Score:5, Funny)
Re:Linux Optimization (Score:2)
Mine smokes. Q3A on my P42ghz w/GF4-Ti4200 128m is insanely fast. My son says it blows away ANY console he has ever seen and he has a PS2 and plays everything that comes out.
I'm building him a new machine this weekend, a 2.2ghz Mandrake 9.1 box w/+GF4MX440 128m for his gaming..
So far I'm very highly impressed with Nvidia preformance. I was highly disappointed and distressed with driver installation previously but they've made great imp