Carmack On ATI's Driver Modifications

CitizenC points out that John Carmack's .plan file has been updated to discuss ATI's driver optimizations. If you weren't paying attention, ATI put code in their drivers to optimize for Quake3, based on the name of the executable - so when running Quake3, you'd get a (good) set of optimizations for the game, but when running the same game after changing the name of the executable, you'd get a default set of optimizations with lesser performance. Some people called this cheating since Quake3 is a typical benchmark application these days.

To boil it all down...

[trilucid]

"Making any automatic optimization based on a benchmark name is wrong. It subverts the purpose of benchmarking, which is to gauge how a similar class of applications will perform on a tested configuration, not just how the single application chosen as representative performs.

It is never acceptable to have the driver automatically make a conformance tradeoff, even if they are positive that it won't make any difference. The reason is that applications evolve, and there is no guarantee that a future release won't have different assumptions, causing the upgrade to misbehave. We have seen this in practice with Quake3 and derivatives, where vendors assumed something about what may or may not be enabled during a compiled vertex array call. Most of these are just mistakes, or, occasionally, laziness."

Carmack seems pretty well decided on this one, and not in favor of it. He *does* show a bit of support for having super-ultra-tweak control panels on driver config screens, but that's (almost) an entirely different matter.

It seems pretty cut and dried, at least from his perspective. I for one have got to agree with this viewpoint. Of course, to anyone who has said or will say that you should just rename the executable, I completely agree that this is the only real way of getting an objective test out of it.

Card manufacturers who do this sort of thing *will* get egg on their faces when they start hearing all about the crappy performance of their cards after a new game version comes out with different thinking on what's important visually. Unfortunately, as John points out, a lot of the flack will end up on the developers' doorsteps (misplaced, but a lot of gamers won't know that).

Re:Just out of interest...

[Dun Malg]

The driver is just software. When Quake3 asks it to do something for the first time, it simply checks the filename attached to the process handle and sets a flag accordingly.

Why NVIDIA's drivers are closed source.

[be-fan]

To all you OSS zealots out there, *this* is why NVIDIA's drivers are closed source. You can bet ATI would love to put this whole driver fiasco behind them and just steal the high level OpenGL code (an OpenGL driver has to implement the whole GL API, not just hardware interfacing) from NVIDIA's ICD.

ATI had the right to do it but it's HOW that's bad

[tcc]

If they would have DOCUMENTED the changes, and put a button in the control pannel (like where all the antialiasing on off, buffer level and all is) it would have been legit, they could have put it in their press release and send them around WITHOUT getting any flames.

There are ways to do things, you can bend the rules, but breaking it will only get fire back. I'm all for specific-engine optimization, and like carmack says, Conformance trade-offs shouldn't be made at the driver level in general, but to this I'll say, if it's documented and available as an OPTION somewhere, it won't apply only to techies, it can be applied to joe schmoe as well "if you click here, game will run faster with a small quality degradation (that you will probably not even notice because at the speed of events in quake 3)". Heck if they can put something like Quancun Aliasing in the Panel options, you can be sure people will understand specific game optimization as well.

Anyways forcing something to the users without their knowledge is just plain bad and lame. Of course people will end up finding out, and of course it will backfire... what is gain from doing that!? there sure is poor judgement at ATI, and there's probably a bunch of people there right now saying "see? told you so!". Management, you should listen to the people in the lower part of your food chain (i.e. QA testers), they might know more about the end-user market and hardware benchmarkarking scene (not the OEM of course) than you do!).

Re:.plan

[Malc]

1) It is a web site, hence the URL in the story worked.

2) I think it's kind of cool that somebody is still using their .plan file they way they are supposed to.

Re:Poor behavior in a rough industry

[Namarrgon]

The two problems with ATI's "optimisations" and their quality tradeoffs are:

a) There's no way to turn them off, except by hex-editing the app. They happen automatically, and without the player (or reviewer) even realising, especially in the high-speed benchmark mode.

b) This is not just any old game, not even a particularly heavily played game these days. Its major importance is as the #1 benchmark used by gaming sites.

The conclusion is inescapable. This "optimisation" was not made for players, it was made to subvert benchmarks, pure & simple.

And if you claim to prefer a higher image quality, take a look at what ATI has actually done [tech-report.com] to the visual quality of the game!

Re:Why NVIDIA's drivers are closed source.

[Animats]

There's another reason. NVidia has two product lines, the "GEForce" for consumers, and the "Quadro" for professionals. The same chips are used for both [geocities.com], there's a jumper on the card that identifies the model, and the driver turns off some features on the consumer model.