Carmack On ATI's Driver Modifications 219
CitizenC points out that John Carmack's .plan file has been updated to discuss ATI's driver optimizations. If you weren't paying attention, ATI put code in their drivers to optimize for Quake3, based on the name of the executable - so when running Quake3, you'd get a (good) set of optimizations for the game, but when running the same game after changing the name of the executable, you'd get a default set of optimizations with lesser performance. Some people called this cheating since Quake3 is a typical benchmark application these days.
Well them... (Score:2, Insightful)
Re:Well them... (Score:0, Insightful)
there is starting to be a little precidence with optimized drivers, recently AMD released a patch to fix the CPU detection in a video encoding program commonly used in cpu benchmarks. the program was just checking for sse compatibility on intel cpus and amd just made it so they would check it on amd cpus as well. most reviewers were quite worried at the time about whether things like that would become common, but ran with it because they considered it as something that allowed a better cpu comparison since it compared chips using the same feature set.
what ati did is vastly different, they tone down texture quality if the game you are playing is quake3, even if you specify high quality textures in game. blocking how the programmers intended something to display in order to make yourself stand higher in benchmarks really IS cheating.
Poor behavior in a rough industry (Score:5, Insightful)
Carmack's right, this whole driver situation in general is a slippery slope. One of the biggest hassles of his job since Quake (#1) came out has been to get all the graphics card companies to play nice and write good drivers. This has not happened for one second. Some companies are better than others (Nvidia is an example of one of the better companies in the field all around), and the situation right now is FAR better than it was. But, between the race for speed, trying to keep up with the most popular applications, and having to support various APIs being pushed by different 500 lb gorillas in the field... it's a mess. ATI was ALWAYS a mess with their drivers, so this doesn't really surprise me. But this isn't right, cause it'll lead to a day when nothing but the lastest Quake game works on new graphics cards. I mean, nothing works AT ALL. And if it's easy enough to get to that situation, I can just see bribes and payoffs... ahem, "strategic partnerships" being made to accomodate game vendors who want to publish a working 3D game at some point in time...
But anyway...
One small comfort is that no one buys ATI cards for performance. Any cards of the GeForce (Nvidia's brand) variety handily whip anything at the same price point with ATI. The big thing ATI has is OEM agreements, and they also sell some really exotic TV/Video Capture/MPEG recording cards that are really snazzy sometimes. I know cause I'm using one of em right now. But their drivers suck in a lot of ways, they were never the fastest, and I'd love to see them stick to a product release schedule EVER. This is the kind of company that gets wiped out when someone new on the scene releases something better/faster/cheaper.
Unfortunately, the last time we saw better/faster/cheaper in the graphics industry was five years ago. Nowadays, you usually get one of those improvements in a new release video card... never all three.
Re:Tell me again: How is this bad? (Score:3, Insightful)
If they were optimizing for Quake 3 because that is where their largest audience was, and therefore they could satisfy the greatest number of their customers in that way, then yes, I would have a hard time faulting their actions from the standpoint of the free market(although I still wouldn't buy an ATI card, because that is a stupid engineering decision). However, this is not why they optimized for Quake 3.
Confining the dicussion solely to first-person shooters, it should be pointed out that Quake 3 is *not* the most popular game in this genre. Gamespy.com [gamespy.com] tracks playing statistics -- the numbers they display for today have been consistent for some months(excepting the addition of RtCW). I presume Carmack didn't want to dwell on this at great length in his .plan, and who can blame him? Quake 3's failures are not failures of technology(for which he is responsible) but of gameplay(for which he is not).
The reason that ATI optimized for Quake 3 is that it is used as a benchmarking tool. Their expectation, probably, is that gamers will see relatively high benchmarks for Quake 3 in reviews of ATI cards, and generalize from that to assume that all OpenGL games will enjoy excellent performance on ATI's card. This is likely a false assumption for anything other than Quake 3 or a Quake 3 mod. (Even games based on the Quake 3 engine will not share the performance benefits, unless you rename their executable -- and possibly not even then, depending on how modified the engine is.)
So in other words, the problem is not that ATI has cheated on their drivers to please the massive Quake 3 community. The problem is that they have cheated on their drivers to deceive reviewers, and(they hope) customers who read reviews. This is "self-interest," to be sure, but I do not find it to be "enlightened."
Re:How to not have an opinion. (Score:0, Insightful)
I haven't seen it posted yet. And it saved me a lot of time from having to visit a slashdotted link.
+5 informative.. Bitch!
- I throw rocks at retarded kids
Re:It's not cheating (Score:2, Insightful)
It's really pretty sad that we have to say stuff like this. Isn't one of the (many) mantras around here to use whatever tool works the best for you (which is often argued when switching from win2k to linux). For those of us who can't just "code our own" if a solution isn't available in one platform we use another. sheesh.
Well, there goes my karma.
The way it used to be... (Score:2, Insightful)
Evidently ATI never told anyone they were buying a "Quake Accelerator". And therein lies the problem.
Quality is lowered to boost benchmarks (Score:5, Insightful)
While many companies focus upon optimising for benchmarks, most simply optimise the driver paths for the specific cases that those benchmarks use. Any other app that uses similar settings can gain performance from that work. But ATI have made their optimisations dependant upon the name of the app, so no other apps can benefit from their work. While a Quake3 player might not mind, Q3 isn't as widely played today as it once was, and that same player might be less pleased when the card fails to perform to the same standard on any other of their games.
But what makes this particular "optimisation" underhanded is that it's not better or more tuned code, it trades off quality. Have a look here [tech-report.com] to see the mess that it makes of textures!
Now, if players wanted to see blurry textures in exchange for more performance, they'd simply lower the texture quality slider in the Quake3 game. ATI's drivers do this for them; they're forcing the mipmaps two levels down - a 16x reduction in texture detail - to get the extra speed. And this isn't optional. You can't turn it off, short of using a hex editor.
Particularly, a reviewer running the standard Quake3 High Quality benchmark will never notice the difference (as the frames run by way too fast). There's no extra sliders in the driver, no other indications or switches, and of course no notification in the driver documentation. All the reviewer sees is higher framerates, because the drivers are, quite literally, cheating. They're giving low-texture numbers on the high-texture setting.
I for one applaud ATI's renewed efforts to improve their drivers (and I still plan to buy an 8500DV alongside my existing QuadroDCC), but I feel ATI really have attempted to subvert the benchmark process, and shot themselves in the foot. The strong implication is that, even though their hardware is fine, they don't feel they can compete with nVidia's driver team so they have to resort to methods like these.
Very unwise - they've lost a lot of the support they had as the underdog to nVidia, through these tactics.
Re:Dirty hack, not optimization (Score:2, Insightful)
Though I get your point... I guess it depends on which side of the coin you are sitting on... looks vs. speed.