IE 5.5 Beats IE6 and IE7 On Acid 3

Steven Noonan sends us to a page where he is collecting and updating results for various browsers on the newly released Acid 3 test. No browser yet scores 100 on this test. (We discussed Acid 3 when it came out.) He writes, "It's not surprising that Internet Explorer is losing to every other modern browser, but how did IE 5.5 beat IE 6.0 and 7.0?" All of the IE versions score below 20 on Acid 3.

Re:IE8 Beta 1?

[MightyYar]

Why has there been no discussion on Slashdot of IE 8 beta 1?

Like, say, a front-page story [slashdot.org] from four days ago? :)

It even has your same link right in the summary...

Re:Uhhh

[The Ancients]

You're confusing intent with result.

The difference is that the teams working on Safari, Opera, Firefox, et al want to improve their product. Microsoft didn't care for a very long time. In fact, the Safari team even have a bug filed for the rendering issues Safari has with Acid3 [webkit.org]. Further, they're communicating frequently with their user base and anyone else interested with regard to their progress [webkit.org].

Re:Uhhh

[Shados]

On Slashdot, you keep hearing about "This is the standard!!!", but the W3C and other such entities do not make standards. They propose standards. Then the market decides if it wants it or not. Since a lot of bodies don't have the time or manpower to make anything better (and even if they could, it would be quite a waste of time and money), they take what the W3C spits out and implement it. As good a spec as any. And -then- it becomes the standard. Stuff like Acid Test helps meet that goal.

That being said, as time went on, the W3C really started spitting out real crap, so I'm not sure that its such a good goal to have... As opposed to standards like WS-I, which represents the real world and really do help on a day to day basis.

Re:Uhhh

[pembo13]

Not really. I often hear/read of browsers degrading compliance for the sake of rendering what would otherwise work only in IE.

Re:And older firefox versions do better too

[calebt3]

Firefox 3 beta 3 is worse. The nightly build (the one that theoretically most resembles the finished product at this point) is third in the rankings. Right behind two other nightly builds.

Re:And older firefox versions do better too

[bunratty]

Actually, according to multiple [drunkenfist.com] sources [wikipedia.org], Firefox 2.0.0.12 score 50%, lower than Firefox 3 builds. No, the quality of Firefox is not decreasing.

Re:Celik Strikes Back

[Bogtha]

wasn't IE 5.5 written with code which came over from IE5 for Mac, the first really well done major browser with nice CSS support?

No, it wasn't. Internet Explorer 5.x for Windows uses the Trident rendering engine. Internet Explorer 5.x for the Mac uses the Tasman rendering engine. They are totally different codebases.

which MS dumped when building IE6 for Win, because they could have their flagship internet browser rendering better on those other guy's OS and not their own.

Actually, in most ways, Internet Explorer 6 has better standards support than Internet Explorer 5.x for the Mac.

Re:Simple answers for simple questions

[bunratty]

The 100 subtests are nearly independent of each other. It's possible for a browser to fail a subtest simply because it failed an earlier subtest, but failing one subtest is not going to make a browser skip a major portion of the test. You can click on the A in Acid3 after the test is completed to see a report of exactly which tests failed.

Re:safari

[Idiot with a gun]

I would like to point out that Safari's HTML renderer, Webkit, is indeed open source.

Re:Maybe this is obvious but...

[bunratty]

You don't. On the other hand, if there is a flaw in Acid3, it will be found as the developers of web browsers attempt to pass it. Safari developers found a flaw in Acid2.

Re:Uhhh

[Bogtha]

Since a lot of bodies don't have the time or manpower to make anything better (and even if they could, it would be quite a waste of time and money), they take what the W3C spits out and implement it.

That's not really true. Browser vendors participate in the W3C working groups that publish these specifications. They have an active role in creating them. Take a look at the acknowledgment section of the CSS 2 specification [w3.org], for example.

This specification is the product of the W3C Working Group on Cascading Style Sheets and Formatting Properties. In addition to the editors of this specification, the members of the Working Group are: Brad Chase (Bitstream), Chris Wilson (Microsoft), Daniel Glazman (Electricité de France), Dave Raggett (W3C/HP), Ed Tecot (Microsoft), Jared Sorensen (Novell), Lauren Wood (SoftQuad), Laurie Anna Kaplan (Microsoft), Mike Wexler (Adobe), Murray Maloney (Grif), Powell Smith (IBM), Robert Stevahn (HP), Steve Byrne (JavaSoft), Steven Pemberton (CWI), Thom Phillabaum (Netscape), Douglas Rand (Silicon Graphics), Robert Pernett (Lotus), Dwayne Dicks (SoftQuad), and Sho Kuwamoto (Macromedia). We thank them for their continued efforts.

A number of invited experts to the Working Group have contributed: George Kersher, Glenn Rippel (Bitstream), Jeff Veen (HotWired), Markku T. Hakkinen (The Productivity Works), Martin Dürst (W3C, formerly Universität Zürich), Roy Platon (RAL), Todd Fahrner (Verso), Tim Boland (NIST), Eric Meyer (Case Western Reserve University), and Vincent Quint (W3C).

The section on Web Fonts was strongly shaped by Brad Chase (Bitstream) David Meltzer (Microsoft Typography) and Steve Zilles (Adobe). The following people have also contributed in various ways to the section pertaining to fonts: Alex Beamon (Apple), Ashok Saxena (Adobe), Ben Bauermeister (HP), Dave Raggett (W3C/HP), David Opstad (Apple), David Goldsmith (Apple), Ed Tecot (Microsoft), Erik van Blokland (LettError), François Yergeau (Alis), Gavin Nicol (Inso), Herbert van Zijl (Elsevier), Liam Quin, Misha Wolf (Reuters), Paul Haeberli (SGI), and the late Phil Karlton (Netscape).

The section on Paged Media was in large parts authored by Robert Stevahn (HP) and Stephen Waters (Microsoft).

Robert Stevahn (HP), Scott Furman (Netscape), and Scott Isaacs (Microsoft) were key contributors to CSS Positioning.

And of course, one of the four editors of the specification is Håkon Wium Lie, CTO of Opera.

So you see, far from the poor old browser vendors not having the resources to make anything better and passively reacting to anything the W3C says, you can see that the browser vendors are substantially the people who made the specifications.

Re:And older firefox versions do better too

[klui]

Windows 2003: 2.0.0.12 = 51%; 3.0 beta 3 (portable version) = 58% here.

Re:IE8 Beta 1?

[Bogtha]

It is a problem, but it's not the hard-coding people seem to think it is. The problem is not that Internet Explorer 8 is checking for www.webstandards.org, the problem is that the mirrors that are failing are changing the test in a way that is important to Internet Explorer. Part of the test refers to a page that intentionally doesn't exist in order to check a fallback option. The trouble is that this page is referred to with an absolute URL, which means that when you simply copy the test to another host, it becomes a cross-domain issue.

Ian's pointing out that it's still a failure so it should be subject to the same fallback, which is correct, but the important point is that it's failing to load in a different way to how it would on the www.webstandards.org host because the mirrors didn't take the cross-domain issue into account. I expect the final version of Internet Explorer 8 to correct this problem, but it's not at all a case of Microsoft attempting to "cheat", just an unfortunate coincidence.

Re:And older firefox versions do better too

[neunon]

I don't know how, but I messed up when making the table. I reconfirmed the results with the ones I had written down and realized the Firefox 2.0.0.12 Mac OS X entry was incorrect. I've corrected the error. The actual value is 52%. So 3.0b3 is actually doing better than the current release. Sorry about that error. - Steven

Re:IE8 Beta 1?

[99BottlesOfBeerInMyF]

Why is this modded down to -1? I'm running IE8b1 right now and yes, it runs Acid2 completely.

It was probably modded down because we've already had this discussion in three different articles over the last week. IE8 beta passes the Acid 2 test only when run on webstandards.org, but fails if you run it on almost any mirror. The discussion further continued with speculation that MS had hardcoded a workaround specifically for the test and was "cheating". This turned out to be untrue and the reason was that webstandards.org references a page that exists incorrectly but the mirrors reference a page that doesn't exist. Both cases should be handled, but IE8 beta fails on the latter.

Probably people were modding the post down because it was factually incorrect. A better way to deal with the problem is probably to post a factual response, but several people have done so and those posts have not been modded high enough so that the facts are more easily read than the misleading evidence presented in the post you are asking about. Either that or a dozen people with mod points just groaned and thought, "do we have to go through all this again?"

not like it matters

[MSDos-486]

One of the primary components of the Acid test are to see if a browser will properly handle out of spec code.In this case "proper handling" means ignore it. IE is counter intuitive in this sense because it has facilities to "guess" what should happen.

<rant>

Today I was borrowing someones computer and i went on a few websites with IE. When they came back they were disappointed because all of the sites i went to messed up there "recently visited" listing in IE. They were frustrated that that there would have to manually type the URLs of the pages to go to. Then I introduced them to the wonderful world of Favorites/Bookmarks, something I learned about back in 97. Now when I was in High School i tested out of all the intro to computer courses in order to take programming, so can anyone tell me what they teach in these classes. I mean seriously. Sometimes it surprises me how little people know about computers. Maybe its because I grew up in a city whose major employers included HP, Oracle, and BAE Systems( who bought Sanders, the inventors of the Magnavox Odyssey) . So maybe I just used to most people having a general understanding. It seems when I went to college the average computer skill per person I associate with dropped.

</rant>

Re:safari

[99BottlesOfBeerInMyF]

Safari development builds are doing well on Acid3, and Safari passed Acid2 quickly, because Safari developers fixed the problems that the Acid tests demonstrate. If you look at the stable release builds of Safari, they do far worse than the stable release builds of Opera and Firefox.

I think your idea here is a bit off. The stable version of Safari does perform more poorly than the stable versions of Firefox and Opera, but I think this is more likely attributable to Apple's more leisurely release schedule. The article referenced here was obviously put together by someone more focused on Windows and OS X. They only tried to test one browser on one version of Linux, compared to the dozen or so for the other OS's. It is, then, understandable that you would get that impression from the data presented. What a lot of people forget is that Safari uses the Webkit rendering engine which is also used in a variety of other browsers whose developers also contribute to it. The stable version of Konquerer 3.5.8 uses the same rendering engine and scores a 52 on the Acid 3 test, better than either Firefox or Opera. So Webkit is being updated and did, in fact, do better than Gecko or Presto for stable release versions when Acid 3 was published. (Note Konquerer 4.0.2 scores a 62, but I don't know if that is considered a "stable" branch.)

Mind you, this is not to imply that the Acid 3 test can really judge the respective compliance of the engines in general. This is not the case. The test was designed with bias in mind, bias against Webkit and Gecko. The criteria for inclusion in the test was that one or the other had to fail it and we don't know how many of the Acid 3 authors were focusing on one engine or another. If anything Opera and IE should be doing better than Firefox or Konquerer or Safari, since there are probably a number of tests those browsers fail, but which were excluded from Acid 3 simply because both the Gecko and Webkit engines passed it.

Safari is doing well on Acid tests because the developers put a lot of effort into making Safari do well on Acid tests, not because Safari is "ahead of the game" on standards.

I know for a fact that developers of both Gecko and Webkit are specifically using these tests as a way to find problems to fix, which is great since that is why the tests were written; not to try to measure "compliance."

There's far too much bickering about which browser is best and which browser is behind the curve. It seems that Safari, Opera, and Firefox are all very good browsers each with their own strengths in standards compliance and user interface, with IE constantly playing catch-up.

This is true enough, well except about IE maybe. In my own personal experience every browser other than IE works just fine for rendering everything I create to the standards. There might be the occasional bug or edge case, but I never run across them. IE, on the other hand, I have to create work arounds every single time. I'm not sure it is "playing catch up" so much as deliberately failing to implement huge portions of many standards as a way to prevent cross platform compatibility and keep Web applications that undermine their platform lock-in from being a real threat.

mirrors not introduce change

[HeroreV]

The mirrors did not introduce anything new. From the linked IEBlog:

It's worth mentioning that although most sites allow navigation like http://webstandards.org/ [webstandards.org] (note: the no www) this is also considered a cross domain access as www.webstandards.org != webstandards.org. This will also cause us to fail the ACID2 test and render the picture that you see above. So please type www.webstandards.org!

The test should work from http://webstandards.org/files/acid2/test.html [webstandards.org] and http://www.webstandards.org/files/acid2/test.html [webstandards.org], but IE8 fails the first one. The mirrors might exacerbate the problem, but they certainly did not introduce anything that wasn't in the original test.

However, it is true that this issue has nothing to do with hardcoding a certain URL and trying to cheat.

Some insight regarding the Acid 3 Test

[trixy_1086]

I found this information regarding the Acid 3 test on a Webkit developer's site (http://webkit.org/blog/158/the-acid-3-test/) As much as I hate to debunk any article bashing IE, here is the information from the article:

If you run Acid 3 on the shipping versions of current browsers (Firefox 2, Safari 3, Opera 9, IE7), you'll see that they all score quite low. For example Safari 3 scores a 39/100. This percentage score is a bit misleading however. The situation with all four browser engines really isn't that bad. You can think of the Acid 3 test as consisting of 100 individual test suites. In order for a browser engine to claim one of these precious 100 points, it has to pass a whole battery of tests around a specific standard. In other words it's like the browser is being asked to take 100 separate exams and score an A+ on each test in order to get any credit at all. The reality is that all of the browsers are doing much better than their scores would have you believe, since the engines are often passing a majority of the subtests and experiencing minor failures that cost them the point for that section.

Re:mirrors not introduce change

[Bogtha]

It's not an intentional part of the test, it's accidental, a side-effect of webstandards.org failing to canonicalise their hostnames. If you read the press release, it only refers to the www version and nowhere is the no-www version mentioned, nor is this issue brought up in the technical guide. If they were trying to include this in the test, they'd have picked a hostname foreign to both the www and no-www versions so that it failed reliably.