Acid3 Test Released

An anonymous reader writes ""The Web Standards Project has announced the release of Acid3, the latest test designed to expose flaws in the implementation of mature Web standards in browsers. 'By making sure their software adheres to the test, the creators of these products can be more confident that their software will display and function with Web pages correctly both now and with Web pages of the future. The Acid3 Test is designed to test specifications for Web 2.0, and exposes potential flaws in implementations of the public ECMAScript 262 and W3C Document Object Model 2 standards.' Screenshots at the Drunken Fist site show the success of Safari 3 (which originally scored 31, but is now Scoring 87/100) IE6, and IE7 (massive fail, of course)'." There are additional discussions of the new test happening around the web.

Firefox 2.0.0.12

[poetmatt]

I get a 51/100 with firefox 2....wonder how 3 will do.

Re:Firefox

[brunascle]

the test is here [acidtests.org].

i'm getting a 50/100 in Firefox.

Link to the actual test

[I kan Spl]

Why does slashdot keep linking to dead blogs?

The actual test is http://acid3.acidtests.org/ [acidtests.org] here.

Re:Firefox 2.0.0.12

[The MAZZTer]

3b3 gets a 61. Opera 9.5 is the best I tested at 65. Safarai 3.0.4 for Windows got a 39. IE7 got a 12 and also managed to mangle the page the most.

Re:Bad day for IE8

[Jugalator]

At least that beta doesn't crash. When I ran on a recent Opera 9.50 beta build, it counted, stalled, stalled, crashed. ;-)

Latest Safari nightly scores...

[The Ancients]

90/100 [mothership.co.nz].

Getting pretty close.

Re:Opera

[wile_e_wonka]

I just tried it on Opera 9.5 Beta, build 9755. I got a 60/100. Then I tried again and got a 61/100. Then a 60/100 on a third try.

All of the rectangles are grey (two different shades), the test name is red and does not have a shadow, and there is an x in the upper right hand corner.

Re:Firefox

[Anonymous Coward]

Confirm the 50/100 on firefox 2 and I get 46/100 on Opera 9.25

Re:Firefox

[Jugalator]

Ooh, my Opera 9.50 weekly actually didn't crash this time. Maybe the test was changed, or something in Opera did.

Anyway, Opera 9.50.9807 receives a 65.

Re:Firefox

[Cap'n.Brownbeard]

Same score for me using FF 2.0.0.12 on WinXP.

Konqueror

[kevmatic]

I haven't seen anybody answer konqueror yet!

I tried it in Konqi 3.5.8 with Gentoo. It asked me what I wanted to do with "empty.txt" then segfaulted. Anyone fairing better?

Re:Konqueror

[Anonymous Coward]

That crash is fixed in 3.5.9 (which does 41)... 4.0.2 does 61, 4.1-pre 63. Early 4.0.x versions do less.

Re:Firefox 3 beta 3

[corychristison]

... and 64/100 for Firefox 3.0b5pre ("Minefield")

Results in major browsers

[Lobais]

See http://browsershots.org/http://acid3.acidtests.org/ [browsershots.org] for the test in 75 different browsers.
Wikipedia: http://en.wikipedia.org/wiki/Acid3 [wikipedia.org] also lists the results for the developversions of browsers:
Webkit: 87
Firefox: 67

Re:Firefox

[bunratty]

The test consists largely of 100 JavaScript tests designed to throw an assertion on failure and return a certain value on pass. The score is how many of the tests out of 100 pass. You can see which tests failed by clicking or shift-clicking the A in Acid3 after the test completes. In the sense that each test can relatively independently pass or fail (although some tests depend on previous tests), yes, it is a quantitative test.

The other part of the test is rendering the Acid3 text with shadow and the colored rectangles. By seeing how the Acid3 test fails in many other browsers, you can see that it can also render X, Fail, and a picture of a cat on failure of some rendering tests, typically in red so they stand out.

Re:Of Course IE will fail, ACID test is biased...

[gsnedders]

Actually, trying to make IE fail it wasn't an aim: the aim was to include tests that one of Firefox, Safari, and Opera fails. If IE happens to fail them too, so be it.

Not to mention...

[rsborg]

The latest webkit (Safari) nightly is just amazingly fast.

Faster than FF3 beta 4, much much faster than FF2 or IE7.

Re:IE8 Cheats ACID2!!

[Anonymous Coward]

The eyes for the test at acird2.acirdtests.org is cross-domain, which is causing it to fail because of boundary trusts.

Re:IE8 Cheats ACID2!!

[prockcore]

No. The test got screwed up. Everything fails that second one. Including FF3 and Safari 3.

Re:Firefox 3 beta 3

[BZ]

Why? The test expressly picked things that one of Opera, Safari, and Firefox would fail, preferably more than one, and tried to balance the number of tests each would fail.

Put another way it looked really hard for things to test that would give browsers low scores.

There's nothing to say that the things it tests are necessarily useful. Some are, some are not.

Re:Perhaps....

[ivan256]

The ideal here actually is that if a reasonable number of mainstream browsers scored 100 on the test, web developers could use all of the features the test exercises and have a reasonable expectation that their page will display correctly for end users.

The test is about making life better for web developers, and about making the web more interoperable, instead of having sites which jump through browser predicated hoops, or restrict users to "IE7.0 or newer on 32-bit Windows" or the like. Thus having your favorite browser, and your least favorite browser score well is in the best interests of all web users.

Re:Too late for IE8?

[edwdig]

If they actually implemented the standards well, they wouldn't have to worry about specific tests, they would just do well on them by default.

Have you ever tried reading the HTML/CSS specs? They're huge and often vaguely worded. There were often sections that just weren't intuitive, and the only real approach to implementing them was to just figure out what other browser did and copy it. The specs were created by people who have no intention of implementing them themselves, and it really shows.

Re:Geek version of a measuring contest?

[Anonymous Coward]

CSS3 isn't an "international standard", it's a draft specification

Very true, but not quite the killer point you probably intended it to be; Acid3 does test some of the more widely implemented CSS3 features, but it's primarily concerned with JavaScript and the DOM, not with CSS at all.

Re:IE8 Cheats ACID2!!

[Anonymous Coward]

This was already explained, and the behavior should have been updated as per a previous slashdot story.

IE8 fails when it runs in quirks mode, and passes when in standards mode. Before it would run in quirks by default, and only change behavior when it visited certain key sites, or sites had a tag.

That url would be one of those "key sites"

However if the previous slashdot story is true, IE8 should eventually operate in standards mode by default, so it will pass both.

Re:Firefox

[dal20402]

It's misleading for the summary to say "Safari" gets 87/100 when the version of Safari that does that is not yet released.

Run current WebKit nightlies to get the high score now. The changes will be in the upcoming Safari 3.1 release.

Re:IE8 Cheats ACID2!!

[ejtttje]

Both work for me with Safari 3...

Re:Of Course IE will fail, ACID test is biased...

[djcapelis]

I believe the standard for the last 16 tests were webkit and firefox trunk must fail.

So IE or Opera failing was actually regarded as insufficient.

Re:Firefox

[caerwyn]

Good point. Running now gets me a 90 with the webkit nightly, and examining the earlier failure shows at least one test (69) which explicitly states may be a networking issue- something that seems highly likely given how overloaded the acid3 server was earlier in the day.

Re:IE8 Cheats ACID2!!

[Bodero]

Why Isn't IE8 Passing ACID2? [msdn.com]

Basically, it fails because of XSS on the other sites.

Re:W3C validator

[Millennium]

If these acid tests are based on standards. Why is the only acid test that passes the W3C validator, the Acid 1 test?

Because more recent Web standards include sections on how certain kinds of errors are supposed to be handled. These need to be tested just like everything else, but up until Acid2 many browsers weren't very good about that.

Remember, the point of Acid tests is to be a thorn in browser developers' sides: find areas of the standard that no one currently does well and test for them. Browsers shouldn't pass Acid tests when the tests first come out: that would be missing the point of the tests in the first place.

Re:Bad day for IE8

[99BottlesOfBeerInMyF]

At least that beta doesn't crash. When I ran on a recent Opera 9.50 beta build, it counted, stalled, stalled, crashed. ;-)

What OS? Opera 9.5 beta works fine for me on OS X and gets 59/100. The only things that crashed for me were Shiira on OS X and Konquerer 3.5.2 on Kubuntu.

Note, the best score I'm getting is from Safari 3.0.4 with a nightly Webkit on OS X, with a score of 86/100.

Re:Firefox 2.0.0.12

[Anonymous Coward]

Err... wha?

Acid 3 is mostly Hixie, who's currently at... Google, I think. Previously Opera/Netscape.

Aside: yes, he did ask for browser developer feedback (in particular, to look for things that will fail in under-development versions of Mozilla and Safari; presumably, Opera too.) Apparently new things that only failed in IE wasn't interesting. Reference: his blog [hixie.ch].