NSS Labs Browser Report Says IE Is the Best, Google Disagrees 205
adeelarshad82 writes "Independent testing company NSS Labs recently published a report on the ability of popular browsers to block socially engineered malware attack URLs. The test, funded by Microsoft, reported a 99 percent detection rate by Internet Explorer 9 beta, 90 percent by Internet Explorer 8, and 3 percent by Google Chrome. However, Google doesn't entirely approve of this report's focus and conclusions. According to Google not only didn't the report use Chrome 6 for the tests, the current version is Chrome 8; it also focused just on socially engineered malware, while excluding vulnerabilities in plug-ins or browsers themselves. Google defended its browser by claiming that it was built with security in mind and emphasized protection of users from drive-by downloads and plug-in vulnerabilities."
It's Clear to Me Why They Waited (Score:5, Informative)
It's not clear why Microsoft and NSS Labs waited until December to release the results.
Maybe it's like the last time this happened [microsoft-watch.com]?
Furthermore, Moy said, the study started as a private test for Microsoft's engineering team, which was seeking to make internal improvements. "They decided to release it based on the positive results. Many of the test reports we write do not get released by vendors, but they do get used to improve products. So what does 'sponsored' mean in this case?"
So you (internally) strike a deal to test your browser (but also your competitors') with an "independent company" that you pay to perform this service. You get to define the "success parameters" of the test. Then you get the results back and you fix everything. After that time spent fixing has passed, you release the report and add that you have fixed all the problems with your product. Unsurprisingly, you look really really good when this news hits. Since your competitor is not also paying NSS Labs, NSS has no reason to update the report to meet the latest and greatest version of browsers. Meanwhile you can decide if your competitor's browser performed inadequately enough or not for the report -- maybe you even select the success parameters afterward? Heck, you already waited to see if you could release the report.
Independent? HA!
Bad summary? (Score:3, Informative)
According to Google not only didn't the report use Chrome 6 for the tests where as the current version is Chrome 8...
Should it be:
According to Google not only did the report use Chrome 6 for the tests, whereas the current version is Chrome 8...
Re:Check the funding (Score:5, Informative)
This: "The test, funded by Microsoft"
The real warning flag is that it doesn't say that on NSS Lab's site nor does it say it anywhere in the report. So if I was being paid to do this, I would have that in big bold letters as a disclaimer on the front page of the report if I wanted to maintain credibility. So either the Google response article is wrong (which the same IE8 report from last year [thetechherald.com] was funded) or you're just being flat out disingenuous when you say "independent." We just happen to receive funding from one of the participants and they decide when and if the report is released.
One more thing, if you dig into this report, the parts where they reference Microsoft read like an advertisement:
It became obvious from this test and comparisons to the earlier test that Microsoft continues to improve their IE malware protection in Internet Explorer 8 (through its SmartScreen® Filter technology) and in Internet Explorer 9 (with the addition of SmartScreen application reputation technology). With a unique URL blocking score of 94% and over-time protection rating of 99%, Internet Explorer 9 was by far the best at protecting against socially-engineered malware. The 89% zero-hour block rate suggests a far superior malware identification, collection, and classification method.
"What kind of registered application reputation technology did you say they used? Simply revolutionary progress!" Compare that section to that same section on Chrome:
With a protection rating of just 3%, Chrome 6 dropped more than 14% from our last test. And, Chrome’s unique URL score of 4% was also a major decline. Chrome’s overall poor protection makes it difficult to compare it to other Safe Browsing API-related products.
"Boo, Chrome sucks!" Hahaha oh my this is too funny. Google shouldn't have to explain themselves. Just take what you can to improve from this report, become aware of your opponent's tactics and move forward.
Re:Attack urls? (Score:4, Informative)
Re:Socially engineered attacks ARE a huge problem (Score:3, Informative)
Tests like this are done for marketing purposes. The professionalism of the tester will make sure the test is rigged to give Microsoft the result they want. Get the facts.
beta Apples to outdated Oranges (Score:5, Informative)
I think you missed the other important part: "Also, the version of Chrome that NSS says all this about is two major versions behind the current stable release, while the version of IE they say is better is the current beta release."
A more relevant comparison would be IE 8 to Chrome 8 (current generally release version of both version), or IE 9 to Chrome 9 (current publicly available pre-release version of each browser.)
Perhaps someone should do a similar comparison, but using Chrome 9 and IE 6, instead...
Re:Socially engineered attacks ARE a huge problem (Score:5, Informative)
What the Faceless Google rep said was that this test cannot be peer reviewed because they did not release all the data (specifically the URLs visited). Now releasing a report that does not allow for independent review does not make for good science.
The tests may be valid. But until there is enough information to confirm this, I can only be skeptical of the faceless Microsoft rep.