Chinese Security Vendor Qihoo 360 Caught Cheating In Anti-virus Tests 63
Bismillah writes: China's allegedly largest security vendor Qihoo 360 has fessed up to supplying custom versions of its AV for testing according to an investigation by Virus Bulletin, AV-Comparatives and AV-Test. "On requesting an explanation from Qihoo 360 for their actions (PDF), the firm confirmed that some settings had been adjusted for testing, including enabling detection of types of files such as keygens and cracked software, and directing cloud lookups to servers located closer to the test labs. After several requests for specific information on the use of thirdparty engines, it was eventually confirmed that the engine configuration submitted for testing differed from that available by default to users."
Re: (Score:2)
Not shocked at all
I am not shocked, but I am confused. Why would they give bad software to their customers, but give good software to the testers? The marginal cost of software is zero. So, if they have good software, why don't they give it to their customers? Can someone please explain how any of this makes sense?
Re:Is this shocking? (Score:4, Informative)
I am not shocked, but I am confused. Why would they give bad software to their customers, but give good software to the testers? The marginal cost of software is zero. So, if they have good software, why don't they give it to their customers? Can someone please explain how any of this makes sense?
It's really easy to "detect" everything so you get a high detection rate. It's really hard to do so without a ton of false positives.
Very few of the tests out there check for false positives, so it is easy to game the results. You could never ship the product to customers that way because you'd drown in support calls from customers complaining about programs not work, broken websites, etc.
Re: (Score:2)
Very few of the tests out there check for false positives, so it is easy to game the results.
I see. In that case, shouldn't the story be "AV Tests are Stupid" rather than "Chinese Company Sort of Cheats on a Test Designed to Make Cheating Easy"?
Re: (Score:2)
Very few of the tests out there check for false positives, so it is easy to game the results.
I see. In that case, shouldn't the story be "AV Tests are Stupid" rather than "Chinese Company Sort of Cheats on a Test Designed to Make Cheating Easy"?
No, the testing organizations here are competent. It is the "let's have the intern do an antivirus review" articles in publications having no particular reputation in security matters that should be treated with suspicion.
Re: (Score:2)
Re: (Score:2)
if they ask the vendor for a version to test (and money? then the test is suspect.
Re: (Score:2)
All the major testing houses check for false positives alongside detections, but perhaps they decided more false positives would still look better on benchmarks than a lower detection rate.
Re: (Score:2)
All the major testing houses check for false positives alongside detections, but perhaps they decided more false positives would still look better on benchmarks than a lower detection rate.
It's not that they don't claim to test for false positives... It's that their FP testing tends to be... rudimentary.
To be fair, I haven't worked with these specific test houses. I have, however, worked closely with some very well-known and trusted test labs. Perception and reality don't line up very well
Re: Is this shocking? (Score:1)
Probably because the customers don't want keygens to flag unless there's an actual Trojan?
Re: (Score:2)
Probably because the customers don't want keygens to flag unless there's an actual Trojan?
For me this is true of all security software. Why do they flag keygens if there isn't an actual Trojan? It's supposed to be security software, not anti-piracy software.
Re: Is this shocking? (Score:1)
And it is probably why pirated software are the main attack vector. Can't be scanned
Re: (Score:2)
I am not shocked, but I am confused. Why would they give bad software to their customers, but give good software to the testers? The marginal cost of software is zero.
The good software is not theirs, it is Bitdefender's, and it does not have a zero marginal cost unless they steal it. That would not be unknown, of course, but this company may be too large, and have big enough aspirations, for that not to be an option.
I also tend to agree with those who suspect they are selling to customers who don't like to be reminded that using keygens is risky.
Finally (Score:2, Interesting)
Qihoo has been a joke in China for a long time. They finally made their way to the international platform. Good.
A Chinese.
Re:Finally (Score:5, Interesting)
Chinese here too.
360 is no "joke" in all seriousness. They are bullies, really badass bullies.
They "kidnapped" hundreds of thousands of terminals (PC/Phone/browser) by disguising themselves as a "security guard", telling users what is bad and what is good, and then blackmail developers and websites to bribe them to get into their "good" list.
My company has a website that only shows text and picture news and contact info and stuff. One day 360 decided to reported our website as "security threat" and show warning on ALL 360 browsers (which is A LOT).
We contacted them, they told us to put "a security script" into our server. Once they confirmed the script is in place, they re-score our website to 100-OK, without asking us to modify/patch anything.
What that script does (thankfully it's PHP so it's naturally "open source") is scanning our whole www directory, upload whatever info they want, and even modify our code whenever they like.
Oh, and they also labeled my company's phone number as scam in their "smartphone guard", even though we've been using it for years.
Re: (Score:3)
Re: (Score:1)
Yes, that's exactly what I did.
Re: (Score:1)
Re: (Score:3)
How about the fact that if you think the NSA does some crazy malware stuff with Flame and Stuxnet, at least they tend to confine it to foreign political targets. China has probably the largest censorship and MITM infrastructure in the world, and actively uses it to pull average citizens into a government run botnet to DDOS western sites.
Not to mention that any sufficiently large business needs to have the explicit blessing of the powers that be in China.
All of that combined means you would have to be crazy
Re: (Score:2)
Re: (Score:2)
Isn't "Chinese Security Vendor" an oxymoron? (Score:3)
Any sufficiently sophisticated Chinese security security product to be of any use will either be compromised by the Chinese government "in the interest of domestic social harmony" or for national security/military/espionage.
Re: Isn't "Chinese Security Vendor" an oxymoron? (Score:1)
The difference being here in America we can take our government to task for such infractions, and we do, and even our "tech giants" fight back against government meddling. Go try that in China. Drop me a line and I'll be happy to help remove the boot from your ass after you find out how far it gets ya.
Re: (Score:2)
I suppose you're not familiar with the genesis of the phrase "illegal flower ceremony" or the history of internet censorship in China.
Re: (Score:2)
Ah yes we "take them to task" with stern words and a disapproving glance.
Re: (Score:2)
I dont think you really have any idea in how the MSS is different than the NSA.
Lets start with the fact that the MSS gives no craps, they straight up block sites like Google who dont play the censorship game, and they inject malicious javascript into millions of citizens sessions to enact a government-run DDOS of foreign sites.
The things the NSA does that are violations of our principles are extra-ordinary. The things that the MSS does on that scale are ordinary, expected, and well documented.
Re: (Score:2)
There's no rule of law in the USA either, if you have enough money and your skin is the right color.
Re: (Score:2)
You are just projecting US thinking onto the Chinese government. They have little interest in turning AV software into a trojan, because they don't want or need to spy on their citizens that way. They have more direct means, and prefer censorship over mass spying because it's cheaper and easier.
Unlike the US, China does have an interest in keeping its citizens safe so doesn't break their security software.
Broken test? (Score:4, Insightful)
If the test is checking for non-virus files like keygens it sounds like the test is broken. AV software should detect things that are harmful to your computer, not things that software vendors don't like but are otherwise harmless.
I'm not surprised they ship with keygen detection off in China.
Re: (Score:3)
Sure, sometimes keygens are trojans as well, but those are covered under the heading "virus". Most anti-virus software also detects perfectly harmless keygens these days, supposedly to "protect" the user from "accidentally" generating a key and pirating software.
I use some keygens for old software that can't be bought any more. It would be lost to the world without those keygens. I even had keys for some of it, e.g. a Windows 98 serial that was stuck (with a non-removable sticker) to the side of an ancient
Re: (Score:2)
Actually, most keygens people run into are infested with malware - Trojans and viruses and all that. Usually they're wrapped with a "dropper" application - run the keygen, and the dropper downloads the malware then launches the keygen.
The reason for
Re: (Score:1)
What to know the real best way not to get malware? Stop buying software.
Seriously, pirated software has been proven to have a lower infection rate than commercial software.
Not really an issue (Score:4, Informative)
Re: (Score:2)
Re: (Score:3)
Right. There's no issue with them putting their best foot forward if this is something you can get with the basic product.
However, if you have to enable these features AND you have to pay for them, that's a different product. The danger is that the reviewers rate their "basic" product as a top-rated AV product. Then people flock to get this basic product over the basic offerings of other AV companies who did not rate as well, but might well have a better "basic" product.
It's basically bait and switch, an
Re: (Score:2)
Re: (Score:1)
Which raises the question: Why do they have two products that are free? One that they market, and one that they test, and pawn off as the marketed item?
The problem here is that they were submitting one product for testing, and using the certification gained by that testing to represent another product.
My guess is that this was done so that the product they distribute in China is 100% Chinese, but they get the one that's essentially BitDefender certified to raise acceptance.
Re: (Score:2)
Re: (Score:1)
Which testing organisation are you regularly paying to write unbiased reviews?
Thought so.
Corparate security hijinx (Score:1)
I keep imagining some scenario like in Blade Runner and the crazy Chinese eye-doctor or something.
In other news.. (Score:2)
The major American AV vendors announced a joint task force today to respond to these results.
When asked how they would ensure that corporate members of the task force would be held accountable for this sort of cheating, their spokesperson responded with the following:
"Accountable for cheating? No, no, no, the point of the task force is to keep from getting caught like this."