Google, Facebook and Twitter To Block "Hash Lists" of Child Abuse 177
An anonymous reader writes: Facebook, Google, and Twitter are teaming up with the UK's Internet Watch Foundation (IWF) to share hash lists of blocked indecent images. The move is intended to ensure that a picture pulled from one site can't show up again elsewhere. The BBC reports: "Online security specialists welcomed the move as a positive step, but said it would not block content on the 'darknet' — a network with restricted access — where abusers often posted images."
keep honest people safe (Score:2)
Re: (Score:3)
Well ... this is great if no two things can have the same hash.
But as soon as it starts blocking my picture of my dinner as kiddie porn, having Facebook and Twitter block it becomes fairly meaningless.
I mean, are people using Google, Twitter, and Facebook for this stuff?
Re:keep honest people safe (Score:5, Informative)
Re: (Score:3)
False positives will be hell. It's bad enough that the content might be blocked or de-indexed, but imagine if perfectly innocent photos were tagged as child pornography. It's also worth noting that the IWF is not regulated or overseen in any meaningful way, and once broke Wikipedia with it's overzealous blocking of an album cover that had been on sale in the UK for decades.
I'm just astounded that this is even worth doing. How dumb do you have to be to post child pornography to Twitter or Facebook?
Re: (Score:2)
How dumb do you have to be to post child pornography to Twitter or Facebook?
Think of the 17 yo girl who sends her boob pic to her boyfriend. This protects her against her own bad judgment, or against the bad judgment of her boyfriend, or the many, many "friends" who will see and re-share that pic, which to the casual observer could just as easily be an 18yo.
Re: (Score:2)
Re: (Score:2)
I guess you've never heard of Google Reverse Image Search then? It searches based on the contents of the image, not a direct binary comparison. It is quite awesome, in fact.
https://www.google.com/search?... [google.com]
Re: (Score:2)
Re: (Score:2)
I've always wondered how society will react over time as perspectives shift, and things that were previously acceptable become taboo.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Given that how would they have the original image to match against?
Re: (Score:2)
Re:keep honest people safe (Score:4, Informative)
Possession of child pornography is illegal in itself. Given that how would they have the original image to match against?
Almost every child pornography possession statute that I've seen has an exception for law enforcement activities. For example, a jury examining photos in a jury room wouldn't be guilty of possession if those photos are evidence presented at trial.
Re:keep honest people safe (Score:5, Insightful)
No. I use a similar algorithm to deduplicate my obscenely large stash of furry pornography*. It works, but there's a problem.
Let's say that the chance of two unrelated images matching is, say, one in million. Great. That sounds amazing - and it is, that's ridiculously optimistic for phash alone, but we can assume they have something better involving composite hashes.
Now feed into that a sizable database of child abuse imagery - say, ten thousand images. And a copy of the facebook photo library for one day, which is 350 million photos. Yes, that's facebooks claim, do not underestimate the number of compulsive photographers. That's 3,500,000,000,000 comparisons, and at your optimistic one-in-a-million error rate, 3,500,000 false positives to investigate every day.
It can be done, but it's going to need a bit more than just perceptual hash comparisons.
*Thus posting as AC.
Re:keep honest people safe (Score:5, Funny)
Got to caught up in checking the math I forgot to tick the box. Bah. Well, no-one cares anyway.
Re: (Score:3)
Bravo on your stoicism!
Re: (Score:2)
Re: (Score:2)
What exactly would lead you believe that the IWF is competent enough to do that? [wikipedia.org]
Re: (Score:2)
Re: (Score:2)
An idea that relies on unrealistically competent (or benevolent) implementers is itself unrealistic.
Re: (Score:3)
Let's say that the chance of two unrelated images matching is, say, one in million. Great. That sounds amazing - and it is, that's ridiculously optimistic for phash alone, but we can assume they have something better involving composite hashes.
For the reasons you outline, a hash with a one-in-a-million collision rate (one in 2^20) is worthless for this purpose and for many purposes. Maybe this is an accurate rate for phash. That's because it's a fuzzy hashing algorithm. Typically, all of these law enforcement applications use MD5 or SHA-1, which have collision rates around 2^128 to 2^160 (not including manufactured hash collisions).
Now feed into that a sizable database of child abuse imagery - say, ten thousand images. And a copy of the facebook photo library for one day, which is 350 million photos. Yes, that's facebooks claim, do not underestimate the number of compulsive photographers. That's 3,500,000,000,000 comparisons, and at your optimistic one-in-a-million error rate, 3,500,000 false positives to investigate every day.
It can be done, but it's going to need a bit more than just perceptual hash comparisons.
The numbers are, of course, much different when your hash has collision rate that's many, many orders of magnitude l
Re: (Score:2)
PhotoDNA is a perceptual hash. It's functionality is similar to phash - it might be a bit better, but I wouldn't expect it to be much better. Neither are going to come close to one-in-a-million: I picked a number that was intentionally optimistic.
Cryptographic hashes would eliminate the false positive problem, but are trivial to alter. Beyond trivial: It happens frequently without even intending to do so.
You don't need to look at collisions within the facebook sex - only those involving an image from each s
Re: (Score:2)
True, but a cryptographic hash gets the collision rate down to the point it can be considered negligible.
Re: (Score:2)
No they don't. The likelihood of collisions depends only on the effective number of bits in the hash. If you have a 32 bit hash, and more than 2^32 images, you are guaranteed to have collisions.
Yes, they do, because the shortest available cryptographic hash has a 128-bit output.
Re: (Score:2)
"The likelihood of collisions depends only on the effective number of bits in the hash."
True for cryptographic hashes. Perceptual hashes are another thing entirely. You don't compare them for identical values, you look for similar values, usually defined by low hamming distance.
Re: (Score:2)
I didn't say there weren't.
Re: (Score:2)
*Thus posting as AC.
Oops?
Re:keep honest people safe (Score:5, Insightful)
by SuricouRaven (1897204)
Get some help.
Re: (Score:2)
I got too caught up counting zeros to make sure the math was correct, and forgot to tick the box.
Re: (Score:2)
Congratulations, you have invented the perceptual hash. That is exactly how they work.
Re: (Score:2)
They almost exclusively use binary hashes (MD5, SHA-1).
Re: (Score:2)
Re: (Score:2)
IWF, the organization named in the summary who is providing "hash lists" to Google, Facebook, and Twitter.
Re: (Score:2)
But as soon as it starts blocking my picture of my dinner as kiddie porn, having Facebook and Twitter block it becomes fairly meaningless.
The question you need to ask yourself is "Why am I uploading pictures of my dinner?" Seriously. People do that all the time, and I have to wonder why. Does anyone really enjoy seeing other people's dinner? What's next? People posting pictures of their poop?
Re: (Score:2)
Well ... this is great if no two things can have the same hash.
With even the worse of the acceptable cryptographic hashes, it is essentially true that no two things can have the same hash*.
* Barring manufactured collisions, which are best avoided but may or may not be a problem depending on your application.
Re: (Score:3)
I see stuff I don't want to see all the time. That doesn't mean that other people don't have a right to post it. I can't recall a time where I ever came across an illegal image on Facebook or Twitter. Against TOS/AUP, sure. Trashy or tasteless? Definitely. But not illegal.
And such a policy would be easily be circumvented. Flip the image horizontally. Crop it. Change the resolution slightly. Add more jpeg. Write a meme on it. Change the color balance slightly. It might stop the exact same image from spread
Re: (Score:2)
Of course none of this will work.
But it gives the appearance of doing something, even if it doesn't have a hope in hell of doing anything.
I'm sure this is considered largely a PR move to show you're tackling the issue. But if anybody believes this will have any impact, they're kidding themselves.
Re: (Score:2)
They'll be using a perceptual hash. I recall Microsoft has one that's already in use in law enforcement for this purpose.
If it's a perceptual hash, changing resolution will achieve nothing. Nor will color balance, or jpeg compression. Meme might. Cropping or flipping certainly will though, at least for the phash algorithm I'm familiar with.
Re:keep honest people safe (Score:5, Insightful)
This probably isn't a bad idea even though it won't stop the perverts. It greatly lessens the chance someone will come across something they didn't want to see.
When they cam for the perverts, I said nothing, for I was not a pervert?
This exact technology will allow governments to exercise very powerful censorship across the internet (or at least the part of the internet most people see). Want all pictures from that protest rally to vanish? Just twist the arm of any of these companies into adding a few hashes, or just slip them into a list the FBI no doubt routinely provides, and, just like that, down the memory hole. Plus, as you say, this won't stop the perverts. The only thing this actually accomplishes is empowering the totalitarian state.
We seen a couple of stories here on /. already where IP blocklists were abused by governments to slip in websites of opposing political parties. It's a bit hard to believe this won't be abused similarly.
Re: (Score:2)
It's a bit hard to believe this won't be abused similarly.
Unfortunately you're right. History shows that any and all technology has been used repressively, and this won't be different. The problem is not technology but human nature.
I'm all for preventing abuse of children or for that matter abuse of anyone. But if someone thinks they have a simple answer that isn't a two-edged sword, they're dreaming.
Re: (Score:3)
The only thing this actually accomplishes is empowering the totalitarian state.
Since I'm usually on your side of that argument, I will respond by saying that there are already plenty of filtering/sifting technologies in place. I think you're getting to the point in your argument where "anything that can be used by the totalitarian state is a bad thing", which almost sounds like insisting we don't build or implement anything new that's useful to deal with large amounts of information.
Re: (Score:3)
If I catch my daughter camming for the perverts, I certainly am not saying nothing.
Re: (Score:2)
But... But...But...
How are folks supposed to find r@ygold, hussyfan, kingvid, pthc, babyshivid ???!?!?!? :P
Oh no, the darknet! (Score:1)
Why is the darknet Google's responsibility, again?
Will this work? (Score:2)
This isn't too different from our approaches to spam emails. But are these services actually used to share those kinds of images? I wonder who curates the list of hashes, and how long before someone starts adding pictures of stuff they don't like to the list.
Re: (Score:2)
Hit the nail on the head.
Who watches the watchers?
What accountability will there be?
Re: (Score:2)
Re: (Score:2)
We have to take it on trust that the lists are accurate and contain no false entries, because there is no independent confirmation and, given the contents of the lists, just possessing a copy without proper authorisation from law enforcement is a crime in most of the world.
Re: (Score:2)
Re: (Score:2)
From the wording of the article, I would imagine Internet Watch Foundation (IWF) would be the party that is responsible for curating/publishing the list of hashes. Here in the states, the National Center for Missing and Exploited Children, FBI, and Justice Department maintain a similar database [fbi.gov].
ah, the moronic slippery slope argument (Score:2, Insightful)
because we take enforcement action {X} against {Y}, then enforcement action {W} against {Z} in inevitable
try these on:
"we can't legalize marijuana, because then we have to legalize methamphetamine and heroin"
"we can't legalize gay marriage, because then we have to legalize marrying the dead and marrying animals"
do you see the problem? good, then know yourself: the slippery slope argument is failure, appeal to emotion, fear
the slippery slope only works if you are dealing with people who never actually think
Re: (Score:2)
The only thing I learned from your comment is that random italics reads as douchey as people who randomly emphasize in their speech, attempting to sound authoritative but winding up seeming pedantic and pretentious.
That's an emotional argument. Which is the only thing your comment requires, as you didn't even touch the substance of my comment.
Image hash too simple to bypass... (Score:2)
Re: (Score:2, Informative)
While it is an interesting concept, it is doomed to fail as a simple single pixel edit or hidden attribute edit will change the file's hash.
Its not a hash in the sense of MD5/SHA etc that hashes the file contents at the byte level. Its a perceptual image hash, a well studied technique.
A image will have the same hash even if a few pixels change or e.g its rotated/mirrored etc.. and resized of course
Re: (Score:2)
Re: (Score:2)
A few pixels change yes. Resistance to the others depends upon algorithm. Flipping or rotating work on the basic form of the phash algorithm. I expect they'll be using something more complicated, probably a composite hash incorporating several functions.
Re: (Score:2)
Re: (Score:2)
I wouldn't even bother.
I'd stick it inside an encrypted rar or 7z file.
You'd have to be a real idiot to upload child abuse images in the clear.
Re: (Score:2)
Re: (Score:2)
If you're dealing with the type of data that major police agencies are actively hunting for, you'd have to be an idiot not to inform yourself on how to get away with it.
Re: (Score:2)
Its not a hash in the sense of MD5/SHA etc that hashes the file contents at the byte level.
It's MD5, SHA1, and PhotoDNA hashes.
The standard in most law enforcement forensic applications is MD5 / SHA1, despite the obvious limitations.
Sadly, it still is reasonably common to encounter byte-identical images that are on the relatively small "known-bad image" hash lists.
Re: (Score:2)
Fortunately, people are idiots and often don't alter the files in any way before re-sharing them. Sometimes even the EXIF tags are intact.
Re: (Score:3)
There are "robust" image hashes out there, so that is not the real problem here. The real problem is the huge opportunity to do censorship this way. And anybody even trying to find out whether this is real or a complete abuse of the law will see some illegal pixels and will have searched for them, and hence can be easily removed to jail for a long, long time. Basically, doing it this way removes any legal possibility for ordinary citizens to evaluate what is going on and complain about abuses (except for ve
Don't they do this already? (Score:1)
I'm surprised it took this long. Google must have such a hash list already, better yet an MD5 list, built from their human reviewers of their robot webcrawled image search, so they won't show up in customer searches.
The real question is who keeps a database of pictures to review the list itself. Police? Google? Any normal prosecutor would happily prosecute Google for it just to add a notch to their belt (of asshole behavior).
Re: (Score:2)
a hash list already, better yet an MD5 list
Can a pedant come along
Of course.
and explain how MD5 is not a hash?
No, because that would not be right. MD5 is a member of the family of hash algorithms, so all MD5s digests are hash algorithm digests, but not all digests are MD5 digests. The GP is requesting a list of hashes, but would prefer the subset of that list that consists only of MD5 hashes.
Censorship (Score:2)
Whenever you start itching to censor content you don't like, just keep in mind that some countries consider pictures of women not in a burka to be illegal pornography.
Can't or won't? (Score:1)
It should have said they can't censor things on the darknet not won't.
If won't is correct are they running the darknet sites in question?
This is an attack on speech; Utterly futile for CP (Score:5, Informative)
Hate crime (also known as bias-motivated crime) is a usually violent (lock em up, kidnapping, sex offender lists, etc), prejudice motivated crime that occurs when a perpetrator (social justice warriors) targets a victim because of his or her perceived membership in a certain social group (paedophilia).
This fight against paedophiles is really just unjustified homophobia in disguise, a form of racism, etc. The Internet Watch Foundation is not attacking child abuse, they are attacking paedophiles. The very article describes this as a " fight against paedophiles". There is no reason to think the vast majority of paedophiles are harming kids. This is why they're attacking child pornography. It's an easy target that won't go away and they can't possibly eliminate.
It shares so many similarities with the war on drugs. If there is one group of people you can attack and generally get agreement on this is it. It's too small, unorganizable, spread out, etc, and nobody would dare defend it out of fear for there lives. This gives the social justice warriors ample room to do what they want. They are really nothing but a misguided group of racists spreading hatred and fear. You wouldn't attack homosexuals because some are sexually abusing little children. It's no different with paedophiles. The entire war on paedophilia is identical to the war on drugs. It's utterly illogical. The idea that porn leads to sexual abuse was disproved long ago. It's just like violent video games leading to violence in the real world. The reality is studies have shown the exact opposite to be true.
This is doing nothing other than implementing a system of censorship and giving people the perception something is being done to stop child abuse. It's not. The article even says they're not able to stop the spread. It's not even illegal to be a paedophile, and yet they have no aversion to expressing there hatred for this group. They accuse an entire group of wrongdoing when there is zero evidence of that. There isn't any means to even produce such evidence because all the studies that back up paedophiles being bad were done on an imprisoned population which wouldn't represent paedophiles as a whole. It only represents violent paedophiles. It's no different than doing a study on homosexuals after locking up homosexuals. There is going to be a disproportionate number of violent homosexuals in custody.
They're not attacking people who abuse children. They're attacking a group of people who are hated for no logical reason. It's no different than attacking homosexuals for what a minority have done.
Go watch the 1950's Anti-Homosexual PSA - Boys Beware to see exactly what I'm talking about:
https://www.youtube.com/watch?v=17u01_sWjRE
They imply all homosexual are evil-doers that want to harm little children. It's utter nonsense.
One Bit (Score:2)
from the ./summary:
...but said it would not block content on the 'darknet' — a network with restricted access — where abusers often posted images."
Well that is one way to defeat the blocking. Or you could flip just one bit in the entire image and that would change the hash. Pick an LSB anywhere and nobody will notice that it is a different image. Assuming it's a hash of the entire image, and not a subsample. And assuming that image compression does not swallow your LSB flip. Even without those assumptions, there will be many trivial and, to the eye, undetectable, transformations which would defeat the hash.
But something tells m
under the carpet (Score:2)
Blocking pictures of child abuse is like sweeping them under the carpet: we don't see them, but pedophiles can still download them. I fail to see how this prevents further abuse of the children in the pictures. And just to be clear: we are talking about adult men, sometimes elderly men, having sex with todlers. Penetration sex that is. And yes, that too.
Telecoms, ISPs and the hash list (Score:2)
Worked at AT&T and T-Mobile and they ran the hash list along with a virus scan on emails, mms messages and photo albums. I'm sure other image hosting services run the same checks. If any photo popped, you had to notify this third party company who acted with the cops. If a cop had a warrant, you would call the telcom to have an engineer to drop a dvd in the admin server, run a collect script that zips everything up, and then have the police department show up and pick up the dvd outta the dvd-rom drive
Re: (Score:2)
There is also no legal way for any ordinary citizen or company to verify the hashes are actually hashes of CP. They could be hashes for something else entirely. This is obviously an intended thing.
Re: (Score:2)
The courts won't convict you of having a file that hashes to a known bad number. They will convict you of having an actual image of child porn. The hash digests are just a way for ISPs and law enforcement to perform a comparison without needing the actual thing to compare against.
There's a lot more to this than just saying "child porn". Once an automated system discovers a file that matches a hash, they involve an investigator. The investigator first views the image in question, and if it's a false posi
Re: (Score:2)
You misunderstand. These hashes can be used to find and identify people that are sending entirely different, and possibly legal, pictures around. That allows them to do things that would require a specific warrant. But as nobody can verify what they are hashing, there is zero possibility for legal action to stop this.
Re: (Score:2)
Ah, so you're saying they would follow people sharing a picture of the "Secret Seal of the Anonymous Collective", and since they're the evil bad guys of the month, let's track them down too. Good point.
I see a problem. (Score:2)
What utter moron of a child abuser would upload their pictures to facebook?
They might not be the brightest of criminals, but seriously... they'd have to be pretty dumb to do that.
Re: (Score:2)
Re: (Score:2)
Right on the mark. And that is why the stated goals are rather obvious lies.
THE DARKNET (Score:3)
This is going to....omg (Score:2)
I have trouble poking technical holes here since, fundamentally the idea of using a hash table is somewhat sound, its used all the time for UUIDs, theres plenty of uniqueness right? I guess maybe we can rule out collisions for the most part....hell maybe pair the hash with a file size?
If we are talking such tried and true technology and not some recently invented "photo hash" that I wouldn't have any faith in the uniqueness of....
but then the implications of just having such a system means things can be inj
Re: (Score:2)
Not to mention that if they're just storing hashes, that a tiny change will generate a completely different hash. Change one pixel, by a single bit, and you have a different hash. Change the resolution, change meta-data, crop out a row or column, etc., etc., a service that serves up the images could do this automatically every time it displays an image.
There's lots of ways around this, including only sharing content with trusted people via sneakernet if it comes to that.
It sounds spiffy to the average perso
Re: (Score:2)
So maybe you missed it but....that doesn't make me feel better, in fact, I find the idea of the mass application of something so new as a pretty frightening prospect.
Re: (Score:2)
That doesn't make me feel better either. So instead of matching on exactly the same image, it is going to flag "yeah, it's kind of like that"? Not a fan.
getting paid to watch child pornography (Score:2)
Who are these people who get paid to watch child pornography all day? (i.e. The people who classify the images.)
There are some experts in this area who would be willing to offer their services for free to help Google out.
Re: (Score:3, Insightful)
Your comment matches a hash submitted to the block list. Please report to the authorities for mandatory castration. Have a nice day.
Re:First porn, and then... (Score:5, Insightful)
Wait, is it abusive to view and distribute? It's abusive to create, for certain; I'm not sure I buy the line about viewing and distributing.
OIG explained to our entire department one day that, each time a person views a child pornography image, the person in the image is victimized again. I've not yet wrapped my head around the idea of someone suddenly stopping somewhere as the finger of God touches them inappropriately, collapsing to their knees and gasping for breath in distress as some dude in Korea looks at their naked 12-year-old body.
Many in the last decade held the opinion that the greater crackdown on child pornography possession was an excuse to draw attention away from the lack of action against child pornography production. What happened? Do we now all accept producers and care most about consumers? That sounds like a by-the-numbers approach confounding two very different things: 10 producers and 990 consumers are not 1000 child abusers, but 10 child abusers. Eliminate 900 of the consumers and you still have exactly as many children being abused exactly as frequently--and my own sense of doing it by the numbers tells me the numbers aren't any better in that case. I'll be the first to push 100,000 kids off a cliff in a bus to save 1,000,000 people from terrible death, but methinks you've simply avoided saving anyone and, perhaps, saved yourself dirtying your hands with the bus.
It's not that I disagree with what you're doing, Mr. Anderson; I just want to ensure you're going about it in the most efficient way.
Re: (Score:1)
Take away the consumers and the producers will have fewer reasons to produce. It isn't a perfect system, but they do go hand in hand.
Re: (Score:2, Insightful)
That would work if most CP was created for profit instead of for the joy of molesting a 12 year old and sharing that with others of the same mindset.
Re: (Score:2)
It seems like the wrong approach. If the war on drugs has shown us anything, it's that targeting the larger pool of users isn't terribly effective. Unlike the war on drugs, however, we wouldn't be fighting a hydra. I would doubt that there's a huge hierarchy, with a host of lieutenants ready to step in to the lead role. Targeting producers, then, is likely to have a far greater impact.
Re:First porn, and then... (Score:5, Insightful)
Re: (Score:2)
How about recognizing that these feelgood solutions are more for getting it outta sight outta mind than for actually stopping the abuse of children?
I had not considered that. A shame you posted AC, as this is well-deserving of a +5 insightful.
More than just the high potential for abuse, it's worse-than-useless.
Re: (Score:2)
You are suggesting legalising the possession of child pornography. Only a child abuser would want that!
BURN THE EVIL ONE!
Re: (Score:2)
There is something inherently wrong in making the possession of any specific configuration of bits illegal. Sure, selling them, creating them is different, but criminalizing possession is just a cop-out by law enforcement to make their job easy so they do not have to prove intent. Hence it has become so easy to kill people socially, just hack their computers and put some CP on then and then give an anonymous tip to law enforcement. In many countries, drawings are already enough to get the desired effect. Th
Re: (Score:2)
This is simple economics: If they stop 10 producers of CP, they have arrested 10 people. If they stop 1000 consumers, the still have the 10 producers out there, creating new consumers and they can claim 100x the "success". Add to that that in many countries no-victim CP (drawings, comics, texts, etc.) are illegal as well, they can present their huge successes in "fighting CP" to get more power and funding, while at the same time they can make sure the sources do not dry up. Of course this means not fighting
Re: (Score:2)
Re: (Score:2)
So you're saying the whole system is taking ineffective measures on purpose because it's corrupt? That wastes my money. That behavior is an economic drain, creating more poverty and slowing wealth growth by engaging in cost-producing services with no productive output.
The goose that lays golden eggs eats entire grain silos of food. You can trade a hundred golden eggs for that food. They're stealing my food and getting an inefficient return. Kill the damn goose so I can eat it.
Re:Justice system (Score:5, Interesting)
We don't find people filming murders for sexual gratification. If that were the case, then that could very well become illegal too.
Although fake kiddie porn is just as illegal as the real thing, filming fake murders for others' gratification (hopefully not sexual, but who knows) is big business. Hollywood makes billions on it. Ditto first-person shooter games.
Something is screwed up somewhere.
Re: (Score:2)
Doesn't the hashtag of a picture change by editing one pixel? Or by removing one line of pixels?
I'm pretty sure that posting an image with the comment "#thisischildporn" is going to raise some eyebrows no matter what how much you change the image.
The cryptographic hash [wikipedia.org] of a file, on the other hand, is usually a little more sensitive.
When dealing with image recognition you really want to account for things like the image being resized, colours adjusted, or borders being added and removed. In this case you would want to do something like take the image, rescale it to a fixed size, reduce the colour
Re: (Score:2)
You're missing the purpose of the digest as it is being used by Google. They are not being given cryptographic hashes, and they are not trying to assure anyone of the integrity of the file. These hash digests are being used only as an index to perform comparisons without needing to keep the original files around. Think "java HashMap", not SHA-1.
When they find a matching value, Google doesn't try to do any other comparisons or investigations. They blindly zip up the file and log information, then turn it ove