San Francisco Team Wins DARPA's De-Shredding Contest 94
New submitter karlnyberg writes with an update to the recently announced de-shredding challenge posted by DARPA: "The team 'All Your Shreds Are Belong To U.S.' has correctly solved all five puzzles, and the Challenge has now ended. You may view the winning team's submissions as well as the complete puzzle solutions by following the links on our homepage. We recognize that many of our participants have devoted countless hours to painstakingly piecing our puzzles back together, and we truly appreciate everyone's efforts. Hopefully you enjoyed the Challenge and learned something new along the way. We certainly did!"
Jackasses (Score:5, Interesting)
WTF? (Score:5, Insightful)
It's not paranoid when they've established a consistent pattern of spying on citizens without cause in the wake of 9/11.
Re:WTF? (Score:5, Insightful)
It's not paranoid when they've established a consistent pattern of spying on citizens without cause in the wake of 9/11.
Yes, because everyone knows they weren't spying on you before 9/11.
Sure, their legally granted powers have increased without bounds since then, but they've been spying on you right from the start.
Re:WTF? (Score:4, Informative)
Actually they weren't allowed to under previous law. So they simply paid the British and Australians to do it for them and did likewise for them.
Now that law doesn't exist so they don't need to bother with the loophole of paying a friendly country to do the spying job for them. You don't think the massive data center being built at Fort Williams in Utah is for spying on other countries do you?
Re: (Score:2, Insightful)
Actually, this is incorrect. Legally, the big five were not allowed to spy on each other: Australia, New Zealand, Canada, The United Kingdom and of course The United States [wikipedia.org], all collaborated on SIGINT, but were not allowed to intercept each other's communications. There were exceptions, however, such as Aid to Police, but this required warrants etc.
If a signal was being intercepted, and it was determined that one of it's end points was one of the five, it was no longer fair game to be intercepted.
If
Re: (Score:3)
Re:Jackasses (Score:5, Funny)
Re: (Score:3)
I use water.
There is a bucket in the corner, documents that have been scanned are tossed in the bucket, water is topped up every so often. Stir the paper around, bag the ball of gunk and recycle or garbage it.
Works quite well.
Re: (Score:2)
Re:Jackasses (Score:5, Insightful)
The government just PAID to prove that five shredding methods suck.
If you're still using them you're just a narcissistic terror poodle.
Re:Jackasses (Score:5, Insightful)
Oh for fucks sake. THEY DON'T CARE ABOUT YOU. There are no feds swooping in in black helicopters to dig through your garbage and piece together your shredded electric bill.
Honestly, mods, giving positive reinforcement to this sort of paranoia is only hurting the people suffering from it.
Re: (Score:3, Interesting)
And in this day and age, everybody's a criminal. The FBI follow up on every anonymous tip sent to them online, and no matter what you tell them, they will follow you around until you're officially deemed uninteresting. Then they'll check up on you every now and then just to make sure. And they will dig up something on you that will justify th
Re: (Score:1)
I'm pretty sure the bar is a teensy bit higher than making a joke on the Internet. I know it's hard to make non-points like yours without hyperbole so you're sort of excused, but just know your bullshit doesn't pass a basic sniff test.
You're wrong [slashdot.org]. The offending comment [reddit.com]:
bombing a mall seems so easy to do. i mean all you really need is a bomb, a regular outfit so you arent the crazy guy in a trench coat trying to blow up a mall and a shopping bag. i mean if terrorism were actually a legitimate threat, think about how many fucking malls would have blown up already.. you can put a bag in a million different places, there would be no way to foresee the next target, and really no way to prevent it unless CTU gets some intel at the last minute in which case every city but LA is fucked...so...yea...now i'm surely bugged : /
Clearly, you can see how he presented a danger to national security theater.
Re: (Score:1)
Comment removed (Score:4, Interesting)
Re: (Score:1)
On an unrelated note, don't moderate on laggy browser, or you might accidentally the wrong choice. Posting to cancel the mod. Sorry. IGNORE ME!!!
Undercover? (Score:1)
Do you mean he (or seriously) blew cover to arrest you for a donut joke?
Around here blowing your cover for a non-serious issue can actually get an officer in trouble (according to what I've heard from other police).
Re: (Score:3)
Re: (Score:3, Insightful)
Re: (Score:2)
Re: (Score:3)
Of course they do. As times keep on getting worse, the possibility of Joe 99% engaging in subversive actions gets ever greater, so if the 1% are to keep on looting the rest, they need to tighten the grip ever more. And of course, even in good times, "they" are people who love having power over others, either because they think they know better than everyone else or simply because they have issues.
Re: (Score:2)
If you don't want the government to spy on you, burn the papers and scatter the ashes. Shredders have never provided any real security, since you can simply re-assemble the pieces by hand, it just takes a while.
opportunity (Score:3, Insightful)
There is a business opportunity for better shredders, the kind that would pulverize the paper or better just burn it.
A shredder with a vortex or a burner inside.
Don't get me wrong, it's an interesting technical challenge that this was done, but if you want to keep your paper out of government's hands you shouldn't be just shredding the paper.
Re:opportunity (Score:5, Insightful)
Actually, it's not the "US government reading our stuff" thing that caused this whole contest. If The NSA wants your credit card info, they'll just ask Visa for it, pay the usual fees, then just get it mailed to them.
The government shreds a lot of paper. Probably more in a day than I generate in a lifetime. The Classified shit they shred when they're done (IAW DOD-WTF-1234) because it's easier than burning. Trust me, you don't want to burn Classified stuff.
Someone probably said, "hey, do you guys think that with today's video cards and CPU power, someone could unshred our shit?"
"Oh... aw fuck, Dave, I dunno. Let's ask DARPA."
Thus the challenge.
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
The trouble is the safety bit especially when the device has to be small.
Anyone can make a big incinerator. I suppose in top secret places, you'd shred the documents first then send the results to be incinerated. So the documents are exposed in shredded form only within the premise.
Re: (Score:2)
I think there was an engineering and procedure mod before the incinerator went back online.
Re: (Score:2)
The government shreds a lot of paper. Probably more in a day than I generate in a lifetime. The Classified shit they shred when they're done (IAW DOD-WTF-1234) because it's easier than burning. Trust me, you don't want to burn Classified stuff.
Someone probably said, "hey, do you guys think that with today's video cards and CPU power, someone could unshred our shit?"
"Oh... aw fuck, Dave, I dunno. Let's ask DARPA."
Thus the challenge.
Not at all.
When the government shreds classified documents, at a bare minimum, they turn it into confetti.
At the highest classification levels, you end up with strips less than 1mm x 5mm.
There's nothing recoverable from that level of shred.
Re: (Score:2)
Of course there is, it's just a lot of work. Scan the pieces, find likely matching edges for each piece based on how closely they match, then start going through various combinations based on image (letter shape) recognition, filter based on a dictionary (what combinations create legible words).
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Why does it hold your secrets?
No matter. Encrypt.
Re:opportunity (Score:4, Interesting)
Not really. There are already tons of better shredders, and they don't cost a whole lot. I bought one that cuts pieces into little diamond-shaped bits about 1/3 of the size of the shreds shown in the DARPA challenge. It cost me about $125 at an office supply store.
The one that is at my office, for non-classified documents, cuts into little squares about 2mm x 2mm. The one for the Confidential documents does a better job that that. Most of the stuff looks like powder instead of shreds.
Anyway, this isn't about spy vs spy stuff. It is for basic corporate and citizen espionage. How simple is it to piece together stuff you grab from drug dealers and other criminals who bought a shredder at an office supply store?
Re: (Score:3)
Anyway, this isn't about spy vs spy stuff.
Funny you should say that, because actually it was. See the solution to the challenge and the puzzles for full details.
Re: (Score:1)
Anyway, this isn't about spy vs spy stuff. It is for basic corporate and citizen espionage. How simple is it to piece together stuff you grab from drug dealers and other criminals who bought a shredder at an office supply store?
Whatever happened to flash paper
Re: (Score:2)
Re: (Score:1)
I wouldn't be concerned about any environment at all if the threat to me is from a gov't. I would only be concerned with my secrets. Environment is not even in the distant back of my mind when I want to hide my secrets from a gov't.
Re: (Score:2)
I don't think it's practical tech for home use.
Ob Rainbow's End reference (Score:5, Interesting)
Vernor Vinge's 2006 novel Rainbow's End [wikipedia.org] explained how a library was being digitized by shredding all the books, thus destroying the analog knowledge.
One step closer...
Re: (Score:2)
You're right, s/knowledge/artifact/. People have an attachment to the physical objects long after it's become obsolete, which is why I still buy hardcover books in this age encroaching e-books. The novel explores these themes well.
Re: (Score:2)
Re: (Score:2)
I have a working Zip Drive, SCSI. I also have the cables and a card, along with the drivers and an old 386SX that will read it.
Not that I have a VGA monitor anymore....
Re: (Score:1)
Meh (Score:5, Funny)
Wake me up when they have the Ash Challenge.
Re: (Score:1)
and that is why a wood stove is so nice, you have a warrent to go through my home, or you break in to steal confidential information, fine and if you want my documents they are in that bucket of ash i haven't had the time to dump in the woods behind my house. go to.
$50,000 (Score:3)
Previous topics included that the abilities gained were worth far more than $50,000.
So was this a fancy Job Application?
As I said before... (Score:5, Interesting)
Why, then, would we expect them to use a shredder for their documents, when they can much more easily set fire to the documents? No amount of de-shredding is going to put back together documents that have been incinerated.
I suspect we are much more likely to see this used by the FBI than the CIA or DOD.
Re:As I said before... (Score:5, Informative)
You might be suprised. A fire can be dangerous, a shredder is convenient. Also, we can reconstruct documents that have been burned, and if you have a big stack it can actually take longer to burn them beyond recovery. Why? It's about oxygen availability - the corners, top and bottom pages will burn first, but the center of a pile of documents will often remain intact, yielding valuable intelligence if recovered.
Having looked up the shred, it seems to be standard commercial shred sizes - the DOD goes quite a bit smaller than that.
Re: (Score:3)
A fire can be dangerous, a shredder is convenient
A warlord is more or less equally as dangerous as uncontrolled fire.
if you have a big stack it can actually take longer to burn them beyond recovery. Why? It's about oxygen availability - the corners, top and bottom pages will burn first, but the center of a pile of documents will often remain intact
That is assuming that someone is burning only paper. Indeed, a stack of paper can be a bitch to burn beyond recognition. But if you're in the business of raping women and torching villages, you just throw the documents into a building that you are going to burn to the ground and then you leave town. Eventually the temperature of the fire gets high enough that the paper incinerates well beyond any hope of recovery.
And fire is one th
Re: (Score:2)
But if you're in the business of raping women and torching villages, you just throw the documents into a building that you are going to burn to the ground
Just for clarification... are you suggesting that a warlord leave his sensitive documents with a henchman to leave in a building that they must then burn to the ground. then leave without verifying that it succeeded?
Heh. That reminds me of the Onion article where the US offered to donate Al-Qaeda 600 million dollars so they could build a fancy HQ building.
Re: (Score:2)
A warlord is more or less equally as dangerous as uncontrolled fire.
Just because he's unconcerned with the safety of others and their stuff, doesn't mean that he's unconcerned about the safety of himself and his stuff.
That is assuming that someone is burning only paper. Indeed, a stack of paper can be a bitch to burn beyond recognition. But if you're in the business of raping women and torching villages, you just throw the documents into a building that you are going to burn to the ground and then you leave town. Eventually the temperature of the fire gets high enough that the paper incinerates well beyond any hope of recovery.
You're positing a pretty one-dimensional warlord in this case, who always has a building at hand to torch whenever he needs to dispose of some documents. Most warlords aren't fighting all the time, and often the important papers are back at his palatial palace of an HQ, where he DOES have electricity most, if not all of the time due to being first on the repa
Re: (Score:2)
Just because they're barbaric in the moral sense doesn't mean they are in the technological sense. They are quite capable of using computers and cell phones and shredders.
Re: (Score:2)
They are quite capable of using computers and cell phones and shredders.
They certainly are. However a shredder requires electricity, and warlords often thrive in areas with inconsistent (or worse) electricity. In comparison, any hack with matches and gasoline can torch a building, day or night.
Re: (Score:2)
DOCEX (document exploitation) is a very active activity within the armed forces.
Re: (Score:1)
It seems to me that a large amount of people commenting in this have not tried to burn a large amount of paper, something easily 20-30 reams worth of documents worth....
From personal experience, burning is an option, but as my oh so wise leadership mentioned during the burning " It would burn quicker if you shredded it first...."
Yes, but ... (Score:2)
Yeah, but ... (Score:2)
...it'll cost you fiddy gees.
As a loyal Merican (Score:1)
I'd be happy to help train DARPA's AI, but not when it's going to assist such an unconstitutional monster as the Patriot Act. That's like, you know, pumping the bellows on the forge building your chains.
simplified test (Score:5, Informative)
The test was actually much simpler than any real-world application might be. Each puzzle was really only one or two (or a few) shredded pages, with various degrees of shredding and various bits of writing. It is a first step, but nowhere near what you would be dealing with in any real-world situation where hundreds or thousands of pages of shredded documents would be mixed together.
I participated (a bit) with the UCSD team that basically made a crowd-sourced jigsaw puzzle to do it - at last check they were in the top 5, but I don't think they got the last puzzle (yet). This approach seems reasonable for the relatively simple puzzles of the challenge, but it really wouldn't scale very well - requires a lot of labor.
It sounds like the winning team had a much better (and more scaleable) strategy, where an algorithm scores all of the pieces for fit in a particular place and then allows the user(s) to choose the best piece from a few high-scoring ones. While I still don't think this would work very well in a real-world scenario, obviously it would work better than depending on massive crowd sourcing.
Re:simplified test (Score:4, Interesting)
The test was actually much simpler than any real-world application might be. Each puzzle was really only one or two (or a few) shredded pages, with various degrees of shredding and various bits of writing.
Yes and no. The finest shred was actually pretty close to the current DOD spec of 1mm x 5mm for classified documents. You'll find that most sensitive but not classified shredders shred into much larger pieces, somewhere in the middle of the range given in the contest. You are correct that a typical shred bag has a large quantity of shredded pages mixed together.
Still DARPA got what they wanted - ideas on how to approach this problem. I doubt crowd sourcing is a viable option for them, but it was interesting to see the method.
Re: (Score:2)
True, but it's unlikely to find a heap of shreds related to 2-3 pages.
It's more likely you'll find the equivalent of hundreds of pages thrown together in a heap.
This makes scanning a much bigger job (*), as well as the actual piecing stuff together again.
(*: I haven't checked the results yet, but did they consider that each piece has two sides ?)
Explanation of strategy for the Shredder Challenge (Score:5, Interesting)
Here's a nice explanation of the participant which reassembled four of the five documents, finishing in third place.
You should probably start burning your mail: What I learned from the DARPA Shredder Challenge [marcnewlin.com].
Re: (Score:1)
Shredders are good enough for classified docs (Score:5, Informative)
If that's not good enough, some locations use burn-boxes - never trust a machine to do thermodynamic's job!
Re: (Score:2, Informative)
The NSA-approved shredder I used made chad so fine that the frayed fibres
at the edges made up a large percentage and maybe the majority of the surface
of each chad. There is so much information lost in that area that it may not be
possible to reconstruct such documents in the future even with state-level resources.
Stasi (Score:2)
Re: (Score:2)
With everyday of sorting, a name might drop out.
Just think what the East had on some EU political parties too