An Advance In Image Recognition Software

Roland Piquepaille alerts us to work by US and Israeli researchers who have developed software that can identify the subject of an image characterized using only 256 to 1024 bits of data. The researchers said this "could lead to great advances in the automated identification of online images and, ultimately, provide a basis for computers to see like humans do." As an example, they've picked up about 13 million images from the Web and stored them in a searchable database of just 600 MB, making it possible to search for similar pictures through millions of images in less than a second on a typical PC. The lead researcher, MIT's Antonio Torralba, will be presenting the research next month at a conference on Computer Vision and Pattern Recognition.

There goes the neighborhood

This will be used to break CAPTCHA-type schemes even worse than they already are.

Re:

I had thought it would be more along the lines of categorising pictures into 'PORN' and 'NOT PORN'. Or possibly even 'PORN', 'GAY PORN', 'SHEEP PORN' and 'NOT PORN' if it's really advanced.

Re:There goes the neighborhood

This will be used to identify YOU, citizen.

Re:There goes the neighborhood

Looks like I'll have to stop attaching my low bit facial image as a signature.

Re:There goes the neighborhood

Incorrect. First of all, in a CAPTCHA, you're trying to very rigorously inspect a single image. This advance seems to be more about taking quick glances at lots of images. Furthermore, in the article, they talk about recognizing flowers and cars. The fact is, computers already have no problem recognizing letters and numbers in images. We got that down a long time ago. The difficult things about reading a CAPTCHA image are removing distortion and splitting the whole image into the component characters. If you read the article, you'd see that this research has nothing to do with that.

Like every other "advance" in image recognition...

...I'll believe it when I see it.

Until then, it's snake oil, as far as I'm concerned.

Re:Like every other "advance" in image recognition

If I read you correctly - and I think I do... You mean to say that snake oil is somehow... invisible?

No wonder those snakes are not only so quiet, but I never even see 'em coming!

Geez. We don't stand a chance.

Re:Like every other "advance" in image recognition

Or rather... I'll believe it when it sees me!

Re:Like every other "advance" in image recognition

The actual paper is at http://people.csail.mit.edu/torralba/publications/nipsRecognitionBySceneAlignment.pdf [mit.edu]

From what I can tell, it's basically, "blur the image down to only a few hundred pixels and then you have less data to comb through!"

Re:Like every other "advance" in image recognition

Yep, I second that. The article is really short on details. Not surprising since they're presenting it at a conference next month. We don't even know what kind of features they are extracting from the images. Are they using wavelets? Texture descripto

Re:Like every other "advance" in image recognition

Read the papers then
http://people.csail.mit.edu/torralba/tinyimages/ [mit.edu]

Oh thats really simple to do...

.... the answer is always "a picture"

Of course it helps if you read the papers...

I hate reading press releases of reading papers with real explanations of what's going on.

I just finished reading "Small Codes and Large Image Databases for Recognition" written by the guy. All he did was implemented Geoff Hinton's idea of databasing images with a binarized coefficients produced by Restricted Boltzmann Machines.

Hinton himself gave a talk on it for Google here:
http://www.youtube.com/watch?v=AyzOUbkUf3M [youtube.com]

Actually I'm wondering, is he plagiarizing Hinton?

Re:

I think plagiarizing is a strong word to throw around. And particular implementations of general approaches can often be very interesting when one considers what tradeoffs are made transferring pure theory to practical applications. If this sort of thing

Re:Of course it helps if you read the papers...

Jeff Hinton worked with them, you really think they're plagiarizing him? That claim doesn't even make sense, this is a novel research domain. A big part of science is taking people's ideas, reproducing them, and applying them to novel domains. That's how it's SUPPOSED to work.

This research involves the use of one of the largest image databases seen in computer vision. It shows that you can do extremely rapid scene matching for databases of this scale. No, that's not obvious no matter what you think. This image data is fairly high dimensional.

This research says something about the space of likely scenes and it might be a key enabling technology to a lot of the heavily data driven computer vision and computer graphics approaches popping up lately.

Oh really?

They're going to distinguish an individual based on images with 256 to 1024 bits of data?

I guess nobody there thought to do the math before making these claims. This story probably shouldn't have made it to the front page; it's less than useful.

Re:

"They're going to distinguish an individual based on images with 256 to 1024 bits of data?"

No one said they were going to identify individual people with this. The main gist of this research seems to be efficiency (in both space and time, if I read it cor

not so much an advance

This is not so much an "advance", and more a demonstration that some image recognition problems can be solved with fairly simple, well known methods and a lot of data.

But can it tell the difference between....

a fake ufo picture and a real one?

How will spammers make use of this? Well just make that viagra pill be reflected in a coke bottle.

anyone for a random bit generator to see what random results gets labeled?

Wonder what fractals might produce?

moral of the qu

Search Jenna Jameson?

Oh my, the soon to be most searched "name" on the web is... Jenna Jameson! Wait a minute, I think I misunderstood "facial" recognition...

Scary.

Given this works under real world conditions, it would make it possible that every shop gets the list of faces of criminals or other people which "you dont want" in your shop, recognize them in real time using little investment only and throw them out. Or

Very cool stuff...

This actually solves a problem I've been stumped on for a while. I need a way to search for similar images such that images that are similar have a searchable value with an inherent "nearness" quality.

That is, there are a number of image similarity algorit

Re:Very cool stuff...

What you're asking for is ill-defined, but much sought after.

A reasonable descriptor which produces distances that seem somewhat correlated with human perception would indeed be Antonio Torralba and Aude Oliva's gist descriptor.

http://people.csail.mit.edu/torralba/code/spatialenvelope/ [mit.edu]

It's become quite popular in computer vision and computer graphics for scene matching.

Re:

Any decent object recognition algorithm supports at least affine transformations, which include rotation.

Some of those scientists are actually pretty smrt.

Re:I forsee nefarious law enforcement uses

There are all kinds of ways, but two simple ones come to mind. If you convert to a polar coordinate system the power spectrum is conveniently orientation independent. You can use the same trick with a shift: the power spectrum of a Cartesian coordinate system is shift independent.

Another way is to somehow identify the orientation. An simple way to do that is to find the axis along which there's maximum variation and rotate until those axes match in both images.

Pixel by pixel co-registration basically does look at a similarity measure for a lot of variations on the affine transform. You generally don't have to look at them all though: you use an iterative algorithm with a clever optimization strategy so your transform gets better and better instead of searching through the parameter space randomly.