PhotoSketch Image Manipulation Tool Taking the World by Storm 193
PhotoSketch, a new image manipulation program that combines stick-figure sketches, internet image search and pattern matching, seems to be spreading like wildfire. Created by five Chinese students at Tsinghua University and the National University of Singapore, the tool takes a basic sketch and simple labels and turns it into a polished image. "Although online image search generates many inappropriate results, our system is able to automatically select suitable photographs to generate a high quality composition, using a filtering scheme to exclude undesirable images," say the PhotoSketch team in an abstract outlining the tool. "We also provide a novel image blending algorithm to allow seamless image composition. Each blending result is given a numeric score, allowing us to find an optimal combination of discovered images. Experimental results show the method is very successful."
Cool idea, but lacks specificity (Score:4, Interesting)
I tried to draw a picture of a man with an erection. I labeled him "porn guy". Then I drew a picture of a woman with her mouth open and labeled her "porn whore cumshot".
The composite picture was fine except that the man and the woman were far apart from each other. In addition, even if I were to draw them closer together (hey, I'm working with a mouse here), the result would still have been sized incorrectly.
This technology holds lots of promise and is already pretty cool. I hope they can work out the kinks.
This is unbelievable (Score:5, Interesting)
I'm not 100% sure, but I can definitely see the potential for Google to snatch this up really fast and incorporate it into Picasa or even google image search or something. The fact that something like this allows anyone (not just artists) to come up with novel images with minimal effort is fantastic. I do wonder how canned the images were though. IE: did they GIS for an image first, then use the image as a basis to draw the stick figure, knowing that their algorithm would pick the image they selected in the first place? I would like to see a live demo with an unplanned audience member doing the drawing. Then I'll really be impressed.
Re:Cool idea, but lacks specificity (Score:1, Interesting)
I wonder how exactly you did that, given there's no actual demo available and stuff.
Rough around the edges (Score:4, Interesting)
Off the top of my head (Score:2, Interesting)
2) Could the algorithm be used to find existing images similar to the one you just drew?
3) When is a demo of this thing available?
Re:Copyright implications, mashup (Score:2, Interesting)
Aren't mashups already in a copyright gray area?
xkcd (Score:5, Interesting)
Turn stick figures into photos? (Score:4, Interesting)
Someone should take all the XKCD comics, mark 'em up a bit, turn 'em into nice pictures, and .... Profit!!
Re:This sums it up quite nicely (Score:4, Interesting)
I'm amazed at how well this seems to automatically extract subjects from their background, something that usually requires a lot of painstaking manual work... honestly that's the real challenge of "photoshopping", becoming a ninja with the selection tools.
You've got it backwards (Score:2, Interesting)
The fact that a Chinese university is doing cutting-edge research is a good thing for you Americans. That means they're getting richer, thus a growing market for the pop culture products and Hollywood entertainment you're so good at exporting. Maybe 80% of the entertainment in the telly here in Denmark (in Europe) is from the US.
Now you just need to teach them to abide your copyrights. Maybe they can teach you how to eat vegetables in return. Fix an obesity problem or two, eh?
Re:Hoax? (Score:3, Interesting)
Why? What are you basing that assessment on?
I'm drawing on my experience with image processing and what I know of it's current capabilities. I've worked on programs to do this sort of thing. So, yes, I "know some shit" about this subject.
We're just not yet at the point where a program could be given an arbitrary image and have it recognize a wide variety of objects in the image. For example, take a single picture of a dog in a park (that is, no stereo imagery, or video, here): we don't have algorithms yet that can recognize the dog as an individual object, recognize it as a dog, and then segment that dog from the image. You might be able to write a pretty good program that specifically finds dogs in an image that is guaranteed to have dogs (like digital cameras recognizing human faces), but it wouldn't be useful as soon as you want it to find something else in addition to dogs. It would frequently see dogs where there aren't any.
Segmenting and object recognition is a lot easier given a pair of stereo images, because the 3D structure of the image is available, but that would greatly reduce the pool of available images. They couldn't be just yanked off Flickr. In video, moving objects are easy to find and segment, even if the camera is moving, but then you'd need video for all the input. Our human brains can recognize distinct objects in a single, flat image because they make may assumptions about the structure of the image, based on instinct and experience, filling in for the incomplete data. This has been hard to do with AI.
We're just starting to have programs that can recognize arbitrarily rotated, rigid, static objects -- like a shoe or a specific tea pot -- under very controlled circumstances (I know someone that was researching this). So, I'd bet their input images for PhotoSketch already have all the recognizing and segmenting done for them by humans. The sketch would be limited to whatever objects humans decided to mark up.
Building that input image library, marking it all up for the computer (object category, segmentation mask, perspective), would be a slow, tedious, and expensive, making this ultimately not very useful. The library would always seem inadequate for everything but trivial sketches.
Given a library with the necessary objects (correct type and perspective) putting these together in a plausable way seems like the part that would actually work fairly well.
So that's why I'm skeptical that this doesn't work nearly as well as advertised.