Slashdot Log In
Seeing Around Corners With Dual Photography
Posted by
timothy
on Tue May 10, 2005 07:12 AM
from the expand-e6-expand-expand dept.
from the expand-e6-expand-expand dept.
An anonymous reader writes "This project (which is part of this year's SIGGRAPH) has absolutely blown my mind. Basically they photograph an object with the photosensor at one point, and the light projector at another, and use the Helmholtz reciprocity algorithm to virtually switch the locations of the camera and projector, showing exactly what the light source "sees"! If that doesn't make sense to you, check out the research page and make sure to watch the 60MB video at the bottom. The playing card trick will leave you speechless!"
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
buh-bye server... (Score:5, Funny)
The exploding server one has already rendered me speechless. Why in the name of god do they do it!
Re:buh-bye server... (Score:5, Funny)
Parent
Direct http mirror here (Score:4, Informative)
Parent
Re:buh-bye server... (Score:5, Informative)
Parent
Why don't they just move the camera? (Score:5, Funny)
Re:Why don't they just move the camera? (Score:4, Informative)
No it can't. The light source must scan the target, not just illuminate it.
The only place I know of with a scanning light source that might be exploited is the confrence room. A photodetector would able to get a raster image of the Power Point presentation in the room and the presenter when he walked in front of the screen and became a scanned object.
Parent
Re:Why don't they just move the camera? (Score:3, Interesting)
Re:Why don't they just move the camera? (Score:5, Interesting)
How much information can you gather from that reflected light?
You could of course recinstruct the image on the CRT, but that's not very interesting.
The TV does not scan a focused image on its surroundings like the projector does, so you couldn't get a TVs-eye view of the room witht eh same technique.
OTOH, it is clear that from sampling even just a single point on the wall, you could get a silhouette of anything occlusion over the screen seen from that point. At least provided you had a pure white image on the CRT, OR knew what image was on and could calibrate for it.
How far could you get with all the information escaping the window in your direction?
Parent
Re:Why don't they just move the camera? (Score:5, Informative)
It's called optical tempest. With a high enough sampling rate you can reconstruct what is being shown on the monitor/TV. Each pixel as it illuminates causes a brief spike in the ambient brightness; by measuring this spike one can reconstruct the pixels being shown. After that, it's pretty simple to find the horizontal and vertical retraces.
more info [slashdot.org]
Parent
Re:Why don't they just move the camera? (Score:3, Informative)
Re:Why use a camera? (Score:5, Insightful)
The article claims that there is no need to describe the geometry of the scene, and I understand why that is true for the structure of the subject, but it seems as though the geometry of the light and camera would still have to be known. Anything that isn't in view of the camera in the first image is unlit in the second image, and vice versa, but I don't understand how you would determine what transformation would result in that exchange without any information on the camera-light geometry in relation to the scene.
Parent
Why don't the editors link to mirrordot? (Score:5, Informative)
Re:Why don't the editors link to mirrordot? (Score:5, Funny)
Kaboom, Slashdot and Mirrordot slashdotted each other!
Parent
University of Virginia Mirror to Video (Score:5, Informative)
Enjoy. [virginia.edu]
Parent
Re:Why don't the editors link to mirrordot? (Score:3, Insightful)
The real question is why nobody made a torrent of this video before the story went live. Bittorrent is one of the posterchildren of open source and legit p2p, it's unfortunate that here on Slashdot, the center of the community, nobody ever bothers to use it for it's intended purpose. We have an opportunity to put a great FOSS project to a vitally needed user, but instead they choose to continue crushing
Re:Why don't the editors link to mirrordot? (Score:3, Interesting)
I don't see any reason a torrent client can't be set up to allow a HTTP seed in addition to all the torrent peers and seeds. Granted it's going to get very poor speed, but as soon as a chunk makes it out into the swarm it should disperse to everyone fairly rapidly, and the more automated it is, the sooner there will be other seeds to take over.
You would still need a database somewhere to provide a URL to torrent mapping, but per
Re:Why don't the editors link to mirrordot? (Score:5, Informative)
Parent
Never! (Score:5, Funny)
I find it highly unlikely that many will manage that :0
rays? (Score:3, Insightful)
Re:rays? (Score:4, Informative)
If you mean in the sense that POV-Ray does, then no, this is very different. It's an "image-based" rendering technique, which means that you create new images using photographs and other such real-world measurements as input. Conventional ray tracing gives you pictures of models built in the computer's memory, which might approximate a real-world object.
The important difference is that you don't have to build a computer model of the geometry you're trying to render. This is both a help because many real-world objects are hard to model accurately in a computer, and a hindrance because you can only render pictures of objects that you actually have in the real world.
Parent
n/t (Score:4, Funny)
Slashdotted already! Google cache here. (Score:4, Informative)
Anyone please mirror the movie?
J.
ARTICLE CONTENTS (Score:5, Informative)
Abstract
We present a novel photographic technique called dual photography, which exploits Helmholtz reciprocity to interchange the lights and cameras in a scene. With a video projector providing structured illumination, reciprocity permits us to generate pictures from the viewpoint of the projector, even though no camera was present at that location. The technique is completely image-based, requiring no knowledge of scene geometry or surface properties, and by its nature automatically includes all transport paths, including shadows, interreflections and caustics. In its simplest form, the technique can be used to take photographs without a camera; we demonstrate this by capturing a photograph using a projector and a photo-resistor. If the photo-resistor is replaced by a camera, we can produce a 4D dataset that allows for relighting with 2D incident illumination. Using an array of cameras we can produce a 6D slice of the 8D reflectance field that allows for relighting with arbitrary light fields. Since an array of cameras can operate in parallel without interference, whereas an array of light sources cannot, dual photography is fundamentally a more efficient way to capture such a 6D dataset than a system based on multiple projectors and one camera. As an example, we show how dual photography can be used to capture and relight scenes.
(a) Conventional photograph of a scene, illuminated by a projector with all its pixels turned on. (b) After measuring the light transport between the projector and the camera using structured illumination, our technique is able to synthesize a photorealistic image from the point of view of the projector. This image has the resolution of the projector and is illuminated by a light source at the position of the camera. The technique can capture subtle illumination effects such as caustics and self-shadowing. Note, for example, how the glass bottle in the primal image (a) appears as the caustic in the dual image (b) and vice-versa. Because we have determined the complete light transport between the projector and camera, it is easy to relight the dual image using a synthetic light source (c) or a light modified by a matte captured later by the same camera (d).
A totally uninformed post! (Score:5, Funny)
This technology proves that there was a third gunman on the grassy knoll. This technique is like what they did in the Matrix, except "backwards." With this technology, any man can find the g-spot. When you look at the videos upside down, you can see into the past.
Re:A totally uninformed post! (Score:5, Funny)
Parent
Re:A totally uninformed post! (Score:4, Insightful)
Unlikely -- you'd still need to get the light source in there somehow...
Parent
A Mirror? (Score:5, Funny)
Another application (Score:5, Interesting)
Other than using electrons instead of light, that's how a scanning electron microscope works. An object is scanned (raster scan) and one or more sensors near the target pick up the reflections to generate an image. In the SEM the image appears as viewed from the scanning electron beam source.
In the optical one mentioned in the article, the light source is a raster scanning projector which lights a target. The image is produced from photodiodes picking up reflected light.
These two systems are very much alike. One uses photons and the other electrons. The end image is generated the same way.
Military applications? (Score:5, Interesting)
It seems like this might have some military applications as a result. Imagine sticking a photo-resistor array under a door or through a window and then getting "viewpoints" from any of the lights in the room. Could aid in target aquisition and elimination.
Not sure how well it works for something like that, but this is a rather impressive (at least to me) research project.
Re:Military applications? (Score:5, Informative)
If you can get to the article, it mentions the light source as a projector. The projector controls the resolution. How it works is a raster scanning video projector lights objects. A photoresistor (in my opinion way too slow. A fast photodiode would be better or photomultiplier tube) picks up the reflected light from the object scanned by the light projector.
A simple street light or the ceiling light in the room will not modulate the light to provide an image signal on a photo sensor slid under a door. On the other hand, if they were doing a video presentation, and the presenter walked between a projector and the screen and you had a photoresistor slid under the door, you would be able to see his arm movements.
You would get the best image when the projector was not showing a slide, but showing a blank screen. Use a CRT projector, not an LCD. LCD's don't raster scan.
Parent
Structured light. (Score:4, Informative)
The 3D part is obtained when you offset the detector and the projector. If I look at a particular point on an object and scan the object with a beam of light, I can get the distance between me and the object as a function of the scanning angle.
Re:Structured light. (Score:5, Informative)
Wow, you remember those?
For those who don't know what they are, it's simply a CRT with a blank raster and a photo detector. Usualy a photomultiplier tube (fast and before photodiodes). The flying spot was simply the bright spot on the CRT. If you put movie film in front of the CRT, the brightness detected by the photodetector was modulated by the film in-between. This was the standard way of showing movies on television in the early days. The flying spot scanner was built into a movie projector with a CRT for the lamp and a photomultiplier tube where the projection lens would go.
In this example, it's a very big flying spot scanner. The lightsource is a projector. (raster scanning light source) The target is a 3D object instead of movie film, and the detector is offset so the 3D object casts shadows to the detector.
The scanned image looks like it would be viewed from the light source with shadows that look like the light source is from the photo detector.
Parent
It's all very impressive, but.. (Score:4, Informative)
All these people are doing, are using the first barcode technique to, take a picture of the scene. Instead of using a laser, an animation of a moving white dot is sent to the projector. The Camera, is then treated like a light sensor, for each point in the animation, the camera is queried for the brightness of the perhaps, brightest dot in it's field of view. Gradually the picture is built up, pixel by pixel, untill, finally a picture is formed in memory. This picture would be from the perspective of the projector.
IANAS, but it looks like reverse 3d rendering... (Score:3, Informative)
this technique works because of the lcd/dlp array in a projector, but i wonder if it can be reproduced if the light source is already a pinpoint(chrismas light, or very small bulb). what happens when the light source is very broad, like that of a computer monitor/ TV? i wonder if this technique could also be used to extrapolate what someone is watching/reading/viewing on screen? taking another stab from a raytracing perspective, i wonder if an environment could be revealed thru image analysis, aka reverse-HDRI?
hats off to the dually photo boys of stanford and cornell... keep up the cool work.
Re:IANAS, but it looks like reverse 3d rendering.. (Score:3, Informative)
Torrent (Score:5, Informative)
http://dload.digitalriviera.com/DualPhotography-p
Second part in 30 minutes !
First torrent I host, I hope it's ok.
Re:Torrent (Score:5, Informative)
Parent
Need more coffee (Score:4, Funny)
Was I the only one that saw that as:
Seeing Around Corners With Dual Pornography.
I need more coffee.
Blame The Slashdot Editors (Score:5, Insightful)
It is the laziness and irresponsibility of the slashdot editors to not provide a bittorrent link.
I am disgusted that slashdot raves about a site/file/mpeg then DDOSs
it so that nobody sees it. This is particularly bad when a hobbyist site is crushed.
Mod me into oblivion, I don't care.
You will never see around the corner, 'cause: (Score:3, Interesting)
2. You should know not only amplitude, but *phase* of the source signal, that means for light that you have to use coherent light source and utilize interference on the receiver.
1 + 2 = holography, so what is new?
(Read the article, but still downloading the movie)
Another mirror... (Score:5, Informative)
Torrent file (Score:4, Informative)
Re:Does it work for... (Score:3, Informative)
Re:OK, that's just cruel... (Score:5, Funny)
Parent
Re:around corners? (Score:5, Informative)
Parent
Re:around corners? (Score:5, Informative)
If you watch the video, the very last demonstration is that of them generating the image of a King (of hearts?) that was not directly visible to the camera. Rather, its face was reflected onto the page of an open book - much more complicated that just, say, a mirror. The cards reflection is not visible in the still image of the book and is only made possible through pixel scanning with the projector.
In sum, they are seeing around a corner and are seeing something the camera could not see (directly).
Parent
Re:around corners? (Score:4, Funny)
Parent
Re:Mirrors (Score:3, Funny)
Re:Homebrew solution possible? I own a LCD project (Score:5, Funny)
No, that would be stalker. Still pretty bad, but not quite up to murder.
Parent
Re:He say you Blade Runner... (Score:3)
Bruce