Slashdot Log In
High-Speed Video Using a Dense Camera Array
Posted by
michael
on Wed Dec 29, 2004 06:20 AM
from the e-pluribus-unum dept.
from the e-pluribus-unum dept.
karvind writes "Researchers at Stanford have demonstrated multi-thousand frame-per-second (fps) video using a dense array of cheap 30fps CMOS image sensors. A benefit of using a camera array to capture high speed video is that we can scale to higher speeds by simply adding more cameras. Even at extremely high frame rates, our array architecture supports continuous streaming to disk from all of the cameras. Now we know where to use 100TB tape drives and what to expect in the next sci-fi movie."
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Interesting study (Score:5, Interesting)
Re:Interesting study (Score:3, Informative)
With these you can get more detail than the shitty webcams without shelling out on high end equipment. This has remarkably few uses, but with t
Re:Interesting study (Score:3, Interesting)
These things are cheap as hell, it's much easier to double them up than produce one of twice the quality.
Logical extension... (Score:2)
Now they are CMOS, instead of plate cameras...
Re:Interesting study (Score:3, Informative)
It is funny that you used the word "atmosphere" but that might be one of the applications: combustion research.
A friend of mine works at General Motors doing combustion chamber research. Basically, with a high-speed camera, he films the combustion in what basically amounts to an engine with a glass block and cylinder head. They currently film at 900fps with an industrial film based camera. This is quite expen
Re:Interesting study (Score:2)
You really think a fossile fuel will be the "next big thing". And what's new about Diesel anyway, it's been around for generations...
Re:Interesting study (Score:2)
Apparently 1500 fps still isn't fast enough. (Score:2)
http://www.gotsheep.com/~hirsch/Photos/DCP_0492_3 2 0.jpg [gotsheep.com]
I've found the digital file but not the film that I scanned it from- blowing up eggs is MUCH more fun.
http://www.gotsheep.com/~hirsch/Photos/EGG_3_crop_ RPD_PPost_lut.jpg [gotsheep.com]
(Slashdot is doing wierd things to the links- so you'll have to remove the %20's it's sticking in in the spaces)
Nothing to see here (Score:3, Funny)
Quite convenient for a story about a slashdotted camera.
Questions (Score:3, Interesting)
How can they be sure that none of the cameras capture the same instant of the action?
Re:Questions (Score:3, Informative)
Same way they sync audio and video in sound studios.
Video track (or a seperate track) carry a pulse carrier. Audio track syncs to that.
Re:Questions (Score:4, Funny)
You know, there might be a reason why those people are at Stanford [stanford.edu].
Parent
Re:Questions (Score:3, Insightful)
Are the CMOS sensors designed for 30fps sensitive enough to capture a picture without having long exposure time? i.e. Can it handle the 1/1000 sec exposure time without a very dark image that is destroyed by the noise floor.
Would the sensor analog circuits have fast enough rise/fall time to have bandwidth(1) for that type of frequence response? The overall system bandwidth & sampling rate lim
Question... (Score:3, Insightful)
But my question is this...
Are there any uses for high speed video capture that existing technologies weren't already well suited for, or is this just a cheaper and more readily available option?
Re:Question... (Score:5, Interesting)
The more I look at this, the more I think they are making life difficult for themselves, and the resultant image quality shows.
Since making my first postings on this discussion, I decided to have a look around at how the professionals handle high speed photography and came up with some nice results.
Theres a company called Photron [photron.com] that have a range of single digital cameras capable of megapixel images at 2000fps.
In their gallery [photron.com], they even have an example of a water filled baloon popping, and tbh it looks a lot better than this multi camera version.
Agreed, this is a way to do it on the cheap, but because of the spatial issues and timing complexities, it may be more trouble than its worth, and may well be wise to buy a camera from the professionals.
Parent
Re:Question... (Score:2, Interesting)
One tradeoff is that these high speed cameras are typically event driven - Once you start them, they record onto local memory (Since there is no way of bursting megapixel*kilohertz => gigabytes/second. With the camera array, it is possible to get a continuous stream. Dunno if it is worth anything to anybody though.
Re:Question... (Score:3, Insightful)
Agreed, this is a way to do it on the cheap, but because of the spatial issues and timing complexities, it may be more trouble than its worth, and may well be wise to buy a camera from the professionals.
First off, that water balloon video, which is 4000fps instead of the ~1600fps camera array video, is really awesome. However, if, for some deranged scientific experiment/research, 4000fps isn't good enough, perhaps you can build an array of 52 professional 4000fps cameras to achieve a whopping 208,000fps
Questions answered in their paper. (Score:3, Interesting)
Yeah, they mention Photron in their paper. As nice as that camera is, it can only store a few seconds at 800x600. The system you are looking at will run till you run out of space. The paper is a well written 320kB pdf and more worth your download time than the movies themselves.
Now, here are a few thoughts of my own. Some of the image quality problems you notice might be a sid
Re:Question... (Score:3, Insightful)
I've always wondered how half-speed video from football games looks so damn good. I assume they're using expensive dou
Possible storage solution (Score:3, Funny)
If you are hard up for disk space for this, may I suggest emailing frames to this free email account [hriders.com]
I know it's a hack, but whatever gets the job done, right??
Next SciFi Movie (Score:2, Funny)
Why? (Score:2)
Possible application (Score:5, Funny)
Sensors (Score:3, Interesting)
Porn for the Aware Consumer (Score:3, Funny)
No less than 1000 fps facials.
Re:Porn for the Aware Consumer (Score:2)
Paralax issue (Score:5, Insightful)
Re:Paralax issue (Score:2, Insightful)
Either way, its still some pretty cool tech.
Re:Paralax issue (Score:4, Funny)
Duh! Because obviously it'd take some kind of super-genius to reconfigurize the franglehum reflectus so as to porta-pride the whoozimotron without disrupting the stratus field generator.
Parent
robotics (Score:2)
..but why bother (Score:3, Insightful)
For the same or less money/effort I have no doubt they could have either bought a purpose-made high-speed cam, or built one using something like This chip [micron.com] from Micron, which costs less than $2K and does 500 full-frame megapixel images per second, faster for partial frames. One neat feature is that it can effectively image individual lines at arbitary places in the frame at 500,000 per second - I'm sure these academic types could do some interesting interpolaty stuff with this to synthesise full-frame-like images at pretty high rates instead of messing with a system that doesn't have any realistic practical use.
Re:..but why bother (Score:2)
Now think what'll happen when they put some of these babies into that beowulfish cluster.. (other than complication of computation).
Too slow .. (Score:5, Funny)
Seen similar about 10 years ago (Score:2)
A more recent application is the "bullet time" developed for "The Matrix" movies.
expect in the next scifi movie? (Score:2)
Additional hi-tech needed (Score:2)
Fancy optics tricks (Score:2)
If they were to channel the optics through a single lens somehow and then divided the light among the many cameras, they'd come up with something much more seamless. I think that would be really REALLY expensive and maybe even impossible. Another possibility would be to create a
Stanford invented motion photography (Score:3, Informative)
millions of frames per second (Score:3, Interesting)
One of the bigger problems, especially with this 'array', though has been noted above : exposure time.
This might be correctible post-shooting, though. As each frame's exposure will overlap the next, whatever is similar in both could be presumed a no-motion area. Gets quite tricky, though.
And of course the array posted about has parallax issues, etc. etc.
Here's a fun high-end-ish camera
http://www.cordin.com/productsie.html
The 510 at 25,000,000 fps for example. Only captures 48 frames, but that should be enough for something fun...
Light travels at ~300,000,000m/s
In the delta between frames*, light should thus travel 12 meters.
Over 48 frames, it should travel 576 meters.
In other words... if you set this camera up, hooked the shutter to a flash so that the flash fires the exact moment the camera starts its run, then you should be able to see the light travel down, say, a hallway.
Better yet...if the flash is short enough, you should see a 'shelled sphere' sort of shape pass through the hallway, and bounced light bounce off the walls to other objects where the direct light from the flash wouldn't reach.
Can't say I've seen any real-life animations of this, though. There's a few temporal raytracers that can do this.
* again: exposure time means there's some blurring. You don't take a picture of a single moment in time. If you did, you would likely get no picture at all as no photon / electron / film-state change would occur to be recorded.
Author's Comments on Camera Arrays (Score:5, Insightful)
First, this work is part of a larger research effort. In the past several years, cameras have become cheap, commodity devices, and you still get more processing power for the buck every year. I designed the Stanford Multiple Camera Array (http://graphics.stanford.edu/projects/array [stanford.edu]) not to be a high-speed camera, but to be a research tool for exploring the potential of large numbers of cheap image sensors and plentiful processing. High-speed video is one example of high-performance imaging using an array of cameras. We have also used our array for synthetic aperture photography, using many cameras to simulate a camera with a very large aperture. Such a camera has a very narrow depth of field, a property we exploit to look through partially occluding foreground objects like foliage. We are interested in view interpolation (Matrix-like effects, but with user control over the virtual camera viewpoint), too. If you want to learn more about the array and these applications, check out the links to our papers and my dissertation on the camera array website.
About the high-speed video work in particular, there are plenty of commercial high-speed cameras that run at higher frame rates than our camera array. If you want high-speed video camera, I recommend buying one of them. Using an array of cheap cameras has its disadvantages. You have to geometrically and radiometrically calibrate the data from all the different sensors, and in our case, we had to deal with the electronic rolling shutter. One benefit of this work for us was developing accurate and automatic (very important for 100 cameras) calibration methods for our array. An interesting property of the camera array approach is that parallel compression reduces the bandwidth so we can stream continuously. By contrast, as frame rate increase, most high-speed cameras are limited to recording durations that will fit in memory at the camera, usually well under one minute. That said, one could certainly design architectures to compress high-speed video in real-time.
What's most interesting to me about the high speed work is combining it with other multiple camera methods. One example is spatiotemporal view interpolation--capturing a bunch of images of a scene from different positions and times, then generating new views from positions and times not in the captured data. Think Matrix again, but with user control over the virtual camera view position and time. While the BulletTime setup from Manex captured one specific space-time camera trajectory, my goal is to capture images in a way that would let us create many different virtual camera paths later on. Traditional view interpolation methods use arrays of cameras synchronized to trigger simultaneously so they can reason about shape of the "frozen" scene, then infer how the scene is moving. In my thesis, I discuss how using the high-speed approach of staggered trigger times increases our temporal sampling resolution (effective frame rate) and can enable simpler interpolation methods. The interpolation algorithm I describe is also exactly the correction needed to eliminate the jitter due to parallax in the high-speed video sequences.
I've described just a few of the applications we've investigated using our camera array, but we hope this is just the tip of the iceberg. We're hard at work on new uses for the cameras, so stay tuned.
Re:Haha! (Score:4, Interesting)
I can imagine this working REALLY well for crash simulation studies where the subject is a greater distance from the camera array.
The baloon popping movie needs quite severe modification to the captured images, and doesn't do much justice.
The effect has already been used in bullet time type fx effects anyway, they used an array of cameras firing in a sequence whilst laid around a subject.
The effect with bullet time is a lot cleaner because the captured images are not expected to be spatially aligned, and instead are made to give the effect of moving the camera around a subject.
Parent
Re:Haha! (Score:2)
It wouldn't matter if it's very "clean", as long as it's faithful to the event (no blurry artifacts and so).
Re:Haha! (Score:2)
Re:only limited usability? (Score:2)
I don't know about you...but this seems pretty fast to me.
Re:only limited usability? (Score:2)
Re:What's the big deal? (Score:3, Informative)
Say you require a camera that can record say 90fps. To a manufacturer of electronic parts, this can be achieved with a little bit of engineering. Basically, take 3 of those 30fps CMOS sensors, pack them together, set a uniform color correct
Re:shutter speed (Score:5, Informative)
The trick is that A runs on 0.000, 1.000, 2.000, etc. while B runs on 0.001, 1.001, 2.001, etc., C is on 0.002, 1.002, 2.002, etc. (units are frames relative to a starting time), and then the frames are sequenced appropriately (ABCABCABC etc.). This gives a very high frame rate while using relatively low-cost sensors - effectively, they're exploiting parallelism as a way to increase the array's effective sampling rate.
Basically, if you have N sensors capable of sampling X times per second each, and are capable of accurately triggering each sensor to a high degree of time accuracy, your effective sampling rate can be NX. Neat trick.
Parent
Re:shutter speed (Score:2)
Re:shutter speed (Score:2)
Recording-wise, two thoughts here: for one, high speed cameras are usually used to capture very short events, so they may only need to store a tenth of a second or a half seconds worth of data. Given that that might only be a gig or two of data, you could quite easily back the CCDs with a RAM buffer capable of holdi
Re:shutter speed (Score:2)