Forgot your password?
typodupeerror
Graphics Software

Photosynth Team Does It Again 144

Posted by CmdrTaco
from the see-what-i-see dept.
STFS found an update to the Photosynth stories that we already ran. You might remember the amazing photo tourism demos. Well, this new version kicks things up several notches with paths and color correction to more smoothly transition between photos taken in different lighting conditions. As before, this stuff is worth your time. Check it out.
This discussion has been archived. No new comments can be posted.

Photosynth Team Does It Again

Comments Filter:
  • color (Score:4, Interesting)

    by catbertscousin (770186) on Thursday August 14, 2008 @09:06AM (#24597867)
    The color matching section was quite impressive given the wide variety of lighting and color temp in the starting photos; if they wrote their own software to do that, it sure counts as R/D.
    • Re:color (Score:5, Interesting)

      by Gewalt (1200451) on Thursday August 14, 2008 @09:35AM (#24598173)

      The color matching section was quite impressive given the wide variety of lighting and color temp in the starting photos; if they wrote their own software to do that, it sure counts as R/D.

      AFAIK; adobe created the technology first in response to the needs of automation in the pornography industry. It seriously helped alot of "studios" color match the whole set just by having a wizard scan the pics and correct them all.

      • Lol, why does everything have to be for the porn industry. It couldn't possibly have been because it'd be useful for any other group that uses large numbers of photos... Just about any photographer would find it userful... whether it's a wedding or a fashion or a sports photographer. I'm not attacking you personally, there are just so many insane, "well this format won because the porn people picked it" type urban legends that it gets a bit ridiculous after a while.
        • Re:color (Score:5, Interesting)

          by Gewalt (1200451) on Thursday August 14, 2008 @11:26AM (#24599899)
          Actually, I was a rabid Adobe Forum troll when some self-declared porno studios started clamoring for the feature. The other people it would be useful for actually dismissed it, as they did not seem to think they wanted that step in their workflow automated. But once the feature was added, everyone seemed to appreciate it. Of course, adobe is not one to normally listen to and assimilate feedback, especially not from their forums, so that could have just been coincidence.
          • by jfengel (409917)

            Why are porno studios particularly interested in the feature?

            • by Gewalt (1200451)

              Why are porno studios particularly interested in the feature?

              It's an extremely efficient (read: cheap) way of making a rather amateur shoot look like it was done by a professional.

        • by Bazman (4849)

          I guess someone doing colour matching on a porno movie might get a bit distracted. Computers don't have that problem...

  • by Max Romantschuk (132276) <max@romantschuk.fi> on Thursday August 14, 2008 @09:07AM (#24597873) Homepage

    And THIS is why I tend to take huge numbers of photos and never delete any... Technology like this will account for easy geotagging, date I already have in the EXIF data, whereas people can be tagged with face recognition soon enough.

    That done, I'll be able to navigate my tens of thousands of photos by asking for things like photos taken of the kids while outside at the cottage when they were 3 years old.

    Also, remember to backup! :)

    • Re: (Score:1, Funny)

      by Anonymous Coward

      Why not just use a video camera?

      • by ka9dgx (72702) on Thursday August 14, 2008 @11:33AM (#24600023) Homepage Journal
        Because a video camera is nowhere near the quality of a still image, still cameras will win for a number of reasons:
        • Still Camera - less motion blur, if any
        • Bigger sensor - less noise
        • Focus mechanism - an SLR has a much better focusing mechanism
        • Image Compression - almost all video codecs record a stream of images, and do not optimize the quality of an individual frame
        • Exposure time - A still camera can take from 1/8000 second to 5 minute exposures for a single frame, as opposed to a fixed time of about 1/30 second for NTSC
        • Aperture - A still camera can control the aperture to get desired depth of field

        So, those are the ones I can think of off the top of my head.

        • Re: Exposure time (Score:4, Informative)

          by DamnStupidElf (649844) <Fingolfin@linuxmail.org> on Thursday August 14, 2008 @07:09PM (#24607973)

          NTSC is worse than you described; you have two 1/60th second exposures interlaced together. Utterly worthless for still frames.

          Once progressive HD video cameras become cheap, then video will suck slightly less for the average family archive.

        • Re: (Score:2, Informative)

          If a poor photographer uses a cheap video camera - what can be acheived using image processing? In theory signal processing of a long enough video could:
          • Remove blur using deconvolution - multiple frames can be used to reduce deconvolution noise sensitivity and multiple frames help calculate point spread functions. Of couse motion blur is also avoided in video cameras by using Image Stabilisation (adaptive optics are becoming common even in low end cameras - eg. TX-1).
          • Multiple frames can be used to remove
    • It looks like taking a video would be easier. That way, you wouldn't have to spend time stringing all the stills to together - if I understood correctly.
      • by Max Romantschuk (132276) <max@romantschuk.fi> on Thursday August 14, 2008 @09:46AM (#24598305) Homepage

        It looks like taking a video would be easier.

        Depending on what you are trying to do... My original point was that technology like this will make it possible to navigate the swamps of data we're accumulating.

        I like having a lot of family photos, but traditional albums won't do when we have literally thousands of them. Stuff like this can make it possible to easily call up photos based on suitable criteria. Like I said we need other parts to, like face recognition, but summing it all up we'll eventually have a feasible way to navigate a huge amount of photographic data.

      • by Tim C (15259)

        Well the tech demo is using photos taken by arbitrary people. While it could be used to similar effect on your own photo collection (if you take enough photos from enough positions), the real power would seem to be when it's used on a large collection of user-submitted photos, or if its fed the contents of Flickr, etc.

    • Re: (Score:3, Informative)

      Geotagging is not that hard nowadays, assuming you have a device capable of creating a gps log and are using a digital camera (timestamps). Take the log, load it into gpicsync [google.com] and let the program tag your photos for you. Just make sure the gps device and your camera have their clocks synchronized. I'm still waiting for a decent way to browse photos on a map, though - pretty much what you're looking forward to, I guess. Picasa lets you view geotagged photos in albums in google earth, but it's not much more t
      • by 3dr (169908)

        The Windows program TopoFusion (www.topofusion.com) will merge photos with GPS logs, and place them on a map. As you said, make sure the camera is synced to GPS time.

        I've used TopoFusion for a few years and have been pretty happy with it (primarily used for cycling logs and making rough maps of trails).

    • 4D support? (Score:3, Insightful)

      by Roger W Moore (538166)

      That done, I'll be able to navigate my tens of thousands of photos by asking for things like photos taken of the kids while outside at the cottage when they were 3 years old.

      That raises an interesting concept. Could they do a 4D orbit? For example identify pictures of your kids at different ages and then you could watch them grow up in front of your eyes. Or watch how a city street changes over a decade? That would be really interesting...shame it will probably only every be available for Windows.

  • by pz (113803) on Thursday August 14, 2008 @09:07AM (#24597883) Journal

    Very cool stuff! Does anyone know (are any of the project team members here?) how much foreknowledge of the object being orbited that is required?

    For example, is a 3D wireframe model necessary?

    Is a filtering of the photos necessary to ensure that they are all of the same subject?

    What level of pre-processing is required on the photos, either automated, or manual?

    How well does the system fare when the object being photographed isn't absolutely static? A drawbridge, for example, changes shape. Or Niagara Falls. Or a flag. Or a single person.

    Anyone know?

    • Re: (Score:1, Funny)

      by larry bagina (561269)
      more importantly, does it have an anti-goatse filter?
    • by Minwee (522556) <dcr@neverwhen.org> on Thursday August 14, 2008 @09:40AM (#24598229) Homepage
      This is described in their SIGGRAPH paper, which was prominently linked from the article.

      It's a bit dense and involves some cross references, but here's a part which may answer some of your questions. For more detail you oculd always read the paper yourself.

      We use our previously developed structure from motion system to recover the camera parameters for each photograph along with a sparse point cloud [Snavely et al. 2006]. The system detects SIFT features in each of the input photos [Lowe 2004], matches features between all pairs of photos, and finally uses the matches to recover the camera positions, orientations, and focal lengths, along with a sparse set of 3D points. For efficiency, we run this system on a subset of the photos for each collection, then use pose estimation techniques to register the remainder of the photos. A more prin- cipled approach to reconstructing large image sets is described in [Snavely et al. 2008].

    • by dave420 (699308) on Thursday August 14, 2008 @10:11AM (#24598645)
      You just give it the photos - it figures out the rest. It works by stitching them together in 3D, so if there is a photo of one part of the subject that is not overlapped by one other, the photo won't be part of the finished "model". If you download the old demo, you can see the Yosemite demo, which shows what happens with movement (hikers climbing a mountain). If it can match up most of a scene in an image, the image can still be used. I'm sure it'll only get better. Another great example is in the old demo, where they simply searched Flickr for "Notre Dame", and then constructed the entire cathedral. It picked up a photo of a poster in someone's house, and seamlessly integrated it into the model. It recognised what it was from, and where on the cathedral it was positioned, and reflected that by putting that image exactly where it should be in the finished "model". Of course this is just stuff I've gleaned from watching the demo videos, using the demo, and reading as much as I can about it, so I might be wrong on some of it, but that was the impression I got. If I'm far off, I'd appreciate being put right, as this technology is nothing short of stunning.
  • fascinating (Score:2, Informative)

    by thedonger (1317951)

    Science fiction and VR have primed me to believe someday we would all be walking around some imaginary digital world (oh wait, WoW anyone?), but this is "virtualization" of the real world. Like Google street view on crack. I am simultaneously in awe of the technological achievement and embarrassed that my life in computers hasn't yet created anything so cool.

    I, for one, welcome our new PhotoSynth overlords.

    • I can't wait for the ability to do a search and replace of all the images of women replacing them with naked Playboy playmate pictures. Then again, it would be embarrassing to walk into a church and be staring a Nun up and down with a woody.
      • You will probably not find any nuns with woodies in Playboy, though they did feature Caroline Cossey. Either your source material's suspect, or you've got the nuns and priests mixed up.

    • Re: (Score:2, Interesting)

      by Tryfen (216209)

      Read The Light of Other Days [wikipedia.org] by Arthur C Clarke.

  • Security (Score:5, Interesting)

    by robvangelder (472838) on Thursday August 14, 2008 @09:21AM (#24598013)

    I was on an ocean cruise recently, and a little girl was lost... Ship's Security were looking for her.

    I later heard she had been found, and as I walked back to my cabin I thought of this software.

    Every corridor of the ship has cameras.

    The parent could recall the last time she was with the child. An operator could then fly through a 3d map of the ship, from that point in time, with recorded video overlaid, following the girl in fast-forward until the current time was reached.

    The flying would be like spectators do in first-person-shooter type computer games.
    An observer could even be automatically tethered to the missing person.

    • The husband could recall the last time he was with his wife. An operator could then fly through a 3d map of the ship, from that point in time, with recorded video overlaid, following the woman in fast-forward until her new amore was reached.

      Ultimate stalker/invasive state tool!

    • Convoluted (Score:5, Funny)

      by CmdrGravy (645153) on Thursday August 14, 2008 @09:43AM (#24598265) Homepage

      Erm, isn't that a bit of a long winded complicated way of doing things ? I mean sure, Computer could do that for you but why not just ask instead ?

      "Computer, where is " and that would be that. I mean typically she'd be stowed away in the engine room re-configuring the sensor array for some nefarious purpose but that's just kids nowadays I guess.

    • imagine the salivations of the UK security forces.

      there is a book, 'lacey and his friends' which contains a few short stories about a society with such abilities...

      not pleasant.

    • by houghi (78078)

      Forget the ship, use all of London and any other place with CCTV. Orwell was an optimist.

    • by ozbird (127571)
      An observer could even be automatically tethered to the missing person.

      Or you could just tether the kid to the parent with a kiddy leash.
  • by MobyDisk (75490)

    I've seen some of these articles about Photosynth, and there seems to be a lot of hype. But... I don't get it.

    I see that Photosynth can glue a series of images together so that you can zoom into and move around a scene and get an epileptic-seizure of correlated viewpoints. This group seems to have made a virtual walk-through using this. But I am unclear:
    1) What is the point
    2) What is the breakthrough

    As for #1, Photosynth is ugly. I would much rather have a few good quality same-lighting photos to look a

    • by Anonymous Coward on Thursday August 14, 2008 @09:34AM (#24598169)
      If this was an OSS project, your post would have been rated "flamebait".
    • by Anonymous Coward on Thursday August 14, 2008 @09:40AM (#24598227)

      It needs neither input of coordinates or input of a rough 3d layout. It generates its own 3d model by analyzing the photographs programatically, you do not even need to tell the program they were taking in the same area. The photographs are then automatically applied to the generated 3d model and finally it lets you move freely in the generated 3d world selecting the best photo matching your current viewpoint while applying perspective remapping, color correction and lens correction.

      • by MobyDisk (75490)

        okay, then I am truly amazed. I guess it just seems so far-fetched that I didn't think it was really doing that.

        Even a human brain can't do that. We can't look at an object and determine it's 3D spatial details without additional information. For one thing, we use stereo vision. Then we use our knowledge of scale (cars are about that big, and people are about that big) and light (sun is over there, light bulb is over there). But a computer doesn't know those things.

        I figured this might be possible if i

    • by Jellybob (597204)

      If it's the same project I think it is, this can do it all using image recognition - correlating photos that appear to be of the same location, and then stitching them together.

      It takes a crap load of processor time to do it, but it's largely a hands off process.

    • by hkz (1266066) on Thursday August 14, 2008 @09:42AM (#24598255)

      From what I took away from the original demo, they were doing everything algorithmically. The original demo showed a wireframe of the Notre Dame generated completely from amateur pictures, then overlaid with those same pictures to give it texture. So yes, it is quite impressive. I'd be surprised if Google wasn't doing anything similar for Google Maps though.

    • by dave420 (699308) on Thursday August 14, 2008 @10:14AM (#24598675)
      Because you can, say, search Flickr for a landmark, get the images, run them through this, then you can navigate through the space in 3D, looking at high-res imagery of the subject, from all available angles, without having to previously know anything about the subject. Even the system doesn't need to know anything about the subject, it only needs the photos. It is ENTIRELY automatic, using only images. If you look at the old Notre Dame demo, you can see that it even correctly inserted a photo of a poster of Notre Dame into the 3D model, in the exactly-correct position. 100% automatic. That's the breakthrough.
    • by Catil (1063380) *
      In theory, giving the huge amount of photographs available online, probably already covering every public inch of the populated world, a program capable of automatically crawling, indexing, glueing the images together and autocompleting their geotags could render a fully accessible 3D-'streetview' of the world without any human interaction. Although a final implementation might still be a few years away, this project is most certainly a milestone on the way to achieve it.
    • by will_die (586523)
      It does not need input other then a massive amount of photos. The program does all the piecing together and building of the 3D layout. If you goto the microsoft site on it they have more details. For the video all they did was get a bunch of photos from different people of the same object and feed them into the program and you saw what they got.
      As for the bad part, it takes days and a powerful computer for anything beyond the very basic set of pictures.
    • by jebrew (1101907)

      Can someone explain this to me and why I should be interested?

      If it works as I believe it works (i.e. the stitching is automagic), then tagging two photos with GPS coordinates should be sufficient to tag all of the photos with fairly accurate (two so you have orientation, or just one with direction as well as location).

      Depending on what you want to do with this, you may or may not be excited. I've got a lot of photos that simply do not capture the scale of some of the things I've seen. Not that the photos are bad, just that the medium they're expressed through (i.e.

    • I work in an architectural firm. We do a lot of home/commercials remodels. This involves going out to the site and getting measurements of the existing building, and taking lots and lots of pictures. Inside and out. While working on the project our draftsmen page through these pictures one at a time looking for this or that specific detail. As such I try to take the pictures in a specific order, but there's always those minor details you go back to and photograph out of sync. I can tell you that we'd much r
  • "Error establishing a database connection"
    Nice view indeed!
  • Page Error (Score:1, Redundant)

    by Jason Levine (196982)

    The page says: "Error establishing a database connection"

    I'm not too impressed if that's what Photosynth can do. ;-)

  • Video (Score:5, Informative)

    by c_g_hills (110430) <chaz@@@chaz6...com> on Thursday August 14, 2008 @09:35AM (#24598175) Homepage Journal

    Obligatory link to the youtube video [youtube.com] (not a rickroll, I promise!)

    Thanks, Network Mirror!

  • Is anyone else a little underwhelmed by their so called 'demo'? It seems like the program does not contain any ov the technologies that are being 'developed' at microsoft, and is instead a hard coded 'interactive movie' showing how this 'new technology' is supposed to look, if they ever get it working.

    Most likely this barley interactive photosynth demo will be used to secure patents on '3D Virtual Reconstruction', and the final product will ship with no such features, instead being marketed as an alterna
    • by yuggler (521092)
      I'm just guessing/thinking aloud here. There are two parts to the Photosynth-process. First the computer (allegedly, if you're into the Microsoft conspiracies...) analyses all of the photos and gives them a location. This is the processor-intensive part, that takes both time and processing power. The second part is the viewing. When you already know the position of the photographs it's a small(er) feat to present them in a pleasing way. So I believe that PhotoSynth is real. The reason that you can't upl
    • by zehaeva (1136559)

      Blaise Aguera y Arcas on Photosynth [ted.com]

      the book thing, i don't think can easily be faked by flash, nor the gigapixel resolution image nor the really neat zooming and zooming and zooming into stuff.

      really what photosynth, and by extension seadragon which is what photosynth is built on, promises us is a way to semantically link all those photo's on the web together, to build up those real places into virtual places with little or nor human intervention.

      this should enrich the web in a way we are only just begi

    • by dave420 (699308)
      Why don't you see for yourself [live.com]. It works by intelligently looking at photos and constructing a 3D model out of the photos, entirely automatically. You can't load your own photos yet, as it's still in beta. The demo is the output of their previous tests, as it has not yet been updated with the latest version. And no, it can't be easily faked with flash, as this is 100% automatic, and faking in flash would require a massive amount of work by hundreds of people.
  • The Photosynth technology preview runs only on Windows XP SP2 and Windows Vista. nuff said.
    • by TheSeer2 (949925)

      They've made an effort to make it run on versions of Firefox up to 3.0 (PS was out before FF3.0, so don't think it of it as a "they did it just to say they did it" thing).

      I think it's just more of a they haven't got to it yet as opposed to sheer malice. Although they'll probably aim for FF3.0 compatibility before Linux.

      Although, I'll admit... there's a reasonable chance there won't be a Linux version until this thing is a lot more mature (i.e. your own collections can be formed... but that's when it'll be i

    • by dave420 (699308)
      It's still in beta. Eventually it will be running in Silverlight, which runs on the OS of your choice. And Linux once Moonlight is finished.
  • by replicant108 (690832) on Thursday August 14, 2008 @09:52AM (#24598401) Journal

    There was some discussion recently about the possibility of building an open source photosynth - and creating an 'open voxel space' map of the planet.

    Anyone know if there's been any progress on this?

    http://lists.burri.to/pipermail/geowanking/2008-June/005373.html [burri.to]

  • by 6Yankee (597075)

    Crashed Firefox (3).

    Twice.

  • by Fri13 (963421)

    Everytime when I hear about photosynth, I just remember this http://www.sandcodex.com/ [sandcodex.com]

    But it is nice that Microsoft is understanding the power of this technology and develop it more. But it is not nice that technology actually went to Microsoft so it is patented..

  • What I found more intresting technology than Photosynth http://www.youtube.com/watch?v=556FvXHLtAo [youtube.com]

  • Honestly, this reminds me of DTED and orthorectification techniques used in the satellite imaging world. Just now applied to tagged photos at the consumer level.

    When building a virtual Earth using satellite imaging [fas.org] you basically use the same techniques.

    Great idea for the 'consumer', but nothing revolutionary, just exploiting the same tech that's been around for 10ys.

  • But... (Score:2, Funny)

    by WormSlayer (691389)

    Could it be used to piece together a massively high resolution, totally nude, 3d model of a famous celebrity from the millions of event photo's and nip slips? ;)

  • Keep the Faith (Score:2, Insightful)

    Ok folks, don't worry!

    Just keep chanting the mantra that Microsoft never innovates anything and everything will be ok.

    I'm sure there will be a linux port of this soon and then we can all go back to complaining about how Microsoft copies everything from Apple.

  • by lpq (583377)

    Wasn't too friendly to my browser.
    Even trying setting a downloading it & installing; the content wouldn't activate.

Be sociable. Speak to the person next to you in the unemployment line tomorrow.

Working...