Forgot your password?
typodupeerror
Graphics Technology

Rendering Synthetic Objects Into Old Photographs 134

Posted by timothy
from the improve-on-reality dept.
First time accepted submitter IDarkISwordI writes "A new abstract headed to SIGGRAPH Asia 2011 provides a method for rapid execution of computer graphics, synthesized into photographs with accurate lighting and physics based on limited input from a user and interpretation by their code." The results are impressive; hard to watch the video demo (on linked page) without boggling.
This discussion has been archived. No new comments can be posted.

Rendering Synthetic Objects Into Old Photographs

Comments Filter:
  • by Animats (122034) on Monday October 24, 2011 @02:25AM (#37814588) Homepage

    This is going to be very useful for real estate sales. No need to move furniture into an empty house for the pictures.

    • by Anonymous Coward

      http://www.news.com.au/money/property/beware-of-real-estate-agents-latest-trick/story-e6frfmd0-1226157834885

    • by SpazmodeusG (1334705) on Monday October 24, 2011 @05:11AM (#37815046)

      No problem. Just use context aware fill [youtube.com] to remove the furniture from the image.

      • OTOH they might use virtual furniture to hide problematic things (like wet spots on the wall) behind them.

        • In one of those pics they hid a fireplace. That's the level of intelligence I expect in the Real Estate industry...

  • by No, I am Spratacus! (2281684) on Monday October 24, 2011 @02:25AM (#37814592)

    Especially getting the lighting and the shadows to fit the rest of the image.

    • Unfortunately there doesn't seem to be any software available for download, only the research paper. (If it's there, and I missed it, please let me know.) Hopefully someone will make a plugin for this, so we can use it in GIMP, PS, etc..

      • Re: (Score:5, Funny)

        by NFN_NLN (633283) on Monday October 24, 2011 @03:04AM (#37814726)

        Unfortunately there doesn't seem to be any software available for download, only the research paper.

        The release is delayed because the software is limited to only a few useful objects at the moment: Buddha Statue, Dragon Statue, Pool Table and Dead Hooker.
         

        • Re: (Score:5, Funny)

          by Anonymous Coward on Monday October 24, 2011 @04:04AM (#37814864)

          Unfortunately there doesn't seem to be any software available for download, only the research paper.

          The release is delayed because the software is limited to only a few useful objects at the moment: Buddha Statue, Dragon Statue, Pool Table and Dead Hooker.

          But I need software that can *remove* dead hookers from photos! That's the problem with academia, totally disconnected with the needs of the real world.

          • by daid303 (843777)

            No, no, no. No need for that.

            Just add the dead hooker to a lot of photos, see, that's not a real dead hooker in my photo's, it's added with a computer, see, here is the president with the same dead hooker! Instead of trying to hide the proof, invalidate the proof.

            • by Dunbal (464142) *
              That's fine so long as you have the mesh and textures for the dead hooker... I guess you could always add a different dead hooker though to a bunch of photos. The president, the judge, the prosecutor, etc.
            • by fritish (1630461)

              "I've never seen so many dead hookers in all my life!"

              "Lord knows I have..."

            • Wait til someone combines random Facebook pics with this plugin.
            • With some Presidents, it would not even be all that unreasonable to have caught him on film with the same dead hooker as everyone else...
          • That capability was added [slashdot.org] to both Gimp and Photoshop shortly after a similar presentation at SigGraph several years back. Although if my memory serves me correctly, it made it into Gimp first.
      • Often software releases for submission to SIGGRAPH, don't appear until the following year. The number of individuals working on the software is quite limited and it may be too buggy still for release. Another possibility, and very unfortunate if so, is if they intend to market this to an interested buyer, which would likely be MANY.

        • I've only skimmed the PDF (it's way above my level anyway) but I didn't see any notes about license or copyright. It looks pretty detailed... perhaps enough to allow an open-source project to replicate the software. That would be cool.

        • by lennier1 (264730)

          Equal opportunity. Just sell licenses instead of selling the whole package to a single buyer.

      • by i8degrees (410294)

        Resynthesizer [logarithmic.net], a plug-in for GIMP. The author also has details of the algorithm used [logarithmic.net] published in his thesis paper.

    • posting to undo mod.

      signed,
      dumbass.

  • I love it when slashdot news do that to me
  • ... the next time someone wants to frame someone else for murder ;P

  • Too real (Score:5, Interesting)

    by Wolfling1 (1808594) on Monday October 24, 2011 @02:49AM (#37814674) Journal
    Well, that pretty well wraps it up for anyone trying to prove anything supernatural or extra-terrestrial on earth. Who would ever believe any video evidence now?
    • Good point. I suspect there will be ways to detect artifacts of such changes, but those could probably be obscured by converting to a low-res format. Grainy-smudgy video has always been the friend of woo-woo purveyors.

      • by AJH16 (940784)

        True, but there are fewer and fewer reasons to release grainy video now. Many people have HD capable video camera's on their cellphones. There is no reason for video to be grainy anymore, so low quality video should be the first warning sign that someone may be trying to hide something.

    • by msobkow (48369)

      You mean people believe pictures without wondering whether it's photoshopped?

      True, this is much more advanced technology and seems to be amazingly effective, but a good photoshop editor has been able to fool the public for quite a long time now.

      • Re:Too real (Score:5, Informative)

        by adolf (21054) <flodadolf@gmail.com> on Monday October 24, 2011 @03:08AM (#37814738) Journal

        True, this is much more advanced technology and seems to be amazingly effective, but a good photoshop editor has been able to fool the public for quite a long time now.

        The best part about this tech is that it does not require a "good" photoshop editor to sort out the light paths and shadows/reflections/etc manually, but just a person willing to graphically describe the scene using a GUI. After that, arbitrary 3D objects can be more-or-less added arbitrarily with uncanny realism.

        This includes, perhaps unfortunately, realtors.

        (And to the English Nazi(s) reading this: "graphically" and "GUI" are not redundant terms in this context.)

    • Why not? We believe Gadaffi's dead.

    • by Ihmhi (1206036)

      Maybe if it was live? I don't think this sort of stuff can be done in real-time.

      Star Trek DS9 had something interesting with Cardassian data technology. Data was stored on crystal "rods", like their version of a flash drive. But there was a "write once" rod that can't be altered after data is finished being written to it. Perhaps we need something like this for verified reporting/journalism.

      • by swalve (1980968)
        Not a bad idea. Probably will also have to have GPS data burnt into the photograph too.
      • by Culture20 (968837)

        Star Trek DS9 had something interesting with Cardassian data technology. Data was stored on crystal "rods", like their version of a flash drive. But there was a "write once" rod that can't be altered after data is finished being written to it. Perhaps we need something like this for verified reporting/journalism.

        If you can read the device, you can alter the data and store it to another write-once device. You need a write-once, read-never device. I suggest /dev/null.

      • by Kyont (145761)

        I don't think we're that far off from having the processing power to do this in real time, for simple things at least. I know it's a different technique, but we already have real time line markers painted on the field for (American) football broadcasts, and they even look consistent from multiple camera angles.

        I read a sci-fi story a long time ago (early 80s?) in which one of the plot points was hackers (or whatever) hijacking of the outgoing TV signal of a person announcing election results, of some kind

  • by zebadee (551743) on Monday October 24, 2011 @02:53AM (#37814682) Homepage
    Right at the end of the video. Now you can have a weeping angel moving through your very own lounge!
    • They should have had a TARDIS materialize. That would have been awesome.
    • by Agripa (139780)

      Could a weeping angel be used as a Turing test? Or would a camera recording video also lock one in place? What about a long exposure camera?

      • but any thing that holds the image of a angel becomes one as i recall. they had a video of on in a locked storage container in one episode the image became the angel and infect emily ponds mind. so recording one would probably be bad.
  • If the manual touch-up bits could be automated, this is just about everything augmented reality is supposed to be.
    • If you have a moving camera, you can already compare neighbouring frames to build a perspective model of the 3d geometry. So yeah, with a fast computer and a wobbling camera, this could be done without human input. Sorry, rotoscopers ;-)
  • This photo http://www.mitre.org/about/photo_archives/photos/low_res/whirlwind_f5001.jpg [mitre.org] is begging for a larger than life teapot.
  • Finally! Now we know how Dude Perfect [youtube.com] makes all those shots!
    • Notice how every time they have an audience in the shot they look bored out of their minds until it goes in? That's because they've watched him miss about 300 times already.
  • The shadows may look "better" but the shade and reflected light on the placed subject still needs work. The palette of the placed object also seems to lack white balancing to the picture.

    As long as someone tells you there's fake in the picture, you can still tell what the fake bit is, without using a computer to spot anomalies. It's cool to see that tools get this powerful, but it's not good enough to fake any sharp observer yet, let alone a decent forensic study with computer aid.
  • The communists under Stalin were "fixing" photographs to remove undesirable people for a long time, many years before electronic computers and graphics were invented. Of course, the undesirable people were removed from real life as well...

    • Re: (Score:3, Funny)

      by biodata (1981610)
      In Soviet Russia pictures shop you.
    • by Nidi62 (1525137)

      The communists under Stalin were "fixing" photographs to remove undesirable people for a long time, many years before electronic computers and graphics were invented.

      Even after Stalin's death, they were removing undesirable people from photographs and movies. Except this time, they were removing Stalin. I have seen one case where, in a video of Stalin walking with (I believe) Lenin, surrounded by Soviet military officers, they superimposed the silhouette of a Soviet officer over Stalins face, facing Lenin and away from the camera.

  • Modern-day Yezhovs [wikipedia.org] already tremble in fear. With this technology, Syria, China (or soon the US) will be able to disappear people by millions! :p

  • by dotancohen (1015143) on Monday October 24, 2011 @04:23AM (#37814932) Homepage

    ...will have a field day with this. Please, someone keep him away from whatever _is_ left of the original Star Wars film!

    Didn't someone once suggest that we refer to these techniques as lucassizing?

    • by daid303 (843777)

      ...will have a field day with this. Please, someone keep him away from whatever _is_ left of the original Star Wars film!

      How cool, now we can add JarJar to EP4-6. A good idea, me-sa thinks?

    • Just submitted it to the Urban Dictionary, and attributed it to "A. Miles Davis" (I copied the definition from his Google Plus post): lucasize Verb 1. To take a creation and modify it in ways that make sense to you, but confound everyone else. 2. To retcon a work heavily, claiming prior lack of ability as to the long delay in original product versus retcon(s). The real estate salesmen were enthusiastic about the new computer software which let them lucasize fake furniture into real photos of empty rooms.
  • Combine the software and techniques in this article, with the software and techniques in this video [youtube.com], and you've got some endlessly useful software.
  • Where can I download the software? Is it open source? I am afraid it won't be made available, and if it does it will be in a form where you need a few days to get it running. It seems that these kind of solutions developed in the academic world are often sold to commercial companies or made into commercial products.
  • by damburger (981828) on Monday October 24, 2011 @05:32AM (#37815092)

    So, peak oil arrives, there is a superflu pandemic, 99942 Apophis impacts and blocks out the Sun, etc. etc. we all die.

    ...then, centuries later, technological civilisation reemerges, and starts analysing data storage devices they dig up. Most of them are unreadable, but they do get fragments of data with which they can start to piece together what happened before The Event.

    And what do they find? Pictures of people listening to iPods at the Battle of Stalingrad and Asimo raising the flag at Iwo Jima.

  • by Just Brew It! (636086) on Monday October 24, 2011 @06:02AM (#37815156)
    Once we've also got the ability to render realistic 3D models of real people we're going to be in big trouble.
    • That exists already to a degree. It just hasn't caught on yet. As well there is a slightly uncanny valley still.

      Tron 2.0 by Anonymous below. Here is a YouTube video with a similar thing: http://www.youtube.com/watch?v=FrI5AHYADRg [youtube.com] .

    • Once we've also got the ability to render realistic 3D models of real people we're going to be in big trouble.

      Just imagine all the porn!

  • by john82 (68332) on Monday October 24, 2011 @07:20AM (#37815378)

    Given that this approach is able to account for light sources, geometry (perspective) and physical objects in an original image, it should also be able to remove objects and allow for realistic rendering of that loss. Combine that with the capability described in the proposal and the use of photographs as evidence at trial may soon be inadmissible. Or at the very least, a legal team could reasonably claim that a photo had been doctored (whether true or not) and therefore render such evidence unusable by the prosecution.

    • by swb (14022)

      Maybe there will be a new class of "Licensed & Bonded Reference Photographers" who work independent of any single entity and whose only job is to produce reference photographs which are cryptographically signed by the camera and the photographer at the time they are taken.

      The "client" gets a copy of the photo and the cryptographic signature which they can use to verify that the photo is the "original" and not an altered version.

      The cameras would have to be smart, perhaps with the photographer's public k

      • That's a really interesting idea; I hope you get modded up. I bet police departments would want something similar to this for evidence chain of custody, if they don't use a similar system already.
  • Surprised nobody's mentioned how this could affect unscrupulous media outlets and court cases.

  • Instead of recording just the actors live and having to rely on CG to recreate everything else, with this we could film all physical objects live, and decorate it with CG, like an updated form of rotoscoping [wikipedia.org]. I, for one, would welcome the demise of green-screen films. CG is still unconvincing, no matter how much money they keep spending on it. I still notice it. every. single. time.

  • like dragons and Buddha. (Joke. I know its at Siggraph Asia.)

  • This is pretty cool but I'm waiting for version two when they extend it to work on video background plates. Shouldn't be that difficult because tracking is a well studied problem.

  • Perhaps I'm missing something but, for a while now I have been doing this in 3d Studio with HDRI lighting (you would have to make a synthetic HDRI light probe in this case) and render the whole thing in V-Ray. Very realistic.

Federal grants are offered for... research into the recreation potential of interplanetary space travel for the culturally disadvantaged.

Working...