Erasing Objects From Video In Real Time 175
Smoothly interpolating away objects in still pictures is impressive enough, but reader geoffbrecker writes with a stunning demonstration from Germany's Technical University of Ilmenau of on-the-fly erasure of selected objects in video. Quoting: "The effect is achieved by an image synthesizer that reduces the image quality, removes the object, and then increases the image quality back up. This all happens within 40 milliseconds, fast enough that the viewer doesn't notice any delay."
Perfect Application (Score:5, Insightful)
We need this built into our televisions to automagically remove those network logo "bugs" and other crap they have started putting on the screen during the shows.
Re:Perfect Application (Score:4, Insightful)
In reality the networks wil use it to blur out any logo's from companys that do not sponsor the show. F1 cars will be red instead of filled with sponsors.
Re: (Score:2, Insightful)
Somehow I doubt that would work, unless the racing teams could get money some other way. Who would sponsor a car when your logo won't be visible on it? The race organizers would probably require networks that buy rights to air it to not scrub logos.
Re: (Score:3, Insightful)
The race organizers would probably require networks that buy rights to air it to not scrub logos.
I'm pretty sure they already do.
You're right (Score:3, Insightful)
Racing competitions, NFL, etc. own the copyright to the original footage. They're not going to license broadcast of that footage to any television station which threatens their revenue model, unless the station is going to pay so dearly that their previously existing revenue stream looks paltry in comparison.
Even if the TV stations were to put more cash on the table, they still might not agree to such a practice as it gives a large degree of control to a single party (which means more finiancial risk if th
Re: (Score:2)
Re:Perfect Application (Score:5, Interesting)
A simpler version of this has already been used to edit billboards visible in broadcasts of baseball games.
Re: (Score:2)
Or they'll just use it to remove rude gestures, nose rings and nipples, in a visual analogue to profanity bleeps. Welcome to the world of tomorrow.
Re: (Score:2, Insightful)
Re: (Score:2)
Re: (Score:2)
I'm sure that for events like F1, preserving the sponsorship messages is part of the exclusive licencing agreement the broadcaster has to sign up to.
Re: (Score:2)
F1 cars will be red instead of filled with sponsors.
You mean like before everything was invaded by advertisement? Wow, what an improvement!
(yes, slashdot, I can type more than one comment every 20 minutes, I have 10 fingers, not two... argh...)
Re: (Score:2)
I've noticed a steep drop-off of sponsorship in F1 in the past few years. The Ferrari, for example, has a solid red shell behind the cockpit, where it used to be littered with ads.
Hard economic times will do the same thing as this German video tool.
Re: (Score:3, Interesting)
First off, I don't think we'll get control over this on our TVs. The networks aren't gonna let us delete their "bugs".
I'm actually more concerned over something like Running Man where you can't trust the news reports you see because someone selectively tweaked the image to hide/alter the bits they don't want you to see.
Now, of course, the technolo
Re: (Score:2)
They can already do that - most news reports aren't in real time, they can tweak the video before they air it.
Re:Perfect Application (Score:5, Interesting)
Or stripping out the thousands of peaceful middle aged protesters and the hundreds of uninvolved pedestrians being tear gassed to show only the one or two violent people who actually have nothing to do with the protest.
Next up, witnesses will disappear from police video to discredit them in court.
Re: (Score:2)
Re: (Score:2)
The Christian Coalition wants this for Super Bowl half time show boobs.
I want it to remove the entire half time show.
Cool, but probably still has a ways to go. (Score:4, Interesting)
Pretty good, but take note that all the examples where objects sitting on pretty flat colored backgrounds. I'd like to see what happens when you try to remove an object in a complex environment. Like removing a single person standing in a crowd.
Re:Cool, but probably still has a ways to go. (Score:4, Informative)
Take a look at the explanation part of the video. The background texture is tiled. You can see some strange deformation in the regular pattern where the object used to be. Also in the drain example there is a strange crater effect as the camera angle changes.
It seems like smooth colour graduations work well, but patterned backgrounds have more obvious deformations.
Re: (Score:3, Interesting)
All of their samples (except the brick one) use solid/high contrast surfaces that are somewhat evenly lit. Still kind of cool though, but they should have waited until it's more mature to impress us old timer motion graphics guys.
Re: (Score:3, Interesting)
Right. I'm pretty sure the underlying technology is based on Seam Carving , where a continuous background region is collapsed 'seamlessly' (or an object within such a region is removed). This doesn't work so well when the background is discontinuous, so it's not going to remove logos from clothing. It's also not going to work well on live video, since the object to be removed needs to be identified manually before the excision can occur. But it works nicely on prerecorded media. I'm impressed at how we
This is just step one. (Score:3, Interesting)
In addition to dealing with reflections, which I consider just a part of polishing step one, step two will use the position of something in the video as an anchor and substitute the image of something else.
How far off do you think *that* is? I give it two years to the the lab demo with problems.
Re: (Score:3, Interesting)
There were some examples of that in the clip if you watched closely. The removal drain on a pebbly asphalt caused a weird swirly pattern to occur as the camera moved. I expect the same would be true for live attempts at the same. It probably works best on static th
Do we still believe what we see? (Score:5, Insightful)
This has some frightening ramifications for how much we believe video. Videos similar to the ones Wikileaks leaked, or news videos "live" on scene, could be doctored in near enough to real time that we consumers might never know it. Scary.
Re: (Score:2)
The Running Man (Film version) (Score:3, Interesting)
comes to life then?
Its bad enough people believe lines said by comedians are the actual lines of some high profile people, how can we hope that people will care enough to know if the video they are seeing is not edited? Hollywood doesn't need the tech to make movies, maybe to "fix" reality shows, but I figure politics is where the mileage comes in.
Re: (Score:2)
how is that any different than now? how can i tell if a video i see now isn't either edited or staged? the only thing this changes is it makes it possible to do it live.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Anything non-live can be doctored with since decades (including "Videos similar to the ones Wikileaks leaked"). The ability to alter live video in that way however is, AFAIK, new. But as long as you can't verify that any given video broadcasted on TV is actually life, that's more a moot point (see countries with a strict censoring that require "live" video to be delayed for a short time so censoring can happen).
Re: (Score:2)
Yeah, just watch what they do with the top CGI films these days. Parts of Alice in Wonderland (the latest film I watched with CGI) are pretty much indistinguishable from reality, except for the bit where the thing is nigh-on impossible without huge expense and months of prep time for a single shot (like long shafts full of rough edges and trinkets that *could* be dug and prepared, but are realistically going to be graphics).
LotR painted all sorts of stuff in, and Gollum was painted over the top of Andy Serk
Re: (Score:2)
With live video, you don't need to doctor it. You just need to convince the audience that your sound stage is really "on-site".
Re: (Score:3, Interesting)
Re: (Score:3, Informative)
This is only news for real-time feeds. For anything that is not live (and you can verify that it's live! A lot of what you see labeled "live" on TV actually isn't!), assume that the stream has been messed with, already today. Most of the times, it is "artistic" messing - improving picture quality, editing out distracting background content, cleaning up artifacts, etc.
Re: (Score:2)
Re: (Score:3, Informative)
Video can already be manipulated without restoring to high-tech wizardry. There's been plenty of examples of news reporters "on scene" when they're just in front of a blue screen. Cinema (which has much higher resolution, so it is harder to fake) constantly amazes us with simple tricks like flattening the depth of field, rotating the camera to make small inclines look like cliffs, adjusting zoom while moving the camera to distort depth perceptions, etc.
And we aren't even getting close to the easier techni
Re: (Score:2)
Actually, it's not restricted to cinema. Anyone with a half-decent camera can do it. Still or video (especially since modern dSLRs support recording video). Heck, a surprising amount of video these days is captured on nothing more than
Video evidence? (Score:2)
Also, if there is video evidence presented in a courtroom, people should be aware that technology like this exists and it can and will be used.
Re: (Score:3, Funny)
Are you saying the pixels are wrong? Have you seen a lot of shops?
Re: (Score:2)
Also, if there is video evidence presented in a courtroom, people should be aware that technology like this exists and it can and will be used.
And it seems likely an entropy analysis on the resultant images will show up bright problems with the re-created area.
Journalistic Integrity (Score:4, Interesting)
Great - it'll start off by making eyesore real estate disappear from "live coverage," then be required as a precondition for live celebrity interviews (not just makeup to cover that acne), moving on to inconvenient points to the story that would take too much time and effort to explain, then images which might "disturb the children" (number of student bodies in Tienanmen Square?), and finally develop to ubiquitous studio-in-a-cameras such that we'll have little assurance of whether live coverage is fact or fiction.
Of course that's just pessimism speaking. Really I'm looking forward to watching live reports without those obnoxious people waving at their mothers, or holding up witty slogans about taxation.
Re: (Score:3, Insightful)
Here's an optimistic thought — it might make people skeptical of the images they see, which is a useful attitude reagrdless of this technology.
Re: (Score:2)
I sincerely hope you're right, but more likely cognitive dissonance will take over [innovations-report.com] and people will be more likely to believe what they see even if they know it likely to be false.
Don't worry, I'll be as optimistic as you like on a Friday afternoon. The second morning back after a three day weekend still tracks closer to Monday before caffeine for me.
Re: (Score:2)
Communal CCTVs could edit out the hookers, dealers, pimps, garbage and other 'untouristy' stuff on the fly.
Now we just need it miniaturized and built-into our glasses, preferably rose-tinted, to see the world in a whole new light.
Re: (Score:2)
Anyone who uses Journalist and Integrity in the same sentence should be ... (oops I did it too)!
Re: (Score:3, Insightful)
Ed Murrow was a journalist who practiced integrity [wikipedia.org]. There, I did it again. So what was supposed to happen? Perhaps I should be taken in front of a special committee hunting for non-American behavior. Except wait - they disbanded that one after some punk journalist risked his career and took out McCarthy.
I actually think this is a pretty neat idea. I think the thing which concerns me is that, given the current state of affairs, the public will have even less reason to trust the press. If the public do
I thought what I'd do is... (Score:5, Insightful)
... I'd pretend I was one of those deaf-mutes.
Re: (Score:2)
The Ring of Gyges [slashdot.org]
Re:I thought what I'd do is... (Score:4, Interesting)
That phrase was familiar to me, but I wasn't sure where I had seen it... now I remember:
https://secure.wikimedia.org/wikipedia/en/wiki/Laughing_Man_(Ghost_in_the_Shell) [wikimedia.org]
Re: (Score:3, Informative)
Catcher in the Rye you meant right? That classic book? Bill Gate's favorite as well (just throwing that line out for comedy).
Re: (Score:2)
No, I recognized it from that anime. I've only seen a handful of episodes of GinS so I had no idea what was going on story-wise, but that distinctive logo stuck with me.
Overlooking the Wiki article on the book makes me think I was not introduced to it because I went to a Catholic school. It wouldn't have been banned from the library, but it probably wouldn't have been allowed as part of a class curriculum, either.
Re: (Score:2)
It wasn't part of my class curriculum either, and I've not read it. I only found out about it through GitS (actually, GitS:SAC - it is the "Stand Alone Complex" series rather than the film). They do mention that it is from Catcher in the Rye in one episode, but that comes after the phrase has been mentioned a few times. Someone has a rather aged copy and there is a bit of a "these funny people with their ageing paper books" tone, given the setting, but it is acknowledged in a fairly intellectual way.
Re: (Score:2)
Re: (Score:2)
Re:I thought what I'd do is... (Score:4, Informative)
Re: (Score:3, Informative)
He did it before that, though. When he kidnapped Serano from his own home then he was visible to Serano, but not to his guards. As far as the guards were concerned then Serano was just walking out on his own and nothing was unusual. That would probably be more impressive because he did it to lots of people at once, not just a single person who was chasing him as he left a hotel and casually escaped (IIRC).
Re: (Score:2)
Re: (Score:3, Interesting)
Hmmm (Score:2)
I can beat that. I can erase everything in the frame in less than 40ms.
Actually, this is really cool. They could generate hype for it by posting a demo on the web of Episode I with Jar Jar erased. They might be able to stave off Lucas's lawyers by calling it a parody, although in this case it would be more like the original was a parody.
Re: (Score:2)
The Phantom Edit (Score:2)
They could generate hype for it by posting a demo on the web of Episode I with Jar Jar erased.
How would making Jar Jar Binks invisible improve The Phantom Menace? He'd still have all his lines. Better would be to re-edit the film to make Binks only a minor character, and that's called The Phantom Edit [wikipedia.org].
Jar Jar (Score:5, Funny)
Finally, we can restore my childhood memories and eviscerate Jar-Jar from the last batch of Star Wars movies.
Re: (Score:2)
How do you (Score:2)
Re: (Score:2)
Usually by saying "enhance" [youtube.com] to the resident technic freak.
Re: (Score:2)
ob. (Score:2)
ChatRoulette, the PG edition.
Re: (Score:2)
That was actually my first thought - I always imagine that in live broadcast TV events, there are guys sitting at a console with blur-out circles at the ready (I imagine they hire top Quake 3 Arena players for this).
This could automatically blur naughty bits, saving millions of uptight Americans from Janet Jackson boob, and also most of the people on Chatroulette from unwanted dongs (they have an adult section now if you want the dongs).
ET! (Score:2)
Perhaps the next BluRay spec should include this technology so that when you watch ET you can select for the walkie-talkies to be erased and your choice of guns/rpg's etc to appear!
What? (Score:2)
Sorry, I started to call bullshit at the "increase the quality back up" sentence. No-one worth their salt in video processing would ever use such a phrase. This makes it equivalent to Bladerunner-esque infinite zooming and Hollywood-style deblurring to a perfect image.
I'm not saying it's not possible, or even not possible in real-time, but that explanation sends all sorts of warning signals to me. Hell, we know you can do stuff like this because Hollywood does it all the time without having to use physic
Re: (Score:2)
Thwartable (Score:5, Funny)
"Zoom in on the reflection...ENHANCE!"
Not thwarted by reflections. (Score:2)
FTA:
Ummmmm, no. The software reads an image with 2 objects, they only deleted one. Maybe the software can only delete one object at a time now. But that's not being "thwarted by reflections," the mirror has nothing to do with it. The software would behave the same if there actually were 2 objects on the screen.
Re: (Score:2)
And here I thought it was a vampire cellphone and the video was shot from the mirror world.
Obvious... (Score:2)
In Stalin's Soviet Russia...
Not buying it. (Score:2)
How does it know what to draw beneath the replaced object?
Re: (Score:2)
The same way any filter of that kind does, interpolation based on the surrounding pixels. It's the video equivalent of Adobe's filter in Photoshop 5.
Re: (Score:2)
Also, it's the same way the human brain hides the blindspot.
Nomenclature (Score:4, Funny)
We need a name for this process. I suggest "to Jar-Jar." Examples:
They Jar-Jared the cell phone and stapler off the desk.
"Jar-jar the 3-D glasses off the chair."
Al Pacino released the "Actor's cut" of Godfather 3 and Jar-jared himself out of the movie.
I'd like to Jar-jar my ex-girlfriend from my brain.
It was a guy! He Jar-jared his webcam!
Enhance! (Score:2)
Enhance! [youtube.com]
Running Man (Score:2)
So one step away from TV networks manipulating live video for their own / political masters requirements.
You didn't see it, it never happened. The camera never lies - right!?
Suspected limitations (Score:2, Interesting)
1) Objects must be sitting on a consistent(ish) surface with a low rate of change compared to the object. Desk, Chair, Bathroom, Wall, Hubcap, etc.
2) It doesn't handle strong shadows (or they are not showing us it doing so).
3) It makes the greatest amount of mistakes with the shadows anyway.
Please add anything I missed to future posts.
I would like to see it erase a boat from a choppy sea where there are 5-7 waves for the length of the boat as I expect that to be a
Fiction Leads Reality (Again) (Score:2)
Perfect for news networks (Score:2)
Not the first attempt... (Score:2)
Ten years ago, one of my friends, who works on movies' restoration and coloring, told me that they had software that was able to remove moving objects from a scene.
The idea was to use the whole scene to recreate the missing parts.
I also remember an article on compression.ru, with plugins able to remove logos or subtitles:
http://compression.ru/video/logo_removal/index_en.html [compression.ru]
http://compression.ru/video/subtitles_removal/index_en.html [compression.ru]
and even TV ads removal:
http://compression.ru/video/tv_commercial_detector/i [compression.ru]
It's the end of the world as we know it. (Score:2)
Really, no one is going to say it?? (Score:2)
Fine..I will say the obvious. ;)
Never again will you need to see a penis in your porn..unless of course you want to in which case I am not judging you.
Can be done with CCTV with encryption (Score:2)
I can't find the paper at the moment, but I've seen examples of something similar done with a CCTV camera and encryption. It has the advantage of having a fairly fixed view, so it can easily repaint the background in, but the idea was that it recognises and encrypts the imagery of a person in the shot. The CCTV then captures everyone who walks through the shot in case actions need to be reviewed in future (e.g. "did we see a guy matching the vague description of the perpetrator in the area before the crime
reflection still visible of course! (Score:2)
Did anyone else notice the matchbox appeared in the mirror in the video?
AdBlocker! (Score:2)
I would pay for glasses that remove advertisement from the real world. In fact, I'd pay quite a lot.
Ads are one of the "unseen evils". By now we know that the processing and even the filtering out that our brains do take up more of our awareness than we become aware of. Advertisement has been positively linked to road accidents, for example, as it is a distracting factor.
In any big city today, you are literally bombarded by advertisement, and all of it has been designed by psychological warfare...sorry, "ad
6 million jews and 2 clowns (Score:2)
What 6 million jews and 2 clowns? I don't see 6 million jews and two clowns.
Confusion Between Reality and Fiction (Score:2)
http://idle.slashdot.org/article.pl?sid=10/10/12/1328215 [slashdot.org]
Re: (Score:3, Insightful)
Given the nonsense in the Bible I'd suggest he manages it without technology.
Pope who? (Score:2)
Re: (Score:2)
Cheep and easy counter measures (Score:2, Interesting)
Somehow l feel like like I shouldn't be g
Prisoner Zero has escaped... (Score:3, Interesting)
Re: (Score:2)
You can barely seem them nipples.
And these guys are really lookin'!
Besides, this seems like a better way to rob banks and do other nefarious deeds where video surveillance is present. It's no longer necessary to build elaborate mock ups of a vault room or anything, just hire these Germans.
Already possible (Score:2)
Just not possible to do in realtime:
http://www.somethingawful.com/d/comedy-goldmine/make-porn-worksafe.php [somethingawful.com]
http://www.somethingawful.com/d/feature-articles/worksafe-pornmovie.php [somethingawful.com]
(Above links are ONLY TECHNICALLY SFW...)