Adobe Previews AI Upscaling To Make Old, Fuzzy Videos and GIFs Look Fresh (theverge.com) 42
Adobe has developed an experimental AI-powered upscaling tool that greatly improves the quality of low-resolution GIFs and video footage. From a report: This isn't a fully-fledged app or feature yet, and it's not yet available for beta testing, but if the demonstrations seen by The Verge are anything to go by then it has some serious potential. Adobe's "Project Res-Up" uses diffusion-based upsampling technology (a class of generative AI that generates new data based on the data it's trained on) to increase video resolution while simultaneously improving sharpness and detail.
In a side-by-side comparison that shows how the tool can upscale video resolution, Adobe took a clip from The Red House (1947) and upscaled it from 480 x 360 to 1280 x 960, increasing the total pixel count by 675 percent. The resulting footage was much sharper, with the AI removing most of the blurriness and even adding in new details like hair strands and highlights. The results still carried a slightly unnatural look (as many AI video and images do) but given the low initial video quality, it's still an impressive leap compared to the upscaling on Nvidia's TV Shield or Microsoft's Video Super Resolution.
In a side-by-side comparison that shows how the tool can upscale video resolution, Adobe took a clip from The Red House (1947) and upscaled it from 480 x 360 to 1280 x 960, increasing the total pixel count by 675 percent. The resulting footage was much sharper, with the AI removing most of the blurriness and even adding in new details like hair strands and highlights. The results still carried a slightly unnatural look (as many AI video and images do) but given the low initial video quality, it's still an impressive leap compared to the upscaling on Nvidia's TV Shield or Microsoft's Video Super Resolution.
Wonder how it fares... (Score:2)
With close ups of women in the 60s, which were generally blurred a bit to "enhance" the shot.
This is an area that AI has been very good at. I'm frankly surprised that this wasn't an announcement much longer ago.
Its just making up stuff (Score:3)
This is real life , not CSI. You can't recover data that isn't there to start with so its simply make a best guess. A human given enough time could do exactly the same but it still wouldn't be the same as if the original had been done in hi def.
Re: (Score:3)
For forensic purposes (like recognizing a specific face), no.
Re: (Score:2)
"is about the same as re-staging the shoot / recording "
Not really. If for example there was a button on someones coat that at a far enough distance wasn't resolved properly at low def it would instead come out as a smear of colour averaged pixels around its location. The AI has no way of knowing if that smear is actually a smear of colour on the coat or was a button. It could guess it was a button but it might be wrong, it might actually have been some bird shit.
Re: (Score:2)
I think that is what "timeOday" meant by re-staging the shoot. When being restaged, it won't be the exact same coat. The director would view the video and decide if that smear was a button or not, and use a different coat for the re-staging. Different directors would make different decisions.
Re: (Score:2)
Re: (Score:2)
you can interpolate and I can imagine with gen AI that it'll be possible to clean up a film, frame by frame or as a group as the TFA indicates.
Re: Its just making up stuff (Score:2)
Interpolation in this case is a euphemism for guesswork.
Re: Its just making up stuff (Score:2)
Interpolation is always educated guesswork.
Re: (Score:2)
Look up what people who restore old or damaged paintings do. At some point its all guesswork, but educated guesswork. As someone above said, for art (assuming its tunable) this is perfectly acceptable. For figuring out who's really under the mask in that old bigfoot footage, not so much.
Re: (Score:2)
When looking at a grainy old video or a 'fake' upscaled version, I think I'll be content with the fake.
If done well, it'll create filler that is at least consistent with the original and it'll be far more pleasing to the eye.
Re:Its just making up stuff (Score:4, Insightful)
I have a bigger concern: This is literally rewriting history (from the AI biased perspective). Sure, most of the time this will be used for entertainment purposes, but there will be some cases that for some people -chiefly for those without prior knowledge of the context of the original material- the modified version will represent what actually happened. I can imagine future generations seeing History through the lens of a creative AI algorithm.
Re: (Score:1)
If they are looking at unvetted material, they are going to see lots of fake, rigged, and doctored crap anyhow.
Re: (Score:2)
This is literally rewriting history
The existence or use of AI isn't rewriting history. People re-write history. Fake video wasn't invented this year. It is (as it always has been) up to users to ensure they get their source of information from a reputable place.
No one cares. (Score:4, Insightful)
This is real life , not CSI.
No one cares. People aren't watching movies or videos for court evidence. Guess away, as long as the result looks good literally no one gives a shit.
Quido shot first! (Score:2)
If people use it for forensic, historic, or scientific purposes, it could indeed result in problems. As long is it's used for entertainment and not passed off as original or authentic, I don't see a real problem.
Unless, it makes Guido shoot first, then all hell breaks loose.
Re: (Score:1)
Correction: Guido. AI turned my "G" to a "Q".
Re: (Score:2)
I just started running Futurama season 1 through Waifu2x. The results are pretty good! Seasons 1-4 were mastered in SD, and they look pretty crap on a 46" TV.
Re: (Score:2)
Yes but for entertainment purposes like the watching of films, it doesn't really matter - its all a creative work from the start anyways. Quality is more important than accuracy in some cases.
Re: (Score:2)
You act as if its a destructive process. The unaltered versions are still there for viewing if anyone has true academic curiousity.
Re: (Score:2)
You act as if its a destructive process. The unaltered versions are still there for viewing if anyone has true academic curiousity.
Not really. Once this becomes more common the originals will stay in the archives and the AI versions are the ones we get to see.
Just like it has happened with 4:3 formatted material. Seeing this in the correct format is getting harder and harder with all the new remasters.
Not sure about live action (Score:2)
Fast motion can sometimes result in a bit of weirdness though (there's a Tekken 2 vid where Marshal Law jumps out of a window and that looks strange in that one frame) but that can be tweaked. The footage looks more like it was recorded on a PS2 DVD
Re: (Score:3)
Of course you can [youtube.com]. It's not that difficult.
It adds detail where there was none before! (Score:4, Funny)
The pictures do have much higher resolution and, you had never realized that Lincoln's stovepipe hat had polka dots before. The picture is better than reality! Sometimes it drips a bit! Now with 15% more tentacles!
This is amazing! (Score:4, Insightful)
Adobe has achieved what Topaz has been doing for several years now. It's funny how being big can mean that when you're late to the party you are somehow newsworthy.
Re: (Score:2)
everything is sharp (Score:3)
Re: (Score:2)
I like sand kicked in my eyes. It's a significant than the rocks which were kicked into it in the form of horrendous compression artifacts and horrendously blurry stuff we have been given previously.
Is it perfect? No.
Is it far better than what we had before? Almost universally.
Don't fade away (Score:2)
This begs the question, (Score:1)
Re: (Score:3)
That's Republicans trying to select a new Speaker of the House.
Re: This begs the question, (Score:2)
Elephant uses a background filter when teleconferencing.
Re: (Score:2)
Effectively you need to tell the AI model how good the source material is. Motion pictures use a shallow depth of field to have the subject in sharp focus and to blur the background. You don't want the AI sharpening the background so it needs to know what counts as sharp (where it should try to increase detail) and what counts as blurry (which should be left blurry).
Old Games (Score:1)
could be much better (Score:2)
The system reported does not take advantage of domain-specific knowledge. The vast majority of low res videos on say youtube contain famous people. Famous people have a lot of very high resolution still photographs in existence, at the very least their head shots. Using current technology, the high res photos can be used for training and then used to enhance video of those people. Similar things can be done for known objects, such as pianos, guitars, or airplanes, but that also depends on domain-specifi
I toldja it was worth keeping my (Score:1)
...ASCII porn.
Re: (Score:2)
Couldn't you just increase the font size and migrate to TTF?
Does it work for porn? (Score:2)
Because that us basically the main use I can see for it: Accuracy irrelevant.
Impressive... considering they started with crap (Score:1)