Adobe's Impressive AI Upscaling Project Makes Blurry Videos Look HD 43
Adobe researchers have developed a new generative AI model called VideoGigaGAN that can upscale blurry videos at up to eight times their original resolution. From a report: Introduced in a paper published on April 18th, Adobe claims VideoGigaGAN is superior to other Video Super Resolution (VSR) methods as it can provide more fine-grained details without introducing any "AI weirdness" to the footage. In a nutshell, Generative Adversarial Networks (GANs) are effective for upscaling still images to a higher resolution, but struggle to do the same for video without introducing flickering and other unwanted artifacts. Other upscaling methods can avoid this, but the results aren't as sharp or detailed. VideoGigaGAN aims to provide the best of both worlds -- the higher image/video quality of GAN models, with fewer flickering or distortion issues across output frames. The company has provided several examples here that show its work in full resolution.
Solving many a crime (Score:3)
If the claim is true, I would imagine law enforcement may be quite busy solving a lot of cold cases as a result. Crime filmed in BlurryHD has ironically become the video surveillance standard regardless of camera tech.
Re: (Score:3, Informative)
It can only make some plausible extrapolations and it can make these less well than a competent human. All the current hype AI has brought us is texts that sound good but often are bullshit and now pictures that look sharp but are not. You know, better sounding or better looking crap. Still crap.
Re: Solving many a crime (Score:3)
Re:Solving many a crime (Score:5, Informative)
it can make these less well than a competent human.
Not true. An AI can unblur much better than a human, especially with video where the AI can look at multiple frames simultaneously.
All the current hype AI has brought us is texts that sound good
If you believe that, you're not paying attention.
Re: (Score:1)
Being a mindless fanboi does not actually make you insightful. You know that, right?
Re: (Score:2)
I would imagine law enforcement may be quite busy solving a lot of cold cases as a result.
If the enhanced image leads to other evidence, they might crack some cold cases.
But the unblurred images are unlikely to be directly admissible as evidence in court.
An AI can unblur a video much better than a human because the AI can combine the pixels from multiple frames.
Even better if the AI also has a clear photo of the scene taken later by the police. By comparing the clear photo to the blurry photo, an AI can figure out the characteristics of the lens and sensor and apply them to the perp's face, hand
Re:Solving many a crime (Score:4, Interesting)
If the enhanced image leads to other evidence, they might crack some cold cases.
As the person who has been responsible for responding to law enforcement video requests and occasionally (three times) testifying as to that process and their authenticity, it's exceptionally rare (never personally seen it) for CCTV footage alone to convict someone. Most of the time it leads the police to a suspect, usually because someone they know recognized them (not for nothing that the police frequently publish these videos/images) and then the idiot convicts themselves by talking to the cops (pro-tip, never do this). Less frequently it leads to other witnesses and/or evidence that gets them convicted.
Most of the time it doesn't do a damn thing because the crime in question isn't worth the police resources to follow up on, even if you have something pretty damning, like a legible license plate.
If AI enhancement results in more arrests for crime, I'd wager it comes about largely through the police releasing the enhanced images to the media, with the suspect(s) then being outed by their friends/family. Cops go talk to the suspect, he's an idiot and thinks he can outsmart them, and ends up saying incriminating things. Same as today, it'll just be higher quality images on the local news.
Re: (Score:3)
Of course not, the video looks HD because what could be called more or less "AI hallucinations" have been added to it. So what you see isn't real, it would never stand in court.
Re: (Score:2)
The "girl demo video" shows AI-generated horror teeth in the girl's mouth that are both unnaturally short and fluctuate in size as if they were made of stretchy tissue.
I have some mild use cases for ChatGPT text, but this image/video GenAI nonsense has to stop. It is not showing information, it adds non-information and takes my precious time with nothing to give in return.
Re: (Score:3)
That's the problem. I bet this will be applied to law enforcement to "zoom and enhance" like all the best crime dramas do. But remember, the enhanced details aren't really there. They're merely plausible fiction based on the information that is available, they're not adding any new information. People will see blurry security camera footage of a crime. The enhanced version will show a weapon where there is none, or no weapon when one was used. Detail will be added to faces that will make them look like on
Re: (Score:2)
Cleaning up old movies or TV shows, for instance.
And porn. I knew I saved all those early aughts videos for a reason. ;-)
Re: (Score:2)
A lot of shit looks better without too much of the gory details. The downside is unrealistic expectations, but it is a minor issue in the typical case, where porn and masturbation have replaced sex.
Re: (Score:3)
Re: (Score:2)
I know you jest, but that would be an interesting test for accuracy from these "ai" type programs.
Re: (Score:2)
That's the problem. I bet this will be applied to law enforcement to "zoom and enhance" like all the best crime dramas do. But remember, the enhanced details aren't really there.
Actually... that's not strictly true. Back in the 90's I worked at a company that worked with a vendor who could essentially motion-track a video and use those motion vectors to do a few things like upscale a video because knowing the motion of the camera means knowing what to do with the sub-pixel information you're getting. (fundamentally this is similar to how a scanner or copier works.) You can also use it to remove motion blur, especially if 3d points from the video can be intuited.
Am I saying the
Re: (Score:2)
Not sure it's good for that .. the AI will fill in an "average" face .. so a person with an average face will be implicated.
Re: (Score:2)
There is nothing new about this technology. I've been using Topaz AI's Video product (ne service) to upscale and enhance videos for years.
Re: (Score:2)
Zoom, enhance, zoom, enhance... (Score:2)
Zoom...enhance.
Includes adding things that are not there (Score:3)
Visual hallucinations, essentially.
Re:Includes adding things that are not there (Score:5, Insightful)
Visual hallucinations, essentially.
I think this is unnecessarily harsh. A more correct description of the process would be "informed guesses", rather than "visual hallucinations".
Of course, the information isn't there to begin with, so the AI has to make up new data that never existed in the original. But the whole reason for the exercise isn't to extract information from the image - like "enhancing" it until the face of the perpetrator is seen in a reflection. It's to create a more visually pleasant photo from one that's technically imperfect.
To do that, the AI does make up stuff and adds it to the image. However, this is not just random hallucination - the AI has been trained on a corpus of similar images and will usually create something that matches them (unless maybe it's a politically hyper-correct Google AI). So, in most cases, it will complete the images in plausible ways. It's just another tool, like all the filters or processes used by photographers for so many years, in the darkroom or digitally.
Re: (Score:1)
There is nothing "informed" about AI.
Re: (Score:2, Informative)
Dismissing the whole field with such a hand wave does nothing to your credibility. If you're right about anything, it's probably by accident.
Re: (Score:1)
There is nothing "informed" about AI.
Well. I'd expect such a statement from somebody who still equates machines with steam engines, or with that new-fangled Diesel thing the kids are going on about, but you can't expect to be taken seriously if you keep spouting this kind of nonsense.
Re: (Score:2)
What a biased AI produces is a biased image completion. Your side of the debate is claiming this is an "informed" and useful thing. Other people would not, especially if the completion is
Re: (Score:2)
Thanks. As a CS type with an engineering PhD in the IT area and as a follower of AI research for about 35 years now, I like to think that I am somewhat informed about the capabilities, and more often lack thereof, of "AI". Any type of "insight" (and "informed" falls under that) is completely alien to any AI we have, except, with strong limits, automated deduction. And that gets bogged down in complexity already for very simple things and must be regarded as a failed approach for anything not very specialize
Re: (Score:2)
Seems like a perfect case of Clarke's first law. I suggest you meditate on it for a bit. It might help you.
When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
Re: (Score:2)
Does not apply here because you are too thick to understand what Clarke was actually saying. But you can shove your condescending attitude up your ass.
Re: (Score:2)
Your side of the debate is claiming this is an "informed" and useful thing. Other people would not, especially if the completion is used by law enforcement or presented in a court of law.
Your "other people" are committing a classic logical error - the fallacy of four terms, in this case due to equivocation. They're making a confusion between two different meanings of the word "informed". They think "informed" means "The AI is informed about the circumstances where the image was taken and can provide data that can be used for law enforcement or presented in courts of law" - just like the "enhance" thing in some police procedural. They are wrong.
The meaning of "informed" used by my side (what
Re: (Score:2)
I'm going to guess that you are not a native English speaker(*), as the meaning of "informed" that you suggest is not the general usage meaning of "informed" in English, but closer to some European usage. That is why I proposed "biased", since that is closer to what you describe as "informed".
(*) or else that you've read too many papers by non-English speakers and have picked up this neologism as a result.
re: Things not there, Visual hallucinations (Score:2)
Happens to me when I take off my undies.
Solved! (Score:5, Funny)
Han shot first via his 3rd arm.
About time (Score:2)
Perhaps now we can get all that bounce and bad focus out of movies like The Blair Witch Project and Cloverfield.
....solved mysteries? (Score:2)
Gotta apply this to all those Loch Ness, UFO and Bigfoot videos, let's see what's REALLY going on!
Real tech finally catches up with reel tech (Score:2)
"Enhance!"
Re: (Score:3)
The problem being that you can't actually extract information that wasn't there, you can only add new information that fits.
If you treat the extra detail as an inaccurate estimate, it's fine. But zoom in on that letter someone's holding in their hands and whatever you read will have been made up by the AI.
Re: (Score:2)
Enhance 57-19, track 45 left, stop
Audio (Score:2)
I would love to see something like this for audio.
Imagine all of those really old recordings, heck even 78s and 45s restored to CD or better quality!
On the hi-fi side, getting 24/192 resolution from LPs or CDs would make some people very happy.
Even if it were illusory.
Wrong application (Score:2)
This is the wrong application. Videos with the greatest interest to people and marketing potential are of famous people. Famous people have plenty of high-res photographs available. It's much easier to apply this technology when you know in advance what a person looks like, and it's more important for the viewer to have a sharp person than a sharp potted plant in the background. Consider, for example all the videos of musicians.