Researchers Devise AI System To Reduce Noise in Photos (venturebeat.com) 69
Researchers from Nvidia, MIT, and Aalto University are using artificial intelligence to reduce noise in photos. The team used 50,000 images from the ImageNet dataset to train its AI system for reconstructing photos, and the system is able to remove noise from an image even though it has never seen the image without noise. VentureBeat: Named Noise2Noise, the AI system was created using deep learning and draws its intelligence from 50,000 images from the ImageNet database. Each came as a clean, high-quality image without noise but was manipulated to add randomized noise. Computer-generated images and MRI scans were also used to train Noise2Noise. Denoising or noise reduction methods have been around for a long time now, but methods that utilize deep learning are a more recent phenomenon.
should have contacted me (Score:5, Interesting)
I have a couple of thousand images that would benefit from noise reduction. Shooting movement in low light means high ISO or blur, so I accept the noise.
If they wanted some serious training data, the whole astrophotography field is full of people that take dozens of pictures of the same thing then sample across all of them to remove noise. That means they have plenty of randomness in their noisy images and a nice clean one for comparison.
Re: (Score:2, Funny)
Natalie Portman, naked and petrified, covered in hot grits.
Re:should have contacted me (Score:5, Interesting)
Low light enhancement:
https://www.youtube.com/watch?... [youtube.com]
What about raw image processing? (Score:3)
One thing I've been slowly trying to get going as a side project is exploring the use of neural networks to process raw image data.
A lot goes into processing a raw image, there is conversion of data from various color matrices, generally some sharpening, and also noise reduction. It seems like a good neural net could possibly handle all aspects and maybe do a better job if trained well, as it might spot patterns in noise or color conversion that algorithm designers to date have not (well except for recognizing color swatches and altering processing based on that... )
I was thinking to train you could just do some very accurate high res close up images of a variety of subjects that were very carefully color corrected. Then you would take images from a wider FOV or farther away, so that you could use the high-res images to determine what a "real" output pixel should be, vs whatever the result of combining various sensor data would be to produce a result.
Seems like a lot of potential here beyond just noise reduction...
Re: (Score:2)
In other words, you think heuristics could do a better job than software that understands the specific properties of the underlying sensor. About as interesting as your other insights.
Re: (Score:3)
One algorithm understands the sensor, the other understands typical images.
The problem is opposite to what you propose (Score:2)
In other words, you think heuristics could do a better job than software that understands the specific properties of the underlying sensor.
You don't have to have just one approach, and could combine both.
However, I don't think you are understand what I am describing. The training data would come from the same sensor, I'm not talking about arbitrary images here - so the "heuristics" would in fact be learning based on the properties of the sensor for the raw data it would be working with. In fact it probabl
Digital forensics (Score:1)
With this kind opf digital processing going on, how will we know if something has been photchopped in the future? How will we expose Deepfakes? All this processing makes this sort of thing much harder....
Keep going please, I personally am looking forward to Sandra Bullocks porn releases!
Re: (Score:3)
Because somebody with Sandra Bullock's acting ability is best suited for porn.
Computer! (Score:4, Funny)
Magnify and enhance sector A5.
Once again, life imitates science fiction.
Obligatory Futurama (Score:2, Offtopic)
https://www.youtube.com/watch?... [youtube.com]
Re: (Score:2)
Re: (Score:3)
Availability? (Score:2)
Tired of AI This and AI That (Score:5, Funny)
Re: (Score:2)
I am with you, but sadly I must inform you that we lost the battle. I always read it as Algorithmic Interface these days, real AI is still only science fiction.
The data set they used to program it was artificially generated noise, what happens when it encounters real noise?
Re: (Score:2)
Do you also complain when people say their dog is intelligent, while it can't even speak decent English, brew coffee, or take the day off ?
Re: (Score:2)
Do you also complain when people say their dog is intelligent, while it can't even speak decent English, brew coffee, or take the day off ?
Depends on whether it was willing to fetch the morning paper in the dog's case.
Re: (Score:2)
It's not Artificial Free Will. Is artificial sugar sugar? No, but it is sweet like sugar. Is artificial intelligence intelligence? No, however, it looks as thought it is intelligent to an ignorant observer. For example, playing a game.
Some sugar substitutes are not artificial and some are sugars. So there’s that. Anyway, I call the game playing computer/program an expert system.
Re:Tired of AI This and AI That (Score:5, Informative)
The abstract states "We apply basic statistical reasoning to signal reconstruction by machine learning — learning to map corrupted observations to clean signals — with a simple and powerful conclusion: under certain common circumstances, it is possible to learn to restore signals without ever observing clean ones , at performance close or equal to training using clean exemplars."
The results show dramatic improvements that are very close to original image (before random noise is introduced to generate the input)- a level of improvement that is simply not possible with conventional image processing/denoising filters.
If this is not AI, I don't know what else would be.
Re: (Score:2)
Re: (Score:2)
Article needs image diffs (Score:3, Interesting)
It would be interesting to see a visual diff between the denoised result and the source image before the random noise was added, in order to see what kinds of artifacts were generated during the denoising process. For example, did it add any leaves to the image of the koala?
Re: (Score:3)
Re: (Score:3)
Their model might be trained to specifically to their noise generation.
They definitely trained the model to various types of noises. The whole point of the paper is that it can learn to denoise extremely diverse noise types from Gaussian to Monte Carlo to MRI read noise to text overlays.
Re: (Score:3)
Maybe read the paper in the link? They provide before and after examples.
Re: (Score:2)
You're right, the paper isn't behind a paywall.
There are no image diffs, just comparison shots with closeups pointed out. In the Koala image, the image processing doesn't add any leaves (which is good) but it leaves out some of the stems (which is expected). In the MRI, the image in the area of the cerebellum is different enough not to be trusted.
Re: (Score:2)
It would be interesting to see a visual diff between the denoised result and the source image before the random noise was added, in order to see what kinds of artifacts were generated during the denoising process. For example, did it add any leaves to the image of the koala?
The second image in TFA (a picture of a human head) shows exactly that.
Official Photographer (Score:2)
Do you think this will work on upskirt photos? Asking for a big, wet, orange friend.
blind source separation? (Score:3)
While the images they have shown as examples are really pretty impressive, given that they're using a training set of Image A versus Image A Plus Noise, the problem is akin to blind source separation (BSS). There's been quite a lot of work done on BSS, much of which is very impressive (and based on neural nets).
The critical issue is to see what happens when they take a real photograph that has not been adulterated to add noise, and improve that. Will their model of a noiseless source image with additive noise still hold? The article doesn't touch upon that critical test, unfortunately.
The results they show are very, very cool, though. And if they hold up for MRI work, it would be a game-changer in the medical field. The article shows an MRI adulterated with noise, their recovered image, and the noiseless ground truth. A better test would be to take an MRI that was scanned for too short a time (and thus is noisy), and compare their extraction against an MRI with identical scanning parameters, except for normal imaging time. MRI magnet time is expensive; if it can be reduced by 50% and get equivalent image quality, that's a huge advance.
Re: (Score:2, Informative)
The paper includes similar experiments with Poisson noise instead of gaussian noise. The neural network does need to be trained differently depending on the type of noise.
It will never be equivalent image quality, since the 50% exposure contains less information. Possibly it will be good enough, but the 50% exposure will possibly be good enough anyway without the neural network. The neural network is literally making up information based on what it remembers from the training data, which seems like an incre
Already done for anime art: "Waifu2x" (Score:2)
Today, noise is less an issue in most situations (Score:2)
Already in use in 3D rendering (Score:2)
3D ray tracing shoots out photons with a certain degree of randomness to build up the image.
The more photons you collect, the less grainy the picture gets, which is great for this type of training because you can generate the training data to as high or low a quality as you like.
The end result is a black box you run your image through that maybe cuts your render time in half (I don't remember what the actual improvement rates are), essentially for free.
Re: (Score:2)
They cover this in the paper, under Monte Carlo rendering. Based on the timings they report, a trained CNN was producing results close to equivalent to a much higher ray count, for a low count, real-time Monte Carlo rendered scene. 2000 times faster.
I don't know what current de-noising for Monte Carlo rendering looks like, but this is quite interesting. I've also seen some work combining CNN with RNN/LTSM that might also apply to this.
I use ... (Score:2)
... noise-cancelling headphones while viewing food photos on Facebook.
The audio version of noise-reducing software... (Score:2)
I ran the audio version of this noise-reducing software on Lou Reed's "Metal Machine Music" and ended up with a telephone dial tone.