Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
AI Technology

Researchers Devise AI System To Reduce Noise in Photos (venturebeat.com) 69

Posted by msmash from the how-about-that dept.
Researchers from Nvidia, MIT, and Aalto University are using artificial intelligence to reduce noise in photos. The team used 50,000 images from the ImageNet dataset to train its AI system for reconstructing photos, and the system is able to remove noise from an image even though it has never seen the image without noise. VentureBeat: Named Noise2Noise, the AI system was created using deep learning and draws its intelligence from 50,000 images from the ImageNet database. Each came as a clean, high-quality image without noise but was manipulated to add randomized noise. Computer-generated images and MRI scans were also used to train Noise2Noise. Denoising or noise reduction methods have been around for a long time now, but methods that utilize deep learning are a more recent phenomenon.
This discussion has been archived. No new comments can be posted.

Researchers Devise AI System To Reduce Noise in Photos More

Researchers Devise AI System To Reduce Noise in Photos

Comments Filter:

  • should have contacted me (Score:5, Interesting)

    by Cederic ( 9623 ) on Tuesday July 10, 2018 @01:42PM (#56924324) Journal

    I have a couple of thousand images that would benefit from noise reduction. Shooting movement in low light means high ISO or blur, so I accept the noise.

    If they wanted some serious training data, the whole astrophotography field is full of people that take dozens of pictures of the same thing then sample across all of them to remove noise. That means they have plenty of randomness in their noisy images and a nice clean one for comparison.

  • What about raw image processing? (Score:3)

    by SuperKendall ( 25149 ) on Tuesday July 10, 2018 @01:46PM (#56924350)

    One thing I've been slowly trying to get going as a side project is exploring the use of neural networks to process raw image data.

    A lot goes into processing a raw image, there is conversion of data from various color matrices, generally some sharpening, and also noise reduction. It seems like a good neural net could possibly handle all aspects and maybe do a better job if trained well, as it might spot patterns in noise or color conversion that algorithm designers to date have not (well except for recognizing color swatches and altering processing based on that... )

    I was thinking to train you could just do some very accurate high res close up images of a variety of subjects that were very carefully color corrected. Then you would take images from a wider FOV or farther away, so that you could use the high-res images to determine what a "real" output pixel should be, vs whatever the result of combining various sensor data would be to produce a result.

    Seems like a lot of potential here beyond just noise reduction...

    • Re: (Score:2)

      by dfghjk ( 711126 )

      In other words, you think heuristics could do a better job than software that understands the specific properties of the underlying sensor. About as interesting as your other insights.

      • One algorithm understands the sensor, the other understands typical images.

      • In other words, you think heuristics could do a better job than software that understands the specific properties of the underlying sensor.

        You don't have to have just one approach, and could combine both.

        However, I don't think you are understand what I am describing. The training data would come from the same sensor, I'm not talking about arbitrary images here - so the "heuristics" would in fact be learning based on the properties of the sensor for the raw data it would be working with. In fact it probabl

  • Digital forensics (Score:1)

    by Anonymous Coward

    With this kind opf digital processing going on, how will we know if something has been photchopped in the future? How will we expose Deepfakes? All this processing makes this sort of thing much harder....

    Keep going please, I personally am looking forward to Sandra Bullocks porn releases!

  • Computer! (Score:4, Funny)

    by Pete Smoot ( 4289807 ) on Tuesday July 10, 2018 @01:58PM (#56924418)

    Magnify and enhance sector A5.

    Once again, life imitates science fiction.

  • I'm curious how the results stack up against commercial options like in Lightroom or Aperture. If these can reduce noise without softening the image, I'd be very interested in getting it.

  • Tired of AI This and AI That (Score:5, Funny)

    by Marlin Schwanke ( 3574769 ) on Tuesday July 10, 2018 @01:59PM (#56924424)
    I wish they'd quit with the AI and Artificial Intelligence monikers being applied to everything in tech these days. The day one of these AI's tells me that, no, it won't brew my coffee this morning because it is taking the day off is the day I might buy in to this nonsense.

    • I am with you, but sadly I must inform you that we lost the battle. I always read it as Algorithmic Interface these days, real AI is still only science fiction.

      The data set they used to program it was artificially generated noise, what happens when it encounters real noise?

    • Do you also complain when people say their dog is intelligent, while it can't even speak decent English, brew coffee, or take the day off ?

      • Do you also complain when people say their dog is intelligent, while it can't even speak decent English, brew coffee, or take the day off ?

        Depends on whether it was willing to fetch the morning paper in the dog's case.

    • Re:Tired of AI This and AI That (Score:5, Informative)

      by yaznaz ( 4678625 ) on Tuesday July 10, 2018 @02:33PM (#56924594)
      Did you even check the paper at: https://arxiv.org/pdf/1803.041... [arxiv.org]

      The abstract states "We apply basic statistical reasoning to signal reconstruction by machine learning — learning to map corrupted observations to clean signals — with a simple and powerful conclusion: under certain common circumstances, it is possible to learn to restore signals without ever observing clean ones , at performance close or equal to training using clean exemplars."

      The results show dramatic improvements that are very close to original image (before random noise is introduced to generate the input)- a level of improvement that is simply not possible with conventional image processing/denoising filters.

      If this is not AI, I don't know what else would be.
      • Looking at the samples, it's pretty clear that most of the noise was high-frequency (i.e. pixel-level), high-contrast. That's relatively easy to filter out even without AI. Back in the days of analog TVs, if you had a poor signal the image ended up with a lot of snow. Since the snow was high-frequency, high-contrast (black and white dots), a trick to filtering it out was to cover the screen with pieces of tissue paper. The static (CRTs used electron guns) held the tissues up against the screen, and they
    • It's all marketing hype. The vast majority of what they're trotting out as 'AI' isn't really much different than what they had 20-30 years ago, it's just bigger and faster because there is bigger and faster hardware to run it on. They didn't have Beowulf Cluster supercomputers 20 to 30 years ago, they didn't have ubiquitos 4, 8, 16 core processors, or the fast memory, or the gigantic hard disks, gazillion-core GPUs, and so on, and so on. Same crap, better hardware, slightly better results, and just about as

  • Article needs image diffs (Score:3, Interesting)

    by Ichijo ( 607641 ) on Tuesday July 10, 2018 @02:00PM (#56924440) Journal

    It would be interesting to see a visual diff between the denoised result and the source image before the random noise was added, in order to see what kinds of artifacts were generated during the denoising process. For example, did it add any leaves to the image of the koala?

    • I agree. I'd also like to see some before and after on images that were noisy on their own--not having noise artificially added. I understand the value of adding noise artificially--you have a perfect image to use as a definition of success for training. But to really judge the effectiveness, I'd want to see some non-generated noise. Their model might be trained to specifically to their noise generation. All that said... it's a cool project. I hate how slashdotters gotta be down on everything all the time.

      • Their model might be trained to specifically to their noise generation.

        They definitely trained the model to various types of noises. The whole point of the paper is that it can learn to denoise extremely diverse noise types from Gaussian to Monte Carlo to MRI read noise to text overlays.

    • Maybe read the paper in the link? They provide before and after examples.

      • Re: (Score:2)

        by Ichijo ( 607641 )

        You're right, the paper isn't behind a paywall.

        There are no image diffs, just comparison shots with closeups pointed out. In the Koala image, the image processing doesn't add any leaves (which is good) but it leaves out some of the stems (which is expected). In the MRI, the image in the area of the cerebellum is different enough not to be trusted.

    • It would be interesting to see a visual diff between the denoised result and the source image before the random noise was added, in order to see what kinds of artifacts were generated during the denoising process. For example, did it add any leaves to the image of the koala?

      The second image in TFA (a picture of a human head) shows exactly that.

  • Do you think this will work on upskirt photos? Asking for a big, wet, orange friend.

  • blind source separation? (Score:3)

    by pz ( 113803 ) on Tuesday July 10, 2018 @02:45PM (#56924642) Journal

    While the images they have shown as examples are really pretty impressive, given that they're using a training set of Image A versus Image A Plus Noise, the problem is akin to blind source separation (BSS). There's been quite a lot of work done on BSS, much of which is very impressive (and based on neural nets).

    The critical issue is to see what happens when they take a real photograph that has not been adulterated to add noise, and improve that. Will their model of a noiseless source image with additive noise still hold? The article doesn't touch upon that critical test, unfortunately.

    The results they show are very, very cool, though. And if they hold up for MRI work, it would be a game-changer in the medical field. The article shows an MRI adulterated with noise, their recovered image, and the noiseless ground truth. A better test would be to take an MRI that was scanned for too short a time (and thus is noisy), and compare their extraction against an MRI with identical scanning parameters, except for normal imaging time. MRI magnet time is expensive; if it can be reduced by 50% and get equivalent image quality, that's a huge advance.

    • Re: (Score:2, Informative)

      by Anonymous Coward

      The paper includes similar experiments with Poisson noise instead of gaussian noise. The neural network does need to be trained differently depending on the type of noise.

      It will never be equivalent image quality, since the 50% exposure contains less information. Possibly it will be good enough, but the 50% exposure will possibly be good enough anyway without the neural network. The neural network is literally making up information based on what it remembers from the training data, which seems like an incre

  • Here's a program (in development since at least three years ago) which uses neural networks to upscale and de-noise anime-style art: https://github.com/nagadomi/wa... [github.com]
  • Nowadays a good pro-camera sensor shows almost no visible noise in rather dark conditions (using high ISO to keep taking photos under 20 ms). Not perfect yet, but ISO improved by a factor of 10 in 10 years. Negative had their time, and some important professional features did not evolve in pro digital camera thanks to pro-photographers unable to catch up with progress ; they'll keep talking about noise in 10 years when nobody cares anymore.

  • 3D ray tracing shoots out photons with a certain degree of randomness to build up the image.
    The more photons you collect, the less grainy the picture gets, which is great for this type of training because you can generate the training data to as high or low a quality as you like.
    The end result is a black box you run your image through that maybe cuts your render time in half (I don't remember what the actual improvement rates are), essentially for free.

    • Re: (Score:2)

      by Wizarth ( 785742 )

      They cover this in the paper, under Monte Carlo rendering. Based on the timings they report, a trained CNN was producing results close to equivalent to a much higher ray count, for a low count, real-time Monte Carlo rendered scene. 2000 times faster.
      I don't know what current de-noising for Monte Carlo rendering looks like, but this is quite interesting. I've also seen some work combining CNN with RNN/LTSM that might also apply to this.

  • ... noise-cancelling headphones while viewing food photos on Facebook.

  • I ran the audio version of this noise-reducing software on Lou Reed's "Metal Machine Music" and ended up with a telephone dial tone.

Slashdot Top Deals

In the long run, every program becomes rococco, and then rubble. -- Alan Perlis

Close