This AI Clears Up Your Hazy Photos


Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér. Today we are going to talk about a paper that
builds on a previous work by the name Deep Image Priors, DIP in short. This work was capable of performing JPEG compression
artifact removal, image inpainting, or in other words, filling in parts of the image
with data that makes sense, super resolution, and image denoising. It was quite the package. This new method is able to subdivide an image
into a collection of layers, which makes it capable of doing many seemingly unrelated
tasks, for instance, one, it can do image segmentation, which typically means producing
a mask that shows us the boundaries between the foreground and background. As an additional advantage, it can also do
this for videos as well. Two, it can perform dehazing, which can also
be thought of as a decomposition task where the input is one image, and the output is
an image with haze, and one with the objects hiding behind the haze. If you spend a tiny bit of time looking out
the window on a hazy day, you will immediately see that this is immensely difficult, mostly
because of the fact that the amount of haze that we see is non-uniform along the landscape. The AI has to detect and remove just the right
amount of this haze and recover the original colors of the image. And three, it can also subdivide these crazy
examples where two images are blended together. In a moment, I’ll show you a better example
with a complex texture where it is easier to see the utility of such a technique. And four, of course, it can also perform image
inpainting, which, for instance, can help us remove watermarks or other unwanted artifacts
from our photos. This case can also be thought of an image
layer plus a watermark layer, and in the end, the algorithm is able to recover both of them. As you see here on the right, a tiny part
of the content seems to bleed into the watermark layer, but the results are still amazing. It does this by using multiple of these DIPs,
deep image prior networks, and goes by the name DoubleDIP. That one got me good when I’ve first seen
it. You see here how it tries to reproduce this
complex textured pattern as a sum of these two, much simpler individual components. The supplementary materials are available
right in your browser, and show you a ton of comparisons against other previous works. Here you see results from these earlier works
on image dehazing and see that indeed, the new results are second to none. And all this progress within only two years. What a time to be alive! If like me, you love information theory, woo-hoo! Make sure to have a look at the paper and
you’ll be a happy person. This episode has been supported by Weights
& Biases. Weights & Biases provides tools to track your
experiments in your deep learning projects. It is like a shared logbook for your team,
and with this, you can compare your own experiment results, put them next to what your colleagues
did and you can discuss your successes and failures much easier. It takes less than 5 minutes to set up and
is being used by OpenAI, Toyota Research, Stanford and Berkeley. It was also used in this OpenAI project that
you see here, which we covered earlier in the series. They reported that experiment tracking was
crucial in this project and that this tool saved them quite a bit of time and money. If only I had an access to such a tool during
our last research project where I had to compare the performance of neural networks for months
and months. Well, it turns out, I will be able to get
access to these tools, because, get this, it’s free and will always be free for academics
and open source projects. Make sure to visit them through wandb.com/papers
or just click the link in the video description and sign up for a free demo today. Our thanks to Weights & Biases for helping
us make better videos for you. Thanks for watching and for your generous
support, and I’ll see you next time!

100 thoughts on “This AI Clears Up Your Hazy Photos

  1. The next generation of smartphones better have this built-in.
    I love when multiple cool techs are combined into a super cool tech.

  2. What a time to be alive. I mean, to be fair, most times are good times to be alive. It's better than the alternative. 😱

  3. Would it be possible to get an episode providing a follow-up on how some interesting things you've talked about so far have been used in final products or is planned to be ? It's all so interesting and I'm really curious on what application all of this is finding.

  4. @Karoly When can we expect a successor to the jpeg, where the encoding and decoding is performed using a highly-optimized neural network?

  5. Next Up compile the clips from recently published https://openai.com/blog/emergent-tool-use/ and discuss it.It's way too much cool.

  6. I'm really curious what AI will mean for copyright. If you train AI using an artist's music and let it write a song, is it stealing musical knowledge, or is it inspired by its teacher? If AI removes watermarks and improves resolution on small sample thumbnails, is it pirating photos, or is it dreaming up new photos using reference materials? What if an AI could write and animate a completely new and believable episode for your favorite cancelled cartoon? I think these questions will become even more difficult to answer than current copyright issues.

  7. Simple ideas are the most powerful. The authors were really clever in uniting seemingly very different problems (inpainting, watermarker removal, segmentation, denoising, dehazing) as a single super-problem. Absolutely fantastic!

  8. of course artistically we would like to be able to add realistic haze to the image as well, and for that matter film grain as well.

  9. Sometimes I wonder if the researchers actually think about the consequences of their work. The "remove watermarks" feature is going to be 99 percent abused. Maybe ethical courses should be mandatory?

  10. Please do papers on detecting ML generated text, images, and video. Or papers on generating false data to reduce the effectiveness of ML, in such a way that isnt similar enough to provide it more accuracy.

  11. Great . I cant find an example for the usage of segmentation.py in the github repository. If someone can point to one I would indeed feel: What a time to be alive 🙂
    Thanks

  12. Nice video as usual Mr. Fahir! OK, that's a perfect start! Just a random question from a person who is not trying to sell anything: Is Singularitynet.io onto something exceptional when it comes to AI or are they just pulling our leg?? Is it a good idea to be on the cutting edge of 2 technologies–AI and cryptocurrencies? (They created the world-famous robot Sophia). Thank you Sir and keep doing your thing!

  13. Politicians will love this. (Applies dehazing AI.) "Look! We have the cleanest air, the best air. No need to ban smoggy combustion engines."

    Also 1:42, the art of photographers gets further devalued. Take the low-res promo thumbnail from the catalog page, remove the watermark, upscale, de-noise, don't pay or credit its creator.

  14. This could also be used to take reflections out of glass in photos that you want to use photogammetry to make 3D models from.

  15. Some of the research and development I do with new color models and chromatic gradient segmentations could probably go into some exponential directions when fed into these type of AI image-processing algorithms. Exciting stuff as always, thanks for summarizing all this work. Love your channel, subscribed.

  16. Really need that de-hazing for the snaps from my recent trip to Vancouver. Beautiful scenery in the mountains – but too foggy to get any good pictures!

  17. i am watching this channel for month now and i still don't understand the welcome sentence. "dear fellow skylers"? "scalars"? "skilairs"? help pls

  18. We are inches away from a VFX world where green screens are not even needed: Actors can act in any environment and that environment can be removed automatically with little to no human intervention as though it were green screened.

  19. What does this do to photos where people's faces are blurred for privacy? Are those pics now at risk for potentially placing people at risk? e.g. children's faces for court evidence for abuse cases? This could get awkward very quickly? Any picture using simple blurring instead of a flat color to block identity may be at risk.

  20. So, what will this same group do to provide an alternate solution to the artists and companies who have relied on watermarking until now? It feels a little irresponsible to me for these researches to say, "Hey, as one of our example cases, here's how to swipe copyrighted photo content."

  21. Correct me if I'm wrong, but from what I understand by a quick skim of the paper, it has to be trained for every image, which means every image will take more than a few minutes. That makes the applications of the algorithm much more specific. Not as practically exciting as you might think at first just by watching the video or looking at the results. A lot of applications people are suggesting here are not practical due to processing time.

  22. I wonder how effective the watermark removal could be for identity cards and passports, etc.

    I imagine that there is going to have to be a larger push away from photocopies for more serious business.

  23. Shutterstock and watermark is nice and all, but the real question is whether this AI can uncensor some particular Japanese "artwork"..

Leave a Reply

Your email address will not be published. Required fields are marked *