- 一文道盡傳統圖像降噪方法 link
《Image Denoising with Deep Convolutional Neural Networks》鏈接
《Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising》Dncnn
《Noise2Noise: Learning Image Restoration without Clean Data》Noise2Noise
《Semantic Image Inpainting with Deep Generative Models》鏈接
《Image De-raining Using a Conditional Generative Adversarial Network》鏈接
《DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks》DeblurGAN
課題：如何解決基於生成對抗網絡的去噪任務的領域失配問題？
一、整體思路
1.1 提出問題：如何針對真實的圖像噪聲，設計出有效的圖像去噪網絡(包含網絡框架、損失函數及訓練策略)來解決領域失配問題，是真實圖像去噪領域另一關鍵問題。
1.2 解決辦法：
- 通過對圖像上存在的復雜噪聲進行建模，可以准確刻畫並模擬生成真實圖像噪聲，從而為后續去噪網絡提供大規模可靠的訓練數據對。
- 針對傳統圖像去噪網絡存在的領域失配問題，設計並實現適應於真實圖像去噪任務的網絡框架以及相應的損失函數和訓練策略，從而實現具有較強泛化能力的高性能真實圖像去噪。
二、研究內容
2.1 圖像去噪網絡架構的研究
2.2 圖像去噪網絡的訓練策略研究
2.3 圖像去噪網絡中的損失函數設計研究
三、數據集:
本課題擬采取三種類型的數據對來增強真實圖像去噪的泛化能力。這三種類型的數據對分別為：
3.1 真實的含噪圖像與其對應的干凈圖像
3.2 基於條件生成對抗網絡生成的含噪圖像以及干凈圖像
3.3 原始噪聲圖像以及基於條件生成對抗網絡生成的含噪圖像，需要指出的是，這里輸入𝐲與輸出𝐱 + 𝐧是干凈圖像𝐱的不同含噪圖像。
四、Tips
4.1 三種類型的數據量比例作為超參數進行調整優化。
4.2 損失函數采用 SSIM Loss + L1 Loss
4.3 網絡結構：U-Net、ResNet、DenseNet
【轉】Awesome Image or Video Denoising Algorithms
轉自github (https://hub.fastgit.org/z-bingo/awesome-image-denoising-state-of-the-art)
圖像/視頻去噪算法資源集錦
【導讀】圖像去噪是指減少數字圖像中噪聲的過程。隨着深度學習的發展，也有許多深度學習方法被用於圖像/視頻去噪。本文整理了一些去噪算法與數據集。
Collection of popular and reproducible image denoising works.
I will update the document when I access the new work for image or video denoising. Everyone could pull requests or remind me to update if you access the latest work.
This collection is based on the summary of wenbihan's work.
Contents
- Denoising Algorithms
1.1 Filter
1.2 Sparse Coding
1.3 Effective Prior
1.4 Low Rank
1.5 Deep Learning
1.6 Sparsity and Low-rankness Combined
1.7 Combined with High-Level Tasks
1.8 Image Noise Level Estimation - Benchmark and Dataset
2.1 Novel Benchmark
2.2 Commonly Used Denoising Dataset - Others
3.1 Commonly Used Image Quality Metric Code
Denoising Algorithms
Filter
- NLM [Web] [Code] [PDF]
- A non-local algorithm for image denoising (CVPR 05), Buades et al.
- Image denoising based on non-local means filter and its method noise thresholding (SIVP2013), B. Kumar
- BM3D [Web] [Code] [PDF]
- Image restoration by sparse 3D transform-domain collaborative filtering (SPIE Electronic Imaging 2008), Dabov et al.
- PID [Web] [Code] [PDF]
- Progressive Image Denoising (TIP 2014), C. Knaus et al.
Sparse Coding
- KSVD [Web] [Code] [PDF]
- Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries (TIP 2006), Elad et al.
- LSSC [Web] [Code] [PDF]
- Non-local Sparse Models for Image Restoration (ICCV 2009), Mairal et al.
- NCSR [Web] [Code] [PDF]
- Nonlocally Centralized Sparse Representation for Image Restoration (TIP 2012), Dong et al.
- OCTOBOS [Web] [Code] [PDF]
- Structured Overcomplete Sparsifying Transform Learning with Convergence Guarantees and Applications (IJCV 2015), Wen et al.
- GSR [Web] [Code] [PDF]
- Group-based Sparse Representation for Image Restoration (TIP 2014), Zhang et al.
- TWSC [Web] [Code] [PDF]
- A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising (ECCV 2018), Xu et al.
Effective Prior
- EPLL [Web] [Code] [PDF]
- From Learning Models of Natural Image Patches to Whole Image Restoration (ICCV2011), Zoran et al.
- GHP [[Web]][Code] [PDF]
- Texture Enhanced Image Denoising via Gradient Histogram Preservation (CVPR2013), Zuo et al.
- PGPD [[Web]][Code] [PDF]
- Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising (ICCV 2015), Xu et al.
- PCLR [[Web]][Code] [PDF]
- External Patch Prior Guided Internal Clustering for Image Denoising (ICCV 2015), Chen et al.
Low Rank
- SAIST [Web] [Code by request] [PDF]
- Nonlocal image restoration with bilateral variance estimation: a low-rank approach (TIP2013), Dong et al.
- WNNM [Web] [Code] [PDF]
- Weighted Nuclear Norm Minimization with Application to Image Denoising (CVPR2014), Gu et al.
- Multi-channel WNNM [Web] [Code] [PDF]
- Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising (ICCV 2017), Xu et al.
Deep Learning
-
- Shrinkage Fields for Effective Image Restoration (CVPR 2014), Schmidt et al.
-
- Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration (TPAMI 2016), Chen et al.
-
- Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections (NIPS2016), Mao et al.
-
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising (TIP2017), Zhang et al.
-
- MemNet: A Persistent Memory Network for Image Restoration (ICCV2017), Tai et al.
-
- Learning Pixel-Distribution Prior with Wider Convolution for Image Denoising (Arxiv), Liu et al.
-
- L_p-Norm Constrained Coding With Frank-Wolfe Network (Arxiv), Sun et al.
-
- Non-Local Color Image Denoising with Convolutional Neural Networks (CVPR 2017), Lefkimmiatis.
-
Deep image prior [Web] [Code] [PDF]
- Deep Image Prior (CVPR 2018), Ulyanov et al.
-
- xUnit: Learning a Spatial Activation Function for Efficient Image Restoration (Arxiv), Kligvasser et al.
-
- Universal Denoising Networks : A Novel CNN Architecture for Image Denoising (CVPR 2018), Stamatios Lefkimmiatis.
-
Wavelet-CNN [Web] [Code] [PDF]
- Multi-level Wavelet-CNN for Image Restoration (Arxiv), Liu et al.
-
- FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising (TIP), Zhang et al.
-
- Fully Convolutional Pixel Adaptive Image Denoiser (Arxiv), Cha et al.
-
- Toward Convolutional Blind Denoising of Real Photographs (Arxiv), Guo et al.
-
Noise2Noise [Web] [TF Code] [Keras Unofficial Code] [PDF]
- Noise2Noise: Learning Image Restoration without Clean Data (ICML 2018), Lehtinen et al.
-
- Universal Denoising Networks- A Novel CNN Architecture for Image Denoising (CVPR 2018), Lefkimmiatis.
-
- Neural Nearest Neighbors Networks (NIPS 2018), Plotz et al.
-
- Non-Local Recurrent Network for Image Restoration (NIPS 2018), Liu et al.
-
- Burst Denoising with Kernel Prediction Networks (CVPR 2018), Ben et al.
-
- Multi-Kernel Prediction Networks for Denoising of Burst Images (ArXiv 2019), Marinc et al.
-
- Deep Burst Denoising (ArXiv 2017), Clement et al.
- End-to-End Denoising of Dark Burst Images Using Recurrent Fully Convolutional Networks (ArXiv 2019), Zhao et al.
-
- Image denoising and restoration with CNN-LSTM Encoder Decoder with Direct Attention (ArXiv 2018), Haque et al.
-
- GRDN: Grouped Residual Dense Network for Real Image Denoising and GAN-based Real-world Noise Modeling (CVPR 2019), Kim et al.
-
Deformable KPN [Web] [Code] [PDF]
- Learning Deformable Kernels for Image and Video Denoising (ArXiv 2019), Xu et al.
-
BayerUnify BayerAug [Web] [Code] [PDF]
- Learning Raw Image Denoising With Bayer Pattern Unification and Bayer Preserving Augmentation (CVPR 2019), Liu et al.
-
- A Deep Motion Deblurring Network Based on Per-Pixel Adaptive Kernels With Residual Down-Up and Up-Down Modules (CVPR 2019), Sim et al.
-
- Real Image Denoising with Feature Attention (ArXiv 2019), Anwar et al.
-
- EDVR: Video Restoration With Enhanced Deformable Convolutional Networks (CVPR 2019), Wang et al.
-
- DVDnet: A Fast Network for Deep Video Denoising (ArXiv 2019), Tassano et al.
-
FastDVDNet [Web] [Code] [An Unofficial PyTorch Code] [PDF]
- FastDVDnet: Towards Real-Time Video Denoising Without Explicit Motion Estimation (ArXiv 2019), Tassano et al.
-
- ViDeNN: Deep Blind Video Denoising (ArXiv 2019), Calus et al.
-
Multi-Level Wavelet-CNN [Web] [Code] [PDF]
- Multi-Level Wavelet Convolutional Neural Networks (IEEE Access), Liu et al.
-
- Pyramid Read Image Denoising Network (Arxiv 2019), Zhao et al.
-
- CycleISP: Real Image Restoration via Improved Data Synthesis (CVPR 2020), Zamir et al.
-
- MIRNEt: Learning Enriched Features for Real Image Restoration and Enhancement (ECCV 2020), Zamir et al.
Sparsity and Low-rankness Combined
- STROLLR-2D [PDF] [Code]
- When Sparsity Meets Low-Rankness: Transform Learning With Non-Local Low-Rank Constraint for Image Restoration (ICASSP 2017), Wen et al.
Combined with High-Level Tasks
- Meets High-level Tasks [PDF] [Code]
- When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach (IJCAI 2018), Liu et al.
Image Noise Level Estimation
- SINLE [PDF] [Code] [Slides]
- Single-image Noise Level Estimation for Blind Denoising (TIP 2014), Liu et al.
- CBDNet [Code] [PDF]
- Toward Convolutional Blind Denoising of Real Photographs (Arxiv), Guo et al.
Benchmark and Dataset
Novel Benchmark
- ReNOIR [Web] [Data] [PDF]
- RENOIR - A Dataset for Real Low-Light Image Noise Reduction (Arxiv 2014), Anaya, Barbu.
- PolyU [Web] [Data] [PDF]
- Real-world Noisy Image Denoising: A New Benchmark (Arxiv), Xu et al.
- Nam [Web] [PDF]
- A Holistic Approach to Cross-Channel Image Noise Modeling and its Application to Image Denoising (CVPR 2016), Nam et al.
- Darmstadt (DND) [Web] [Data] [PDF]
- Benchmarking Denoising Algorithms with Real Photographs (CVPR 2017), Plotz et al.
- SIDD [Web]
- A High-Quality Denoising Dataset for Smartphone Cameras.
Commonly Used Denoising Dataset
Others
Commonly Used Image Quality Metric Code
-
PSNR (Peak Signal-to-Noise Ratio) [Wiki] [Matlab Code] [Python Code] [PyTorch Code]
-
SSIM (Structural similarity) [Wiki] [Matlab Code] [Python Code] [PyTorch Code]
-
NIQE (Naturalness Image Quality Evaluator) [Web] [Matlab Code] [Python Code]
去噪論文合集
2019年state-of-the-art
| Model | Published | Code | Title |
|---|---|---|---|
| GRDN | CVPR2019 | Code | GRDN: Grouped Residual Dense Network for Real Image Denoising and GAN-based Real-world Noise Modeling |
| RFCN | arxiv | Code/Web | End-to-End Denoising of Dark Burst Images Using Recurrent Fully Convolutional Networks |
| Deformable KPN | arxiv | Code | Learning Deformable Kernels for Image and Video Denoising |
| BayerUnify BayerAug | CVPR2019 | Code | Learning Raw Image Denoising With Bayer Pattern Unification and Bayer Preserving Augmentation |
| RDU-UD | CVPR2019 | Code | A Deep Motion Deblurring Network Based on Per-Pixel Adaptive Kernels With Residual Down-Up and Up-Down Modules |
| RIDNet | ICCV2019 | Code | Real Image Denoising with Feature Attention |
| PRIDNet | VCIP2019 | Code | Pyramid Real Image Denoising Network |
| RNAN | ICLR2019 | Code | Residual Non-local Attention Networks for Image Restoration |
| VDN | NIPS2019 | Code | Variational Denoising Network: Toward Blind Noise Modeling and Removal |
Image Denoising
https://github.com/wenbihan/reproducible-image-denoising-state-of-the-art/
reproducible-image-denoising-state-of-the-art
Collection of popular and reproducible image denoising works.
Criteria: works must have codes available, and the reproducible results demonstrate state-of-the-art performances.
This collection is inspired by the summary by flyywh
Note: This repo focuses on single image denoising in general, and will exclude multi-frame and video denoising works.
Denoising Algorithms
Filter
- NLM [Web] [Code] [PDF]
- A non-local algorithm for image denoising (CVPR 05), Buades et al.
- Image denoising based on non-local means filter and its method noise thresholding (SIVP2013), B. Kumar
- BM3D [Web] [Code] [PDF]
- Image restoration by sparse 3D transform-domain collaborative filtering (SPIE Electronic Imaging 2008), Dabov et al.
- PID [Web] [Code] [PDF]
- Progressive Image Denoising (TIP 2014), C. Knaus et al.
Sparse Coding
- KSVD [Web] [Code] [PDF]
- Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries (TIP 2006), Elad et al.
- LSSC [Web] [Code] [PDF]
- Non-local Sparse Models for Image Restoration (ICCV 2009), Mairal et al.
- NCSR [Web] [Code] [PDF]
- Nonlocally Centralized Sparse Representation for Image Restoration (TIP 2012), Dong et al.
- OCTOBOS [Web] [Code] [PDF]
- Structured Overcomplete Sparsifying Transform Learning with Convergence Guarantees and Applications (IJCV 2015), Wen et al.
- GSR [Web] [Code] [PDF]
- Group-based Sparse Representation for Image Restoration (TIP 2014), Zhang et al.
- TWSC [Web] [Code] [PDF]
- A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising (ECCV 2018), Xu et al.
Classical External Priors
- EPLL [Web] [Code] [PDF]
- From Learning Models of Natural Image Patches to Whole Image Restoration (ICCV2011), Zoran et al.
- GHP [[Web]][Code] [PDF]
- Texture Enhanced Image Denoising via Gradient Histogram Preservation (CVPR2013), Zuo et al.
- PGPD [[Web]][Code] [PDF]
- Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising (ICCV 2015), Xu et al.
- PCLR [[Web]][Code] [PDF]
- External Patch Prior Guided Internal Clustering for Image Denoising (ICCV 2015), Chen et al.
Low Rank
- SAIST [Web] [Code by request] [PDF]
- Nonlocal image restoration with bilateral variance estimation: a low-rank approach (TIP2013), Dong et al.
- WNNM [Web] [Code] [PDF]
- Weighted Nuclear Norm Minimization with Application to Image Denoising (CVPR2014), Gu et al.
- Multi-channel WNNM [Web] [Code] [PDF]
- Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising (ICCV 2017), Xu et al.
Deep Denoising
- SF [Web] [Code] [PDF]
- Shrinkage Fields for Effective Image Restoration (CVPR 2014), Schmidt et al.
- TNRD [Web] [Code] [PDF]
- Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration (TPAMI 2016), Chen et al.
- RED [Web] [Code] [PDF]
- Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections (NIPS2016), Mao et al.
- DnCNN [Web] [Code] [PDF]
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising (TIP2017), Zhang et al.
- MemNet [Web] [Code] [PDF]
- MemNet: A Persistent Memory Network for Image Restoration (ICCV2017), Tai et al.
- WIN [Web] [Code] [PDF]
- Learning Pixel-Distribution Prior with Wider Convolution for Image Denoising (Arxiv), Liu et al.
- F-W Net [Web] [Code] [PDF]
- L_p-Norm Constrained Coding With Frank-Wolfe Network (Arxiv), Sun et al.
- NLCNN [Web] [Code] [PDF]
- Non-Local Color Image Denoising with Convolutional Neural Networks (CVPR 2017), Lefkimmiatis.
- xUnit [Web] [Code] [PDF]
- xUnit: Learning a Spatial Activation Function for Efficient Image Restoration (Arxiv), Kligvasser et al.
- UDNet [Web] [Code] [PDF]
- Universal Denoising Networks : A Novel CNN Architecture for Image Denoising (CVPR 2018), Stamatios Lefkimmiatis.
- Wavelet-CNN [Web] [Code] [PDF]
- Multi-level Wavelet-CNN for Image Restoration (Arxiv), Liu et al.
- FFDNet [Web] [Code] [PDF]
- FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising (TIP), Zhang et al.
- FC-AIDE [Web] [Code] [PDF]
- Fully Convolutional Pixel Adaptive Image Denoiser (Arxiv), Cha et al.
- CBDNet [Web] [Code] [PDF]
- Toward Convolutional Blind Denoising of Real Photographs (Arxiv), Guo et al.
- UDN [Web] [Code] [PDF]
- Universal Denoising Networks- A Novel CNN Architecture for Image Denoising (CVPR 2018), Lefkimmiatis.
- N3 [Web] [Code] [PDF]
- Neural Nearest Neighbors Networks (NIPS 2018), Plotz et al.
- NLRN [Web] [Code] [PDF]
- Non-Local Recurrent Network for Image Restoration (NIPS 2018), Liu et al.
- RDN+ [Web] [Code] [PDF]
- Residual Dense Network for Image Restoration (CVPR 2018), Zhang et al.
- FOCNet [Web] [Code] [PDF]
- FOCNet: A Fractional Optimal Control Network for Image Denoising (CVPR 2019), Jia et al.
Unsupervised / Weakly-Supervised Deep Denoising
- Noise2Noise [Web] [TF Code] [Keras Unofficial Code] [PDF]
- Noise2Noise: Learning Image Restoration without Clean Data (ICML 2018), Lehtinen et al.
- DIP [Web] [Code] [PDF]
- Deep Image Prior (CVPR 2018), Ulyanov et al.
- Noise2Void [Web] [Code] [PDF]
- Learning Denoising from Single Noisy Images (CVPR 2019), Krull et al.
- Noise2Self [Web] [Code] [PDF]
- LNoise2Self: Blind Denoising by Self-Supervision (ICML 2019), Batson and Royer
- Self-Supervised Denoising [Web] [Code] [PDF]
- High-Quality Self-Supervised Deep Image Denoising (NIPS 2019), Laine et al.
Real Noise Removal
- RIDNet [Web] [Code] [PDF]
- Real Image Denoising with Feature Attention (ICCV 2019), Anwar and Barnes.
- CBDNet [Web] [Code] [PDF]
- Real Image Denoising with Feature Attention (CVPR 2019), Guo et al.
- VDNNet [Web] [Code] [PDF]
- Variational Denoising Network: Toward Blind Noise Modeling and Removal (NIPS 2019), Yue et al.
Hybrid Model for Denoising
- STROLLR [PDF] [Code]
- When Sparsity Meets Low-Rankness: Transform Learning With Non-Local Low-Rank Constraint for Image Restoration (ICASSP 2017), Wen et al.
- Meets High-level Tasks [PDF] [Code]
- When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach (IJCAI 2018), Liu et al.
- USA [PDF] [Code]
- Segmentation-aware Image Denoising Without Knowing True Segmentation (Arxiv), Wang et al.
Image Noise Level Estimation
- SINLE [PDF] [Code] [Slides]
- Single-image Noise Level Estimation for Blind Denoising (TIP 2014), Liu et al.
Novel Benchmark
- ReNOIR [Web] [Data] [PDF]
- RENOIR - A Dataset for Real Low-Light Image Noise Reduction (Arxiv 2014), Anaya, Barbu.
- Darmstadt [Web] [Data] [PDF]
- Benchmarking Denoising Algorithms with Real Photographs (CVPR 2017), Tobias Plotz, Stefan Roth.
- PolyU [Web] [Data] [PDF]
- Real-world Noisy Image Denoising: A New Benchmark (Arxiv), Xu et al.
Commonly Used Denoising Dataset
Commonly Used Image Quality Metrics
- PSNR (Peak Signal-to-Noise Ratio) [Wiki] [Matlab Code] [Python Code]
- SSIM (Structural similarity) [Wiki] [Matlab Code] [Python Code]
- NIQE (Naturalness Image Quality Evaluator) [Web] [Matlab Code] [Python Code]