Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
發表在2017 TIP。
摘要：
Discriminative model learning for image denoising has been recently attracting considerable attentions due to its favorable denoising performance. In this paper, we take one step forward by investigating the construction of feed-forward denoising convolutional neural networks (DnCNNs) to embrace the progress in very deep architecture, learning algorithm, and regularization method into image denoising. Specifically, residual learning and batch normalization are utilized to speed up the training process as well as boost the denoising performance. Different from the existing discriminative denoising models which usually train a specific model for additive white Gaussian noise (AWGN) at a certain noise level, our DnCNN model is able to handle Gaussian denoising with unknown noise level (i.e., blind Gaussian denoising). With the residual learning strategy, DnCNN implicitly removes the latent clean image in the hidden layers. This property motivates us to train a single DnCNN model to tackle with several general image denoising tasks such as Gaussian denoising, single image super-resolution and JPEG image deblocking. Our extensive experiments demonstrate that our DnCNN model can not only exhibit high effectiveness in several general image denoising tasks, but also be efficiently implemented by benefiting from GPU computing.
結論：
In this paper, a deep convolutional neural network was proposed for image denoising, where residual learning is adopted to separating noise from noisy observation. The batch normalization and residual learning are integrated to speed up the training process as well as boost the denoising performance. Unlike traditional discriminative models which train specific models for certain noise levels, our single DnCNN model has the capacity to handle the blind Gaussian denoising with unknown noise level. Moreover, we showed the feasibility to train a single DnCNN model to handle three general image denoising tasks, including Gaussian denoising with unknown noise level, single image super-resolution with multiple upscaling factors, and JPEG image deblocking with different quality factors. Extensive experimental results demonstrated that the proposed method not only produces favorable image denoising performance quantitatively and qualitatively but also has promising run time by GPU implementation.
要點：
-
DnCNN可以處理未知程度的高斯噪聲，即實現blind Gaussian denoising。
-
作者還嘗試了用一個DnCNN，同時處理未知高斯噪聲、多尺度超分辨和未知QF的JPEG圖像壓縮。
優點：
-
應該是第一篇將DNN用於高斯去噪的網絡，通過混合訓練集，可以實現盲去噪。（CNN最早在[23]被用於圖像去噪）
-
打破了傳統方法對模型或圖像先驗的形式約束，讓網絡自己學習先驗項。
-
作者通過實驗發現，BN和殘差學習是互助的。
故事背景
首先，作者回顧了高斯去噪的歷史工作，主要是一些傳統方法，例如nonlocal self-similarity models和馬爾科夫隨機場模型等。其中，NSS方法是SOTA常客，包括BM3D，NCSR等。
盡管如此，這些方法普遍需要優化、耗時；此外，這些方法都需要一些主觀設計的參數。
這就引出了判別式學習方法。例如，[14]提出將隨機場模型和展開的半二次優化模型在一個學習框架中統一起來。又比如，[15][16]提出的TNRD模型，可以學習一種調整的專家場（modified fields of experts）圖像先驗。這些方法普遍只學習一種特定的先驗，並且只適用於某一個特定的噪聲程度。（批注：實際上就是先驗項的形式被提前規定好了，因此先驗類型是固定的）
網絡結構
整體上，網絡采用的是VGG結構，改造：
-
所有卷積核都設為\(3 \times 3\)。
-
移除所有池化層。
在論文第三頁，作者說明了如何根據有效感受野來設定網絡深度。有興趣的可以看一看。最終作者設定DnCNN為17層。對於盲去噪及其他任務，作者設為20層。
作者加入了一系列的關於CNN的優化手段：
-
殘差學習：讓CNN建模潛在干凈圖像與有噪圖像的差異。換句話說，CNN需要從有噪圖像中剝離出干凈圖像（找出自然無損圖像的本質特征）。
-
BN：加快和穩定訓練過程。實驗發現，BN和殘差學習是彼此受益的。
注意幾點：
-
第一層和最后一層沒有使用BN。
-
最后一層不使用ReLU非線性激活。
-
每一層卷積都產生64個通道，最后輸出單通道的灰度圖像，理想狀況下為噪聲圖像。
最后值得一提的是，在傳統方法中，由於通常是基於塊處理，因此會產生比較明顯的塊效應。但在DnCNN中，我們采用的是補零卷積的方法，實驗發現補零不會產生塊效應。
BN和殘差學習
作者采用實驗的方式，探究BN和殘差學習的組合效果：
如圖，無論是SGD還是Adam優化方法，RL和BN的強強聯手都能顯著提升網絡性能。作者對此的解釋是：
Actually, one can notice that in Gaussian denoising the residual image and batch normalization are both associated with the Gaussian distribution. It is very likely that residual learning and batch normalization can benefit from each other for Gaussian denoising. This point can be further validated by the following analyses.
拓展到其他任務
盲去噪：作者在訓練集中混合了噪聲標准差從0到55不等的有噪圖像，來訓練單一的DnCNN模型。
去JPEG塊效應：同理。
超分辨：先插值升采樣，然后再執行相同的流程。
實驗略。這篇文章在現在看來還是相對簡單的。