Residual Dense Network for Image Super-Resolution
發表在2018年CVPR。
摘要和結論都在強調方法的優勢。我們還是先從RDN的結構看起，再理解它的背景和思想。
Residual dense block & network
乍一看，這種block結構就是在內部采用了稠密連接，在外部采用殘差學習。並且，RDN在全局上也是類似的設計：內部稠密，整體殘差。無論是RDB還是RDN，內部都同時采用了\(3 \times 3\)和\(1 \times 1\)卷積。
我們來看看作者怎么解釋這種設計的合理性，以及實驗是否驗證了其有效性。
和DenseNet的不同
-
在RDN和RDB中，我們取消了BN和池化層，因為作者認為它們不僅消耗資源，而且阻礙了網絡學習（批注：在一些去噪工作中，有些作者也發現了BN無益於除高斯噪聲以外的噪聲去除）。
-
在DenseNet中，不同的block之間需要過渡層，但在這里采用\(1 \times 1\)卷積，即所謂的local feature fusion。（批注：本質是一樣的，只不過過渡層多了BN和池化層，因為需要服務於高層視覺任務——圖像分類）
-
全局和局部都有殘差學習，而DenseNet中沒有。這種局部殘差連接，使得上一個RDB的輸出，可以直接聯系至當前RDB的輸出結果。這就是作者所謂的contiguous memory（CM）。
算了，看完解釋，我已經不想看實驗了，因為還是比較trick的（沒有太多能讓人high的思想點，解釋有點勉強）。我們回頭看看摘要和結論吧。
摘要和結論
摘要：
A very deep convolutional neural network (CNN) has recently achieved great success for image super-resolution (SR) and offered hierarchical features as well. However, most deep CNN based SR models do not make full use of the hierarchical features from the original low-resolution (LR) images, thereby achieving relatively-low performance. In this paper, we propose a novel residual dense network (RDN) to address this problem in image SR. We fully exploit the hierarchical features from all the convolutional layers. Specifically, we propose residual dense block (RDB) to extract abundant local features via dense connected convolutional layers. RDB further allows direct connections from the state of preceding RDB to all the layers of current RDB, leading to a contiguous memory (CM) mechanism. Local feature fusion in RDB is then used to adaptively learn more effective features from preceding and current local features and stabilizes the training of wider network. After fully obtaining dense local features, we use global feature fusion to jointly and adaptively learn global hierarchical features in a holistic way. Experiments on benchmark datasets with different degradation models show that our RDN achieves favorable performance against state-of-the-art methods.
結論：
In this paper, we proposed a very deep residual dense network (RDN) for image SR, where residual dense block (RDB) serves as the basic build module. In each RDB, the dense connections between each layers allow full usage of local layers. The local feature fusion (LFF) not only stabilizes the training wider network, but also adaptively controls the preservation of information from current and preceding RDBs. RDB further allows direct connections between the preceding RDB and each layer of current block, leading to a contiguous memory (CM) mechanism. The local residual leaning (LRL) further improves the flow of information and gradient. Moreover, we propose global feature fusion (GFF) to extract hierarchical features in the LR space. By fully using local and global features, our RDN leads to a dense feature fusion and deep supervision. We use the same RDN structure to handle three degradation models and real-world data. Extensive benchmark evaluations well demonstrate that our RDN achieves superiority over state-of-theart methods.
我來翻譯一下：
-
在每個RDB內部，都有一個全局短連接；因此上一個RDB的輸出，會直接送到當前RDB的輸出端；這就是作者所謂的連續記憶（contiguous memory）機制。
-
每個RDB之間采用了\(1 \times 1\)卷積，作者將其稱為local feature fusion；這不就是大家都在用的、降低通道數的方法嘛，有點故弄玄虛哦。作者還強調：該LFF可以穩定寬網絡的訓練。實際上，DenseNet為了降低計算量，特地讓網絡更窄。這是在增大冗余（增強泛化能力）和減小計算量之間的權衡，詳情參見我的博客。