CNN(卷積神經網絡)、RNN(循環神經網絡)、DNN(深度神經網絡)

CNN(卷積神經網絡)、RNN(循環神經網絡)、DNN(深度神經網絡)的內部網絡結構有什么區別？

DNN以神經網絡為載體，重在深度，可以說是一個統稱。
RNN，回歸型網絡，用於序列數據，並且有了一定的記憶效應，輔之以lstm。
CNN應該側重空間映射，圖像數據尤為貼合此場景。

DNN以神經網絡為載體，重在深度，可以說是一個統稱。
RNN，回歸型網絡，用於序列數據，並且有了一定的記憶效應，輔之以lstm。
CNN應該側重空間映射，圖像數據尤為貼合此場景。

Stanford University CS231n: Convolutional Neural Networks for Visual Recognition

作者：肖天睿
鏈接：https://www.zhihu.com/question/34681168/answer/84185831
來源：知乎
著作權歸作者所有。商業轉載請聯系作者獲得授權，非商業轉載請注明出處。

Artificial neural networks use networks of activation units (hidden units) to map inputs to outputs. The concept of deep learning applied to this model allows the model to have multiple layers of hidden units where we feed output from the previous layers. However, dense connections between the layers is not efficient, so people developed models that perform better for specific tasks.

The whole "convolution" in convolutional neural networks is essentially based on the fact that we're lazy and want to exploit spatial relationships in images. This is a huge deal because we can then group small patches of pixels and effectively "downsample" the image while training multiple instances of small detectors with those patches. Then a CNN just moves those filters around the entire image in a convolution. The outputs are then collected in a pooling layer. The pooling layer is again a down-sampling of the previous feature map. If we have activity on an output for filter a, we don't necessarily care if the activity is for (x, y) or (x+1, y), (x, y+1) or (x+1, y+1), so long as we have activity. So we often just take the highest value of activity on a small grid in the feature map called max pooling.

If you think about it from an abstract perspective, the convolution part of a CNN is effectively doing a reasonable way of dimensionality reduction. After a while you can flatten the image and process it through layers in a dense network. Remember to use dropout layers! (because our guys wrote that paper :P)

I won't talk about RNN for now because I don't have enough experience working with them and according to my colleague nobody really knows what's going on inside an LSTM...but they're insanely popular for data with time dependencies.

Let's talk RNN. Recurrent networks are basically neural networks that evolve through time. Rather than exploiting spatial locality, they exploit sequential, or temporal locality. Each iteration in an RNN takes an input and it's previous hidden state, and produces some new hidden state. The weights are shared in each level, but we can unroll an RNN through time and get your everyday neural net. Theoretically RNN has the capacity to store information from as long ago as possible, but historically people always had problems with the gradients vanishing as we go back further in time, meaning that the model can't be differentiated numerically and thus cannot be trained with backprop. This was later solved in the proposal of the LSTM architecture and subsequent work, and now we train RNNs with BPTT (backpropagation through time). Here's a link that explains LSTMs really well: http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Since then RNN has been applied in many areas of AI, and many are now designing RNN with the ability to extract specific information (read: features) from its training examples with attention-based models.

https://www.zhihu.com/question/34681168