https://www.zhihu.com/question/64470274
http://colah.github.io/posts/2015-08-Understanding-LSTMs/
https://jasdeep06.github.io/posts/Understanding-LSTM-in-Tensorflow-MNIST/
https://stackoverflow.com/questions/37901047/what-is-num-units-in-tensorflow-basiclstmcell#
http://keras-cn.readthedocs.io/en/latest/layers/recurrent_layer/
keras.layers.recurrent.LSTM(units, activation='tanh', recurrent_activation='hard_sigmoid', use_bias=True, kernel_initializer='glorot_uniform', recurrent_initializer='orthogonal', bias_initializer='zeros', unit_forget_bias=True, kernel_regularizer=None, recurrent_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, recurrent_constraint=None, bias_constraint=None, dropout=0.0, recurrent_dropout=0.0)
model = Sequential()
model.add(LSTM(32, return_sequences=True, stateful=True,batch_input_shape=(batch_size, timesteps, data_dim)))
model.add(LSTM(32, return_sequences=True, stateful=True))
model.add(LSTM(32, stateful=True))
model.add(Dense(num_classes, activation='softmax'))
類似上述代碼中,加重黑色數字的含義。
下圖是加了peephole的lstm,用來示例,lstm則需要去掉Ct-1和Ct項。可以看到LSTM里面有幾個參數矩陣,Wf、Wi、Wo都是參數矩陣。我的理解,上面的數字32就是這個參數矩陣的組數。比如初始一組參數矩陣,Wf、Wi、Wo,計算一個lstm值,然后再給一組參數矩陣Wf1、Wi1、Wo1,可以再算一個lstm值,共32組。參考的博客里第一個也是類似的解釋。