LSTM 是優秀的循環神經網絡 (RNN) 結構，而 LSTM 在結構上也比較復雜，對 RNN 和 LSTM 還稍有疑問的朋友可以參考：Recurrent Neural Networks vs LSTM【參考李宏毅老師的講課PPT內容】
這里我們將要使用 Keras 搭建 LSTM.Keras 封裝了一些優秀的深度學習框架的底層實現，使用起來相當簡潔，甚至不需要深度學習的理論知識，你都可以輕松快速的搭建你的深度學習網絡，強烈推薦給剛入門深度學習的同學使用，當然我也是還沒入門的那個。Keras：https://keras.io/，keras的backend有，theano，TensorFlow、CNTk，這里我使用的是 TensorFlow。
下面我們就開始搭建 LSTM & BiLSTM，實現 mnist 數據的分類。
一、加載包和定義參數
mnist 的 image 是 28*28 的 shape，我們定義 LSTM 的 input 為 (28,)，將 image 一行一行地輸入到 LSTM 的 cell 中，這樣 time_step 就是 28，表示一個 image 有 28 行，LSTM 的 output 是 30 個。
from tensorflow import keras import mnist from keras.layers import Dense, LSTM, Bidirectional from keras.utils import to_categorical from keras.models import Sequential # parameters for LSTM nb_lstm_outputs = 30 # 輸出神經元個數 nb_time_steps = 28 # 時間序列的長度 nb_input_vectors = 28 # 每個輸入序列的向量維度
二、數據預處理
特別注意 label 要使用 one_hot encoding，x_train 的 shape 為 (60000, 28,28）
# data preprocessing x_train = mnist.train_images() y_train = mnist.train_labels() x_test = mnist.test_images() y_test = mnist.test_labels() # Nomalize the images x_train = (x_train / 255) - 0.5 x_test = (x_test / 255) - 0.5 # one_hot encoding y_train = to_categorical(y_train, num_classes=10) y_test = to_categorical(y_test, num_classes=10)
三、搭建模型 (LSTM, BiLSTM)
keras 搭建模型相當簡單，只需要在 Sequential 容器中不斷 add 新的 layer 就可以了。
# building model model = Sequential() model.add(LSTM(units=nb_lstm_outputs, input_shape=(nb_time_steps, nb_input_vectors))) model.add(Dense(10, activation='softmax'))
BiLSTM 模型搭建如下：具體實現方法差別不大
# building model
model = Sequential()
model.add(
Bidirectional(
LSTM(
units=nb_lstm_outputs,
return_sequences=True
),
input_shape=(nb_time_steps, nb_input_vectors)
)
)
model.add(
Bidirectional(
LSTM(units=nb_lstm_outputs)
)
)
model.add(
Dense(
10,
activation='softmax'
)
)
四、compile
模型 compile，指定 loss function，optimizer，metrics
# compile:loss, optimizer, metrics
model.compile(
loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy']
)
五、summary
可以使用 model.summary() 來查看你的神經網絡的架構和參數量等信息。
model.summary() output: _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= lstm_1 (LSTM) (None, 30) 7080 _________________________________________________________________ dense_1 (Dense) (None, 10) 310 ================================================================= Total params: 7,390 Trainable params: 7,390 Non-trainable params: 0 _________________________________________________________________
BiLSTM 結果如下：多了一層 layer
_________________________________________________________________ Layer (type) Output Shape Param # ================================================================= bidirectional_1 (Bidirection (None, 28, 60) 14160 _________________________________________________________________ bidirectional_2 (Bidirection (None, 60) 21840 _________________________________________________________________ dense_2 (Dense) (None, 10) 610 ================================================================= Total params: 36,610 Trainable params: 36,610 Non-trainable params: 0 _________________________________________________________________
六、train
模型訓練，需要指定，epochs 訓練的輪次數，batch_size。
model.fit(
x_train,
y_train,
epochs=20,
batch_size=128,
verbose=1
)
output:
Epoch 1/20
60000/60000 [==============================] - 11s 184us/step - loss: 0.9702 - acc: 0.6919
Epoch 2/20
60000/60000 [==============================] - 9s 152us/step - loss: 0.3681 - acc: 0.8921
Epoch 3/20
60000/60000 [==============================] - 9s 143us/step - loss: 0.2505 - acc: 0.9263
Epoch 4/20
60000/60000 [==============================] - 9s 147us/step - loss: 0.1985 - acc: 0.9411
Epoch 5/20
60000/60000 [==============================] - 9s 156us/step - loss: 0.1673 - acc: 0.9508
Epoch 6/20
60000/60000 [==============================] - 10s 163us/step - loss: 0.1473 - acc: 0.9563
Epoch 7/20
60000/60000 [==============================] - 10s 162us/step - loss: 0.1311 - acc: 0.9605
Epoch 8/20
60000/60000 [==============================] - 10s 162us/step - loss: 0.1176 - acc: 0.9650
Epoch 9/20
60000/60000 [==============================] - 10s 167us/step - loss: 0.1054 - acc: 0.9688
Epoch 10/20
60000/60000 [==============================] - 10s 165us/step - loss: 0.0991 - acc: 0.9702
Epoch 11/20
60000/60000 [==============================] - 10s 164us/step - loss: 0.0899 - acc: 0.9730
Epoch 12/20
60000/60000 [==============================] - 10s 169us/step - loss: 0.0857 - acc: 0.9741
Epoch 13/20
60000/60000 [==============================] - 10s 166us/step - loss: 0.0781 - acc: 0.9758
Epoch 14/20
60000/60000 [==============================] - 10s 167us/step - loss: 0.0740 - acc: 0.9776
Epoch 15/20
60000/60000 [==============================] - 10s 172us/step - loss: 0.0697 - acc: 0.9786
Epoch 16/20
60000/60000 [==============================] - 10s 171us/step - loss: 0.0678 - acc: 0.9795
Epoch 17/20
60000/60000 [==============================] - 10s 170us/step - loss: 0.0639 - acc: 0.9798
Epoch 18/20
60000/60000 [==============================] - 10s 169us/step - loss: 0.0589 - acc: 0.9817
Epoch 19/20
60000/60000 [==============================] - 10s 172us/step - loss: 0.0597 - acc: 0.9817
Epoch 20/20
60000/60000 [==============================] - 10s 168us/step - loss: 0.0558 - acc: 0.9825
BiLSTM 結果如下：結果更好
Epoch 1/20 60000/60000 [==============================] - 46s 767us/step - loss: 0.6845 - acc: 0.7782 Epoch 2/20 60000/60000 [==============================] - 48s 799us/step - loss: 0.1843 - acc: 0.9435 Epoch 3/20 60000/60000 [==============================] - 45s 751us/step - loss: 0.1241 - acc: 0.9627 Epoch 4/20 60000/60000 [==============================] - 45s 747us/step - loss: 0.0956 - acc: 0.9712 Epoch 5/20 60000/60000 [==============================] - 46s 766us/step - loss: 0.0806 - acc: 0.9754 Epoch 6/20 60000/60000 [==============================] - 46s 771us/step - loss: 0.0667 - acc: 0.9793 Epoch 7/20 60000/60000 [==============================] - 45s 754us/step - loss: 0.0584 - acc: 0.9820 Epoch 8/20 60000/60000 [==============================] - 44s 741us/step - loss: 0.0513 - acc: 0.9835 Epoch 9/20 60000/60000 [==============================] - 45s 742us/step - loss: 0.0445 - acc: 0.9863 Epoch 10/20 60000/60000 [==============================] - 46s 767us/step - loss: 0.0419 - acc: 0.9874 Epoch 11/20 60000/60000 [==============================] - 45s 755us/step - loss: 0.0378 - acc: 0.9885 Epoch 12/20 60000/60000 [==============================] - 46s 758us/step - loss: 0.0332 - acc: 0.9894 Epoch 13/20 60000/60000 [==============================] - 45s 750us/step - loss: 0.0318 - acc: 0.9894 Epoch 14/20 60000/60000 [==============================] - 45s 756us/step - loss: 0.0279 - acc: 0.9911 Epoch 15/20 60000/60000 [==============================] - 45s 745us/step - loss: 0.0262 - acc: 0.9917 Epoch 16/20 60000/60000 [==============================] - 45s 758us/step - loss: 0.0258 - acc: 0.9916 Epoch 17/20 60000/60000 [==============================] - 47s 791us/step - loss: 0.0226 - acc: 0.9923 Epoch 18/20 60000/60000 [==============================] - 47s 791us/step - loss: 0.0223 - acc: 0.9930 Epoch 19/20 60000/60000 [==============================] - 46s 773us/step - loss: 0.0179 - acc: 0.9943 Epoch 20/20 60000/60000 [==============================] - 45s 747us/step - loss: 0.0199 - acc: 0.9935
七、evaluate
通過 model.evaluate() 來實現。
score = model.evaluate(x_test, y_test, batch_size=128, verbose=1) print(score) output: 10000/10000 [==============================] - 0s 49us/step [0.06827456439994276, 0.9802]
BiLSTM 結果：更好
10000/10000 [==============================] - 2s 250us/step [0.055307343754824254, 0.9838]