I am going through the following blog on LSTM neural network:http://machinelearningmastery.com/understanding-stateful-lstm-recurrent-neural-networks-python-keras/
The author reshapes the input vector X as [samples, time steps, features] for different configuration of LSTMs.
The author writes
Indeed, the sequences of letters are time steps of one feature rather than one time step of separate features. We have given more context to the network, but not more sequence as it expected
What does this mean?
=========================================
I found this just below the [samples, time_steps, features] you are concerned with.
X = numpy.reshape(dataX, (len(dataX), seq_length, 1))
Samples - This is the len(dataX), or the amount of data points you have.
Time steps - This is equivalent to the amount of time steps you run your recurrent neural network. If you want your network to have memory of 60 characters, this number should be 60.
Features - this is the amount of features in every time step. If you are processing pictures, this is the amount of pixels. In this case you seem to have 1 feature per time step.
ASK：
can you explain the difference between : X = numpy.reshape(dataX, (len(dataX), 3, 1)) and X = numpy.reshape(dataX, (len(dataX), 1, 3)) How does this affect the lstm?
ANSWER：
(len(dataX), 3, 1) runs LSTM for 3 iterations, inputting a input vector of shape (1,). (len(dataX), 1, 3) runs LSTM for 1 iteration. Which means that it is quite useless to even have recurrent connections since there can't be any feedback from previous iterations. In this case input shape to RNN is of shape (3,)。
其實TimeSteps就是unfold的意思，就是tensorflow中的 NUM_STEPS 的意思。
Features其實就是輸入的維度，也就是特征，一個維度一個特征。
The LSTM networks are stateful. They should be able to learn the whole alphabet sequence, but by default the Keras implementation resets the network state after each training batch.
LSTM網絡本是狀態傳遞的，這種網絡本應該是學習整個序列的; 但是keras的默認實現卻會在每個batch訓練結束時重置網絡的狀態。