Encoder Decoder for time series forecasting

Question

I want to predict for 7 days from training size of 55 days. I tried to apply models given here and here, but I am getting output value for all 7 days as 1.

I am also confused about how to give time series as input to encoder decoder and it's code, I tried based on my understanding.

model.add(LSTM(150, input_shape=(None, 1)))
model.add(RepeatVector(8))
model.add(LSTM(150, return_sequences=True))
model.add(TimeDistributed(Dense(1, activation='softmax')))
model.compile(loss='mse', optimizer='adam')

for i in range(7):
    x=df[i*7:(i+1)*7]
    y=df[(i+1)*7:(i+2)*7]
    x=np.array(x)
    x=np.insert(x,0,len(x))
    x=x.reshape(1,len(x),1)
    y=np.array(y)
    y=np.insert(y,0,len(y))
    y=y.reshape(1,len(y),1)
    model.fit(x, y, epochs=1, verbose=2)

after training I am predicting from entire train sequence for 7 days.

second I tried from link 2

#functions define_models and predict_sequence same as link
for i in range(0,47):
    x1=df[i:i+7]
    print(len(x1))
    x2=df[i+1:i+8]
    print(len(x2))
    y=df[i+1:i+8]
    x1=np.array(x1)
    x1=np.insert(x1,0,len(x1))
    print(len(x1))
    x1=x1.reshape(len(x1),1,1)
    x2=np.array(x2)
    x2=np.insert(x2,0,0)
    print(len(x2))
    x2=x2.reshape(len(x2),1,1)
    y=np.array(y)
    y=np.insert(y,0,len(y))
    y=y.reshape(len(y),1,1)
    model.fit([x1,x2],y,epochs=1)

this is also giving output as 1. I dont know exactly what x2 should be here.

Please correct me where I am wrong.

Have you tried out simpler forecasting approaches? When you say you have data for 55 days, does that mean you have 55 data points in you training series? Do you have additional time series variables? — mloning, Jun 14 '20 at 12:09
@mloning I have tried other approaches like arima, sarima, xgboost and lstm and I have features for this time series. But for understanding I am just trying without features. I am new to these things so taking one step at a time. I am confused about how should be the input format for encoder decoder. — Dee, Jun 15 '20 at 04:46

ahmad asadi · Answer 1 · 2020-06-16T08:49:54.597

The first problem is that to train a deep network you should do the following steps:

Create a clear dataset. By a "clear dataset" I mean an instance of tf.Dataset object. To create an instance of tf.Dataset you should first organize your dataset in a NumPy array with shape (Maximum sequence length, Batch size, Size of each record). In your case, the size of the X array which contains the training data should be (7, 1, 1), and the Y array which contains the labels of the training data should be (7,1).
After organizing the data according to the explained format, you can create an instance of tf.Dataset using the function tf.Dataset.from_tensor_slices()
You should use the model.fit() function using the created tf.Dataset instance and specifying a suitable number of epochs which is more than 1. The parameter specifies the number of times the network should iterate on the dataset to be trained. The value of this parameter is somehow arbitrary, but, you should try different values to reach the best one fitting your problem.

Note that using this process you do not need to make a for-loop anymore. The loop will be executed inside of the model.fit function.

For more information about how to implement and train an encoder-decoder model in TensorFlow take a look at the official sample for neural machine translation.

Can you please explain me which code is correct for encoder decoder, and how should my input be? — Dee, Jun 15 '20 at 04:47
None of your codes make sense. You should provide a well-formed features and labels array for the training and then convert them to `tf.Dataset` object. Then you can use `model.fit`. The true shape of the LSTM input is setting `Time. In your case, It would be <7,1, 1> and <7,1> respectively. Take a look at tensorflow's encoder-decoder code sample here: https://www.tensorflow.org/tutorials/text/nmt_with_attention — ahmad asadi, Jun 16 '20 at 08:48

Encoder Decoder for time series forecasting

1 Answers1