-1

I have seen many questions of this problem online, but there are no definitive solutions and my case might be different, as it is with time series data and a LSTM architecture.

model = Sequential()
model.add(LSTM(50, activation='relu', return_sequences=True, input_shape=(n_steps, n_features)))
model.add(LSTM(50, activation='relu'))
model.add(Dense(1, activation = 'sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy',metrics=['accuracy'])

Logs:

Train on 290 samples, validate on 190 samples
Epoch 1/4000
 - 1s - loss: 0.6896 - accuracy: 0.5586 - val_loss: 0.6846 - val_accuracy: 0.6105
Epoch 2/4000
 - 0s - loss: 0.6890 - accuracy: 0.5586 - val_loss: 0.6843 - val_accuracy: 0.6105
Epoch 3/4000
 - 0s - loss: 0.6889 - accuracy: 0.5586 - val_loss: 0.6829 - val_accuracy: 0.6105
Epoch 4/4000
 - 0s - loss: 0.6884 - accuracy: 0.5586 - val_loss: 0.6827 - val_accuracy: 0.6105
Epoch 5/4000
 - 0s - loss: 0.6883 - accuracy: 0.5586 - val_loss: 0.6825 - val_accuracy: 0.6105
Epoch 6/4000
 - 0s - loss: 0.6882 - accuracy: 0.5586 - val_loss: 0.6822 - val_accuracy: 0.6105
Epoch 7/4000
 - 0s - loss: 0.6882 - accuracy: 0.5586 - val_loss: 0.6820 - val_accuracy: 0.6105
Epoch 8/4000
 - 0s - loss: 0.6880 - accuracy: 0.5586 - val_loss: 0.6818 - val_accuracy: 0.6105
Epoch 9/4000
 - 0s - loss: 0.6880 - accuracy: 0.5586 - val_loss: 0.6806 - val_accuracy: 0.6105
Epoch 10/4000
 - 0s - loss: 0.6876 - accuracy: 0.5586 - val_loss: 0.6795 - val_accuracy: 0.6105
Victor Sim
  • 358
  • 1
  • 13

1 Answers1

2

Couple of things to try:

  • Decrease the learning rate.
  • Is the dataset imbalanced? If it is then the model has learned to predict only one class (Which I think is the cause here).
  • Try giving the imbalanced class more weight check this.
  • Try to reset the model, tf.keras.backend.clear_session.
  • Try ensembling, weak learners.
  • Better yet, try a basic time series regression model like ARMA for baseline results.
Kartikey Singh
  • 864
  • 10
  • 23