0

I get different results for mse. During trainig I get 0.296 after my last training epoch and when I evaluate my model I get 0.112. Does any one know why that is so?

Here is the code:

model = Sequential()
model.add(Dropout(0.2))
model.add(LSTM(100, return_sequences=True,batch_input_shape=(batch_size,look_back,dim_x)))
model.add(Dropout(0.2))
model.add(LSTM(150,return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(100,return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(50,return_sequences=False))
model.add(Dropout(0.2))
model.add(Dense(1, activation='linear'))

model.compile(loss='mean_squared_error', optimizer='adam')
history=model.fit(x_train_r, y_train_r, validation_data=(x_test_r, y_test_r),\
                  epochs=epochs, batch_size=batch_size, callbacks=[es])

score_test = model.evaluate(x_test_r, y_test_r,batch_size=batch_size)
score_train = model.evaluate(x_train_r, y_train_r,batch_size=batch_size)

print("Score Training Data:")
print(score_train)

Batch size and everything stays the same. Does anyone knows why I get so different results for mse?

today
  • 32,602
  • 8
  • 95
  • 115
D.Luipers
  • 127
  • 1
  • 11
  • Could be a number of reasons: your training set is not representative of your test set, you are overfitting to the training set, you have a class imbalance, etc. There are any number of reasons that could be driving the difference. – James Nov 05 '18 at 14:26
  • but what is more representative for the mse of my training set? I know that there is a difference between my test and training set but i use the same data for training so I am real curious where this big difference comes from. Do you know how i can figure it out? – D.Luipers Nov 05 '18 at 14:29

1 Answers1

1

The reason for the discrepancy between the training loss and the loss obtained on the training data after the training is finished, is the existence of Dropout layer in the model. That's because this layer has different behavior during training and inference time. As I have mentioned in another answer, you can make this behavior the same either by passing training=True to dropout call, or using K.learning_phase() flag and a backend function.

today
  • 32,602
  • 8
  • 95
  • 115