1

I am doing an LSTM based model with data sets. I used the standardization method to put my data in the interval (0, 1) like this:

scaler = MinMaxScaler(feature_range=(0, 1))
scaler = scaler.fit(df_train)

df_train = scaler.transform(df_train)
df_test = scaler.transform(df_test)

Just after the standardization, I reshape my data to obtain:

x_train (3125, 50, 5)
y_train (3125, 1)
x_test (1000, 50, 5)
y_test (1000, 1)

The model works and I get predictions, but when I want to inverse_transform my data I get the following error:

yhat = model.predict(x_test)
yhat = scaler.inverse_transform(yhat)

ValueError: non-broadcastable output operand with shape (1000,1) doesn't match the broadcast shape (1000,5)

So I tried this sample code by changing the name corresponding to my variables:

yhat = model.predict(test_X)
test_X = test_X.reshape((test_X.shape[0], test_X.shape[2]))

# invert scaling for forecast
inv_yhat = concatenate((yhat, test_X[:, 1:]), axis=1)
inv_yhat = scaler.inverse_transform(inv_yhat)
inv_yhat = inv_yhat[:,0]

# invert scaling for actual
test_y = test_y.reshape((len(test_y), 1))
inv_y = concatenate((test_y, test_X[:, 1:]), axis=1)
inv_y = scaler.inverse_transform(inv_y)
inv_y = inv_y[:,0]

# calculate RMSE
rmse = sqrt(mean_squared_error(inv_y, inv_yhat))
print('Test RMSE: %.3f' % rmse)

but I get this error in line 2:

ValueError: cannot reshape array of size 250000 into shape (1000,5)

Flavia Giammarino
  • 7,987
  • 11
  • 30
  • 40
Nadhir
  • 528
  • 3
  • 12
  • 2
    What you are doing is performing `inverse_transform` on predicted labels (what the model predicts after processing each sample) And in my opinion you are expecting to get **Unnormalized** test data samples back. the error is stating that the `scaler` module expects you to provide it with a 5d vector instead of 1d scaler label. – meti Oct 27 '21 at 12:06
  • Why do you need to scale the output prediction? Scaling is usually performed on the feature vectors. – DV82XL Oct 31 '21 at 18:29

0 Answers0