1

I am using a simple Keras model for series prediction.

I am feeding it input normalized across the entire series.

The model prediction accuracy seems to be correct during training. However, when I plot the outputs of the model.predict() function, I can see that the outputs have been somehow scaled. It seems to be some kind of normalization/standardization type of scaling.

Changing the batch size on training affects the result. I tried setting the batch size to the size of the input set, so that the training with the entire series is done in a single batch, which improves the result, but it is still scaled.

My assumption is this has something to do with either normalization per input batch or output normalization. I do not have any BatchNormalization layers in my model.

Is there a way to disable the default normalization/standardization of input/output in Keras (and does this default behavior exist)?

I am using Keras 2 with Tensorflow backend and Tensorflow 1.1.

Igor Ševo
  • 5,459
  • 3
  • 35
  • 80
  • What is your final activation layer? Sigmoid? Tanh? Those will always output normalized results. (And that is recommended. It's better to normalize your expected results for training) – Daniel Möller May 21 '17 at 19:38
  • The output of the last layer should be normalized. I am using `sigmoid` activation. That's not the problem. I believe the input is normalized/standardized per batch (or the output is somehow scaled) and there doesn't seem to be a way to turn this off. – Igor Ševo May 21 '17 at 19:55
  • Why do you believe the results are scaled? What is your "ground truth/expected results array"? Do you mean accuracy in training is ok but accuracy in validation data is not? – Daniel Möller May 21 '17 at 20:03
  • Please provide the model, this is not something that Keras does automatically. – Dr. Snoopy May 22 '17 at 06:12

2 Answers2

1

Keras does not insert BN or any other normalization implicitly.

You must be observing something else.

MWB
  • 11,740
  • 6
  • 46
  • 91
1

I think I ran into the same question as you did.

I checked the source code in Keras package for clues. In ../keras/engine/training.py, a function named _standardize_user_dara() was used in train, evaluate and predict function. This function is designed to standardize the input data(by batches, it seems), but a de-standardize function can not (yet) be found in the source code.

It is my guess that Keras does standardization to the input data but no de-standardization was applied to the output data. I am not sure this, because I haven’t checked all the codes, which is impossible… BTW, I read some else’s code implementing LSTM with time sequence data, the SKlearn package was use to do scaling and inverse-scaling. This could be a solution… I know this post is very old, but if you have any ideas about this standardization mechanism in Keras, please let me know, it will be very helpful

jasper2018
  • 11
  • 1