22

I load a Keras model from .json and .hdf5 files. When I call model.evaluate(), it returns an error:

You must compile a model before training/testing. Use `model.compile(optimizer, loss)

Why do I need to compile to run evaluate()?

To add, the model can be passed predict() with no problem.

Innat
  • 16,113
  • 6
  • 53
  • 101
bhomass
  • 3,414
  • 8
  • 45
  • 75

3 Answers3

32

Because evaluate will calculate the loss function and the metrics.

You don't have any of them until you compile the model. They're parameters to the compile method:

model.compile(optimizer=..., loss=..., metrics=...)

On the other hand, predict doesn't evaluate any metric or loss, it just passes the input data through the model and gets its output.

You need the "loss" for training too, so you can't train without compiling. And you can compile a model as many times as you want, and even change the parameters.

The outputs and the loss function:

The model's outputs depend on it being defined with weights. That is automatic and you can predict from any model, even without any training. Every model in Keras is already born with weights (either initialized by you or randomly initialized)

You input something, the model calculates the output. At the end of everything, this is all that matters. A good model has proper weights and outputs things correctly.

But before getting to that end, your model needs to be trained.

Now, the loss function takes the current output and compares it with the expected/true result. It's a function supposed to be minimized. The less the loss, the closer your results are to the expected. This is the function from which the derivatives will be taken so the backpropagation algorithm can update the weights.

The loss function is not useful for the final purpose of the model, but it's necessary for training. That's probably why you can have models without loss functions (and consequently, there is no way to evaluate them).

Innat
  • 16,113
  • 6
  • 53
  • 101
Daniel Möller
  • 84,878
  • 18
  • 192
  • 214
  • would you mind explaining why in https://github.com/JihongJu/keras-fcn/blob/master/voc2011/infer.py, there was no need to compile to call model.evaluate? did load_model(fcn_vgg16_weights.h5) somehow included the loss function? – bhomass Sep 10 '17 at 05:26
  • Unfortunately, I've never ever been able to save and load a model in Keras, I don't know what happens. For me it's totally buggy and I can only save weights. But theoretically, yes, it's possible to save a model with its loss function and the optimizer states. But that doesn't mean that every saved model will be saved like that. – Daniel Möller Sep 10 '17 at 21:04
  • 1
    @bhomass Yes, the `load_model()` function actually has a `model.compile()` call inside of it. As long as the model was compiled before getting saved, it will be automatically re-compiled when it's loaded. – Yu-Yang Sep 11 '17 at 08:47
  • It also appears that you need to recompile every time you change which layers are trainable or not. – CMCDragonkai Jul 17 '18 at 02:52
  • Yes, you need to. – Daniel Möller Jul 17 '18 at 03:00
  • Ideally, shouldn't optimizer be passed in `model.fit`? – Prabhat Doongarwal Oct 07 '19 at 06:53
  • No, that's not done in Keras. Perhaps it's possible with the new eager mode, but the optimizer usually goes in compile. – Daniel Möller Oct 07 '19 at 14:40
  • @DanielMoller If I load a trained model on previous 1 year price and evaluate it daily on present day's price, does it get trained futher on evaluating? Briefly, does evaluating a model trains it? – Shashank Gupta Apr 27 '20 at 10:57
  • No, only `fit` trains the model. – Daniel Möller Apr 27 '20 at 13:16
  • @DanielMoller Okay. So, if I train a model on a data(from say Jan 2019 to March 2020) and then I save it, load it again and .fit it on new data(April 2020), the model gets upgraded right? Or does it forgets the previous training and fits on the new data only? – Shashank Gupta Apr 27 '20 at 15:13
  • It loads already trained. --- But it may forget if you don't include the old data in the new training. – Daniel Möller Apr 27 '20 at 15:16
  • @DanielMoller So is there any way of updating a loaded model by only providing it the new data? – Shashank Gupta Apr 27 '20 at 15:19
  • Only testing can tell if it will give you a good result. You can overfit the new data. – Daniel Möller Apr 27 '20 at 15:58
  • Similarly anything that are passed during compile such as optimizer, loss function are not needed when you just want to predict with trained weights of the model right? – haneulkim Jul 01 '22 at 08:32
  • 1
    @haneulkim, exactly – Daniel Möller Jul 03 '22 at 16:56
0

I know you're asking why, and I believe the answer above should suffice. However if you get this error, which I did, it was simply because I had a coding error. I copied model_1, and pasted it to create model_2. However, I forgot to change part of the code from model_1 to model_2. This was a bonehead move on my part, but I got the same exact error as stated above. See below picture:

enter image description here

This was the fix: enter image description here

Quantum Prophet
  • 337
  • 2
  • 8
0

To add to @Daniel Möller's great answer, recompiling the model also re-enables the (custom) metrics you used to monitor validation loss on or want to calculate now on test data with a simple model.evaluate call. This makes sure you use exactly the same metrics on your test data.

If you pass along y_test, this would even allow calculating the loss on the test samples, which often gets reported in research papers.

Jordy Van Landeghem
  • 17
  • 3