2

I am currently learning Keras and getting confused while using model.evaluate().

I have total 768 data points, but model.evaluate() is evaluating only 32 data points and give the accuracy of 75.52%. I also tried batch sizes of 100, 50, 20, 10 & 1, but it does not cover all the data points but the accuracy remains unchanged.

You can check my Jupyter Notebook file if the implementation here

I am unable to understand if I am doing something wrong, or it is a feature/bug of Keras.

Please help me with the same. Thanks in advance.

PS: I have tried searching the answer for this question over the internet and StackExchange but haven't found any. In case my question is a duplicate one, Please let me know the link to it and I will be happy to remove this question from here!

Marcin Możejko
  • 39,542
  • 10
  • 109
  • 120
thepunitsingh
  • 713
  • 1
  • 12
  • 30
  • This shouldn't happen. Make sure that your Keras and theano version are up-to date. Try building keras from source. If that happens again, raise an issue on Keras github – enterML Jun 06 '17 at 08:34
  • What is your `keras` version? Please try checking the shape of scores. Maybe it's correct and only a progbar is broken. – Marcin Możejko Jun 06 '17 at 09:06
  • @Nain I checked the version, `theano.__version__` is `0.9.0.dev-c697eeab84e5b8a74908da654b66ec9eca4f1291` and `keras.__version__` is `2.0.2`. I am on conda environment and now updated keras to its latest version `2.0.4`. But still `model.evaluate()` outputs the same. Updated the [same file](https://github.com/thepunitsingh/keras_tutorials/blob/master/pima-indians-diabetes-categorization.ipynb) on Git to have the current results. I guess I should raise the issue on Keras github. – thepunitsingh Jun 07 '17 at 09:52
  • @MarcinMożejko current `keras` version is `2.0.4`. I checked the shape of scores, they are list of 2 elements. I tried to replace progress bar by using `TQDMNotebookCallback` of `keras_tqdm`, but since `model.evaluate()` doesn't have a `callback` parameter I couldn't implement tqdm on it, so I cannot say for sure if the progress bar is broken. Please let me know if you could help in this. – thepunitsingh Jun 07 '17 at 15:54

1 Answers1

1

The problem here lies in a progbar not in evaluate function. Your progbar simply prints only a first step of evaluation. You may easily check that by inspecting that all your results are the same up to float32 precision which is about 1e-5 - 1e-7. So I wouldn't bother myself with this issue as jupyter very often has problems with displaying a progbar.

Marcin Możejko
  • 39,542
  • 10
  • 109
  • 120
  • I guess your opinion on this correct. I had noticed the results earlier but didn't know how to comprehend them. Thanks for helping me out. Now I believe this `progbar` problem is to fix by `Keras` community. – thepunitsingh Jun 10 '17 at 17:30