2

While examining the setting for evaluation in Tensorflow's PTB language model, I am perplexed by this setting for the evaluation in eval_config:

  eval_config = get_config()
  eval_config.batch_size = 1
  eval_config.num_steps = 1

in https://github.com/tensorflow/models/blob/master/tutorials/rnn/ptb/ptb_word_lm.py

To the best of my understanding, during evaluation, a window (which could be upto num_steps size) of context words is used to predict the next word, which is stored in a separate target tensor. If num_steps is set to 1, wouldn't it imply that only the preceding word is used for prediction (ignoring a context window size>1) ? Also during evaluation, why is batch_size set to 1 as well. Wouldn't it make sense to have a larger batch being fed into the network, to speech up evaluation ?

Sayan Ghosh
  • 31
  • 2

1 Answers1

0

I believe the unrolling for one step only in the output is to evaluate each word as it is first seen. This might also be why the batch size is 1.

Alexandre Passos
  • 5,186
  • 1
  • 14
  • 19