-2

F.dropout is only used in train, I confused how to use chainer.using_config in it? How does it works and chainer how to know it is in training or predict?

machen
  • 283
  • 2
  • 10
  • Welcome to SO. I don't uderstand your question. Before posting a question please read: **[How to ask a good question](https://stackoverflow.com/help/how-to-ask)**. – KoenV Aug 18 '17 at 13:14

1 Answers1

1

From Chainer v2, function behavior is controlled by the config variable, according to the official doc,

chainer.config.train

Training mode flag. If it is True, Chainer runs in the training mode. Otherwise, it runs in the testing (evaluation) mode. The default value is True.

You may control this config by following two ways.

1. Simply assign the value.

chainer.config.train = False
here, code runs in the test mode, and dropout won't drop any unit.
model(x)  
chainer.config.train = True

2. with using_config(key, value) notation

If we use above case, you might need to set True and False often, chainer provides with using_config(key, value) notation to ease this setting.

with chainer.using_config('train', False):
    # train config is set to False, thus code runs in the test mode.
    model(x)

# Here, train config is recovered to original value.
...

Note1: If you are using trainer moudule, Evaluator will handle these configuration automatically during validation/evaluation (see document, or source code). It means train config is set to False and dropout runs as evaluate mode when calculating validation loss.

Note2: train config is used to switch "train" mode and "evaluate/validation". If you need "predict" code, you need to implement separately.

corochann
  • 1,604
  • 1
  • 13
  • 24
  • Did you mean after with statement context exit, train config is recovered to original value, and thus below code only run in train mode not run in test mode? – machen Aug 21 '17 at 01:41
  • just like if - else statement? – machen Aug 21 '17 at 01:43
  • Yes, you are correct. In method 1 (assign value), you will overwrite the current config. But in method 2 (with statement), the config value will recover to its original value after exit with statement. – corochann Aug 21 '17 at 01:55
  • Supoose I have an 2-layer RNN, I want to write F.dropout in train mode. after F.dropout 1st RNN layer, I want to execute rest code(2nd layer RNN) any way(no matter in train mode or test mode) . But as you said, after with chainer.using_config('train', False) exit , the rest code don't execute in test mode. That is NOT WHAT I WANT? do you understand my consideration? – machen Aug 21 '17 at 02:01
  • In test mode (when train config is False), dropout won't drop any unit, and many of other layer (LSTM layer, Convolution2d layer etc) works same with train mode. Thus in test mode, all the code is executed (I guess as you expected). – corochann Aug 21 '17 at 02:24
  • Maybe I omit the model definition in the answer, `model` is some instance of Chain, for example, below is VGG net example, https://github.com/chainer/chainer/blob/master/examples/cifar/models/VGG.py In this case all the calculation in the `__call__` function (including several times of dropout) is executed in test mode when `with using_config()` block is used. – corochann Aug 21 '17 at 02:26