0

version: chainer 2.0.2 I use Adam optimizer ,then report error, I found it was caused by this code(fix1==0?): in adam.py:

@property
    def lr(self):
        fix1 = 1. - math.pow(self.hyperparam.beta1, self.t)
        fix2 = 1. - math.pow(self.hyperparam.beta2, self.t)
        return self.hyperparam.alpha * math.sqrt(fix2) / fix1

error log:

Traceback (most recent call last):
  File "AU_rcnn/train.py", line 237, in <module>
    main()
  File "AU_rcnn/train.py", line 233, in main
    trainer.run()
  File "/root/anaconda3/lib/python3.6/site-packages/chainer/training/trainer.py", line 285, in run
    initializer(self)
  File "/root/anaconda3/lib/python3.6/site-packages/chainer/training/extensions/exponential_shift.py", line 48, in initialize
    self._init = getattr(optimizer, self._attr)
  File "/root/anaconda3/lib/python3.6/site-packages/chainer/optimizers/adam.py", line 121, in lr
    return self.hyperparam.alpha * math.sqrt(fix2) / fix1
ZeroDivisionError: float division by zero
machen
  • 283
  • 2
  • 10
  • What value did you try to change in `exponential_shift`? Did you know that Adam uses `alpha` as learning rate, and `lr` itself should not be touched. – corochann Sep 04 '17 at 11:29
  • How to use adam algorithm? I can't set lr ? – machen Sep 04 '17 at 11:40
  • @corcochann is there any example code how to use adam, yes , I set lr decay exponential_shift in every epoch = 0.9 – machen Sep 04 '17 at 11:46

2 Answers2

1

Use "alpha" attribute to control learning rate for Adam in Chainer. "lr" is defined as built-in property, it should not be override by other value.

Set "alpha" as an attribute for ExponentialShift (official doc) as well to decay learning rate, if you use Adam optimizer.

from chainer.optimizers import Adam
optimizer = Adam(alpha=0.001)

# --- Define trainer here... ---

trainer.extend(extensions.ExponentialShift("alpha", 0.99, optimizer=optimizer), trigger=(1, 'epoch'))
corochann
  • 1,604
  • 1
  • 13
  • 24
0

I have same ploblem, and tried corochann's approach. However, it didn't slove the ploblem.

my chainer version 2.1.0 Used code is https://github.com/chainer/chainer/blob/master/examples/cifar/train_cifar.py be changed L57 into "optimizer = chainer.optimizers.Adam()".

K.K
  • 1
  • I think this is a problem of how you import the modules. But thank you for pointing out, I updated my answer. – corochann Nov 06 '17 at 05:18