DL4J. After doing model.upgrade(Gradient) are the weights updated?

Question

I am using a MultiLayerNetwork. This network is trained and the results are as expected. As part of an effort to implement Federated Learning I need to upgrade the weights of this network with a new set of gradients coming from an external component.

My question is, in DL4J, once you execute model.update(newGradients), are the weights updated too or do I need to execute something else?

Thanks

score 0 · Accepted Answer · answered Nov 01 '17 at 19:15

I could find the answer to my question.

After updating the gradient, you have to update the weights (params)

If using an SGD, you would do something like this:

model.update(averageGradient);
INDArray params = model.params(true);
params.addi(averageGradient.gradient());

I hope it helps anyone else with a similar problem!

DL4J. After doing model.upgrade(Gradient) are the weights updated?

1 Answers1