0

I'm trying to implement a slightly different version of SGD with pytorch and test it on some datasets. I need to write a custom optimizer on which to train my model, however I cannot find any guide which explains how to do so. Is anyone able to tell me how I can proceed or suggest me any useful reference?

Thanks in advance.

MWB
  • 11,740
  • 6
  • 46
  • 91
Francesco Borg
  • 15
  • 6
  • Did you have a look at the internal implementation of SDG in PyTorch? – Ivan Dec 09 '21 at 20:20
  • @Ivan I did but to me it's not clear how they proceed. At a certain point an sgd function is called which I was not able to find in the documentation (the F.sgd... at the end, https://pytorch.org/docs/master/_modules/torch/optim/sgd.html) – Francesco Borg Dec 09 '21 at 20:26
  • 1
    You should look at the GitHub source code instead: here is [`torch.optim.SGD`](https://github.com/pytorch/pytorch/blob/master/torch/optim/sgd.py#L144). Then you can jump onto the functional definition itself [here](https://github.com/pytorch/pytorch/blob/aab67c6dff3819d3c05b328e964f1642227adbe6/torch/optim/_functional.py#L156). – Ivan Dec 10 '21 at 09:22

0 Answers0