1

After looking at the documentation of pytorch I couldn't find the module of GroupNorm with momentum, there is only an implementation which doesn't use it (which is useless to me since I would want to use AdaBN, or AdaGN I should say).

Therefore my idea was to take the BatchNorm class and extend it with num_groups and override the method that actually calculate mean and variance by firstly reshaping the input and grouping channels together, so to emulate the GroupNorm behavior.

In order to do this I looked into the source code until I found a dead end in torch.nn.functional.batch_norm. That function internally calls another function torch.batch_norm(...) but I couldn't find the source code for it.

Do you know where to look for that part? Or do you think there is a better way to implement this? I tried looking online for other's works, but I only found unfinished ones.

  • 2
    `toch.batch_norm` is a C++ function exposed to Python. Here is the C++ source: https://github.com/pytorch/pytorch/blob/d1623f4cc9334ade7376b3685bb91d3071e3b418/torch/csrc/api/src/nn/modules/batchnorm.cpp – Agost Biro Sep 02 '19 at 14:05
  • oh, thank you, but that's bad actually for me. –  Sep 02 '19 at 14:11

0 Answers0