Questions tagged [batchnorm]
11 questions
9
votes
1 answer
Batchnorm2d Pytorch - Why pass number of channels to batchnorm?
Why do I need to pass the previous nummber of channels to the batchnorm? The batchnorm should normalize over each datapoint in the batch, why does it need to have the number of channels then ?
TheBenimeni
- 111
- 1
- 6
1
vote
1 answer
Pytorch model with multi batchnorm1d layers get errors during inference - "expected 2D or 3D input (got 1D input)"
My model in pytorch with batchnorm1D is like this:
class Discriminator(nn.Module):
def __init__(self, sequenceLength):
super(Discriminator,self).__init__()
self.batchnorm1 = nn.BatchNorm1d(sequenceLength)
self.batchnorm2 =…
Peter Phan
- 25
- 5
1
vote
1 answer
batch size without batch normalization
I'm working on image super-resolution tasks with EDSR as a baseline model. Following EDSR, I'm not using any batch-norm layers in my model. I suddenly came up with a stupid question about batch-sizes.
Currently, I'm training my model with…
이민규
- 37
- 2
1
vote
1 answer
Pytorch Gradient w.r.t. Inputs using BatchNorm
I'm trying to calculate the gradient of the output of a simple neural network with respect to the inputs. The result looks fine when I don't use a BatchNorm layer. Once I do use it, the result doesn't seem to make much sense. Below is a short…
mpr
- 33
- 5
1
vote
1 answer
Problem of exporting batchnorm weight from pytorch to Keras
I follow Pytorch Batchnorm layer different from Keras Batchnorm,
Pytorch Batchnorm implementation, but they do not solve my problem.
I also read Wiki about Batchnorm.
And search source code from tensorflow batchnorm and from pytorch source…
choi
- 33
- 7
1
vote
1 answer
error in BatchNorm2d in pytorch CNN model
my database has grayscale images of size 128 * 128* 1 each with batch size =10
i am using cnn model but I got this error in BatchNorm2d
expected 4D input (got 2D input)
I posted the way which i used to transform my image (gray scale - tensor -…
Shorouk Adel
- 127
- 3
- 20
1
vote
1 answer
Tensorflow 2.0 Keras BatchNorm: how to update the online params in custom training?
How to train the batch norm layer without using any keras.compile methods? Typically layers have losses that are accessible. Here the losses method is empty.
UPDATE:
It seems like there is a lot of confusion about this and even the way the…
mathtick
- 6,487
- 13
- 56
- 101
0
votes
0 answers
How to implement BN layers in the Recurrent Block and Recurrent Residual Block with pytorch（Avoid using same Batchnorm Layer）?
I used the code implemented by bigmb, but the output of R2AttUnet in .train() mode are 10 times bigger than in .eval() mode. The U_net and AttU_net are good. But R2AttU_net and R2U_net always have this problem. I think it's because the same…
Paupaupaul
- 11
- 2
0
votes
1 answer
The differences of BatchNorm layer backpropagation at mode of model.train() and model.eval() in Pytorch?
I test the gradient of BatchNorm layer for two mode: model.train() and model.eval().
I bulid a simple CNN network NetWork and input the same input X to the network at model.train() mode and model.eval() mode.
I know the differences of model.train()…
Harxis
- 1
- 2
0
votes
1 answer
Understanding the keras.layers.BatchNormalization computation
I am playing with the BatchNormalization layer, and I can't quite figure out the numerical results I get.
Let's consider we use BatchNormalization for computer vision.
We have 4D tensors.
Dimensions are: batch size, image height, image width,…
Julien REINAULD
- 599
- 2
- 5
- 18
-2
votes
1 answer
BatchNormalization layer constructing
I'm trying to setup a BatchNormalization layer in C++.
The code as I have it looks like this:
mx::Symbol loadBatchNormalization(mx::Symbol previous, std::istream &file, const std::string &name, const Shape &inputShape, const Shape &outputShape,…
iamacomputer
- 518
- 3
- 14