I am new to neural network pruning.I know the unstructured pruning sets the weights to zero while the structured pruning does actually change the network architecture. However, I am curious about the difference between their performance.
To be specific , I have created two models, a big one ,a small one.
class MyNet(nn.Module):
def __init__(self,channels):
super(MyNet,self).__init__()
self.conv1 = nn.Conv2d(3, channels, kernel_size=3, stride=1, padding=1, bias=False)
self.fc = nn.Linear(channels,10,bias=False)
def forward(self,x):
out = self.conv1(x)
out = F.avg_pool2d(out,32)
out = out.view(out.size(0), -1)
out = self.fc(out)
return out
big = MyNet(channels=64)
small = MyNet(channels=32)
The small model copies weights from the big one, then the big only remains the weights that the small model has copied from.(the remaining weights will be set to zeros) in the first iteration they output the same, but after that,the two models output differently. So I wonder why they output differently after the second iteration?
