3

So I am trying to use torch.nn.utils.prune.global_unstructured.

I did it on a simple model and that worked. model.cov2 or other layers and that works. I am trying to do it on a model that's (nested)? I get errors as:

AttributeError: 'CNN' object has no attribute 'conv1'

and other errors. I tried everything to access this deep cov1, but I couldn't.

You can find the model code below:

class CNN(nn.Module):
def __init__(self):
    """CNN Builder."""
    super(CNN, self).__init__()

    self.conv_layer = nn.Sequential(

        # Conv Layer block 1
        nn.Conv2d(in_channels=3, out_channels=32, kernel_size=3, padding=1),
        nn.BatchNorm2d(32),
        nn.ReLU(inplace=True),
        nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3, padding=1),
        nn.ReLU(inplace=True),
        nn.MaxPool2d(kernel_size=2, stride=2),

        # Conv Layer block 2
        nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3, padding=1),
        nn.BatchNorm2d(128),
        nn.ReLU(inplace=True),
        nn.Conv2d(in_channels=128, out_channels=128, kernel_size=3, padding=1),
        nn.ReLU(inplace=True),
        nn.MaxPool2d(kernel_size=2, stride=2),
        nn.Dropout2d(p=0.05),

        # Conv Layer block 3
        nn.Conv2d(in_channels=128, out_channels=256, kernel_size=3, padding=1),
        nn.BatchNorm2d(256),
        nn.ReLU(inplace=True),
        nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1),
        nn.ReLU(inplace=True),
        nn.MaxPool2d(kernel_size=2, stride=2),
    )


    self.fc_layer = nn.Sequential(
        nn.Dropout(p=0.1),
        nn.Linear(4096, 1024),
        nn.ReLU(inplace=True),
        nn.Linear(1024, 512),
        nn.ReLU(inplace=True),
        nn.Dropout(p=0.1),
        nn.Linear(512, 100)
    )


def forward(self, x):
    """Perform forward."""
    # conv layers
    x = self.conv_layer(x)
    # flatten
    x = x.view(x.size(0), -1)
    # fc layer
    x = self.fc_layer(x)
    return x

How can I apply pruning on this model?

Bando
  • 1,223
  • 1
  • 12
  • 31
user13965184
  • 31
  • 1
  • Need some more info and an error traceback to correctly understand the problem here. – yanarp Jul 21 '20 at 21:20
  • @PranayModukuru ``prune.random_unstructured(module, name="weight", amount=0.3)`` - from where I need to put this line for pruning ? - just after that CNN class or I need to put it in the training loop? – shivang patel Feb 06 '21 at 22:23
  • It should be in Training Loop - if you want to prune it after every iteration. Or if you want to do it only once - then call it once after the training or before the training. – yanarp Feb 08 '21 at 11:36

3 Answers3

0

Your modules are not names 'conv1' or 'conv2', you can see the names using the named_modules generator. From above, you have a 'conv_stem' which can be indexed as model.conv_stem[0] to access. You can iterate over modules to create a dict like:

parameters_to_prune = (
    (model.conv1, 'weight'),
    (model.conv2, 'weight'),
    (model.fc1, 'weight'),
    (model.fc2, 'weight'),
    (model.fc3, 'weight'), )

and pass this in. See for more: https://colab.research.google.com/github/pytorch/tutorials/blob/gh-pages/_downloads/f40ae04715cdb214ecba048c12f8dddf/pruning_tutorial.ipynb#scrollTo=UVFjM079F0Oi

Rahul Deora
  • 157
  • 1
  • 8
0

Use this method to see the names of layers

for layer_name, param in model.named_parameters():
    print(f"layer name: {layer_name} has {param.shape}")

and pass those names to prune method

for eg , in prune.random_unstructured(module_name, name="weight", amount=0.3)

Prajot Kuvalekar
  • 5,128
  • 3
  • 21
  • 32
0

When showing the inner layers' names using print method, it can be found that when nn.Sequential is used, the inner layer cannot be called directly by programmer since their names are like xxx.0, xxx.1 etc, and xxx.0.weight, xxx.0.bias as well. That's actually not the right grammar in Python. So rewrite the code and seperate the layers in nn.Sequential may be a right choise, although it is more complex.

Craftsman Jacob
  • 1
  • 2