multiplying conv layer weights( N,C,H,W) with Logits (H,W) pytorch

Question

I am implementing the paper Deep multiscale convolutional feature learning for weakly supervised localization of chest pathologies in X-ray images According to my understanding the layer relevance weights belong to the last layer of each dense block.

I tried implementing the weight constraints as shown below:

 def weight_constraints(self):

        weights= {'feat1': self.model.features.denseblock2.denselayer12.conv2.weight.data,
            'feat2':self.model.features.denseblock3.denselayer24.conv2.weight.data,
            'feat3':self.model.features.denseblock4.denselayer16.conv2.weight.data}

        sum(weights.values()) == 1

        for i in weights.keys():
            w = weights[i]    
            w1 = w.clamp(min= 0)
            weights[i] = w1
        return weights


 weights= self.weight_constraints()
        for i in weights.keys():
            w = weights[i]
            l = logits[i]
            p = torch.matmul(w , l[0])
            sum = sum + p

where logits is a dictionary which contains out of FC layer from each block as shown in the diagram.

logits = {'feat1': [tensor([[-0.0630]], ...ackward0>)], 'feat2': [tensor([[-0.0323]], ...ackward0>)], 'feat3': [tensor([[-8.2897e-06...ackward0>)]}

I get the following error :

mat1 and mat2 shapes cannot be multiplied (12288x3 and 1x1)

Is this the right approach?

Isn't the logit dimension `(H, W, C)`, if `C` is the number of classes? — Bob, Jan 04 '23 at 10:07
for now i am treating it as a binary classification problem(normal vs abnormal), with output node = 1 for FC layers of each block — learner99, Jan 04 '23 at 10:35
In my understanding for binary classification you should have two classes at the output (logits). — Bob, Jan 04 '23 at 12:08
understood, but as you can see in the paper, binary cross entropy loss is used which requires one output logit. please correct me if im wrong — learner99, Jan 05 '23 at 09:19
Cross entropy uses one logit per class, check documentation for pytorch CrossEntropy [here](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html) — Bob, Jan 06 '23 at 10:39
but the loss formula mentioned in paper seems to be BCE loss as mentioned [here.](https://pytorch.org/docs/stable/generated/torch.nn.BCELoss.html#torch.nn.BCELoss). which takes one output from FC and moreover, Sigmoid is used for BC where as Softmax is used for MC classification with CE loss as mentioned in [this](https://medium.com/dejunhuang/learning-day-57-practical-5-loss-function-crossentropyloss-vs-bceloss-in-pytorch-softmax-vs-bd866c8a0d23) My main question here is, in the paper BCE loss is implemented with Sigmoid, so how can **w.shape[-1] == l.shape[-1] (presumably 3).** be 3 ? — learner99, Jan 17 '23 at 07:51

score 0 · Answer 1 · answered Jan 04 '23 at 09:56

The paper states

The logit response from all the layers have same dimension (equal to the number of category for classification) and now can be combined using class specific convex combination to obtain the probability score for the class pc.

The function matmul you used perfroms matrix multiplications, it requires mat1.shape[-1] == mat2.shape[-2].

If you assume sum(w)==1, and torch.all(w > 0), you could compute the convex combination of l as (w * l).sum(-1) that is multiply w and l element-wise, broadcasting over the batch dimensions of l, and requiring w.shape[-1] == l.shape[-1] (presumably 3).

If you want to stick with matmul you can add one dimension to w and l, and perform the vector product as a matrix multiplication: torch.matmul(w[...,None,:], l[..., :, None]).

thank you!. i want output to be of dimension [C,1] so that i can pass it into sigmoid. In my case, C= 1. My weights dimension is [32,128,3,3] and my logits dimension is [1,1]. in your mentioned solution i get output of dimension [32,128,3,3]. — learner99, Jan 04 '23 at 11:47
this formula worked when i changed weights shape to **w = weights.view(logits.size(0),-1)** — learner99, Jan 17 '23 at 07:52

multiplying conv layer weights( N,C,H,W) with Logits (H,W) pytorch

1 Answers1