Multi-label classification return more than 1 class in each label

Question

How to train a Multi-label classification model when each label should return more than 1 class? Example: Image classification have 2 label: style with 4 classes and layout with 5 classes. An image in list should return 2 style and 3 layout like [1 0 1 0] [1 1 0 0 1]

My sample net:

class MyModel(nn.Module):
def __init__(self, n__classes=32):
    super().__init__()
    self.base_model = models.resnet50(pretrained=True).to(device)
    last_channel = self.base_model.fc.in_features
    self.base_model.fc = nn.Sequential()
    
    self.layout = nn.Sequential(
        nn.Dropout(0.2),
        nn.Linear(last_channel, n_classes_layout),
        nn.Sigmoid()
    )
    self.style = nn.Sequential(
        nn.Dropout(0.2),
        nn.Linear(last_channel, n_classes_style),
        nn.Sigmoid()
    )
def forward(self, x):
    base = self.base_model(x)
    return self.layout(base), self.style(base)

def loss_fn(outputs, targets):
    o1, o2 = outputs
    t1, t2 = targets

This is the very definition of *multi-label* classification, in contrast to *multi-class*, where the samples can belong to one class only. I kindly suggest you do some research first - this question is way too vague and broad for SO. — desertnaut, Sep 29 '21 at 20:35

score 1 · Accepted Answer · answered Sep 29 '21 at 18:32

1

I am not sure what you referring label to but it seems you have a multi output model predicting on one hand style, and on the other layout. So I am assuming you are dealing with a multi-task network with two "independent" outputs which are supervised separately with two-loss terms.

Consider one of those two: you can consider this multi-label classification task as outputting n values each one estimating the probability that the corresponding label is present in the input. In your first task, you have four classes, on the second task, you have five.

Unlike a single-label classification task, here you wish to output one activation per label: you can use a sigmoid activation per logit. Then apply the Binary Cross Entropy loss on each of those outputs. In PyTorch, you can use BCELoss, which handles multi-dimensional tensors. In practice you could concatenate the two outputs from the two tasks, do the same with the target labels and call the criterion once.

answered Sep 29 '21 at 18:32

Ivan

34,531
8
55
100

I mean, i tried multi-outputs model to return 1 class in each label with sigmoid function. Output array usually have 1 '1' and orthers is '0', like [1 0 0 0]. So, can i train that model to get output with more '1', like [1 0 0 1]? – Tùng Dương Quang Sep 29 '21 at 19:14
Which loss function were you using? – Ivan Sep 29 '21 at 19:33
No, i just want to ask is it possible? I can train model myself if it is possible. – Tùng Dương Quang Sep 29 '21 at 19:53
try to paste ur prediction code – Prajot Kuvalekar Sep 29 '21 at 19:54
Yes it is possible if you use the appropriate loss function: `nn.BCELoss` – Ivan Sep 29 '21 at 19:58
net = models.resnet50(pretrained=True).to(device) net.fc = nn.Sequential( nn.Dropout(0.2), nn.Linear(net.fc.in_features, n_classes), nn.Sigmoid() ) – Tùng Dương Quang Sep 29 '21 at 20:00
sholud i combine 2 lales to train with fc layer or keep train 2 label with 2 output layers? – Tùng Dương Quang Sep 29 '21 at 20:02
You may want to keep them separate for the sake of clarity. Get two prediction heads, two prediction tensors and call `nn.BCELoss` twice, then sum the two terms. – Ivan Sep 29 '21 at 20:03
No worries, consider accepting the answer as "valid" by clicking on the green checkmark on the left. If you have any trouble when applying the loss, don't hesitate to come back around here to ask, I should be able to help. – Ivan Sep 29 '21 at 20:25
But, when an image return a Dog and a Cat at the same time, what model learn from this? – Tùng Dương Quang Sep 30 '21 at 03:44
If the label has both "dog" and "cat" classes, then it would suggest the image has both of them in the image. Is that your question? – Ivan Sep 30 '21 at 06:14
I edit question and post my model and loss function. Is it good, and how to imporve it? – Tùng Dương Quang Oct 01 '21 at 08:17
1

That looks good to me, you should replace `self.base_model.fc = nn.Sequential()` with `self.base_model.fc = nn.Identity()` – Ivan Oct 01 '21 at 09:15
Thank you so much. Can i ask u something deeper, like does the correlation between labels can help the model? – Tùng Dương Quang Oct 01 '21 at 09:34
What do you mean by correlation? – Ivan Oct 01 '21 at 10:07

Multi-label classification return more than 1 class in each label

1 Answers1