3

I want to implement a classifier which can have 1 of 10 possible classes. I am trying to use the MultiClass Softmax Loss Function to do this. Going through the documentation I'm not clear with what input is required for the function.

The documentation says it needs two matrices of [N, C] of which one is input and the other is target. As much as I understand, input matrix would be the one my Neural Network would calculate, which would have probabilities given by the neural network to each of the 10 classes. The target is the one that I have from my dataset.

The documentation says - "Target(N, C) - label targets padded by -1 ensuring same shape as the input." What does this mean? Do I pass zeros in incorrect classes and -1 for the correct one?

It would be great if someone could elaborate on this and show even a sample 2d matrix which could be passed as a target matrix.

Vivek
  • 165
  • 3
  • 9

2 Answers2

7

If you know that for each example you only have 1 of 10 possible classes, you should be using CrossEntropyLoss, to which you pass your networks predictions, of shape [batch, n_classes], and labels of shape [batch] (each element of labels is an integer between 0 and n_classes-1).

The loss you're looking at is designed for situations where each example can belong to multiple classes (say a person can be classified as both female and old). I think it's this "multi" that confuses you - it stands for the multiple possible classifications per example, not just multiple potential labels in the whole "universe".

In the sense of two/more labels in the universe, in which you seem to have been thinking, the counterpart to CrossEntropyLoss would be BCELoss (BCE stands for Binary Cross Entropy), which is just a simplification of CrossEntropyLoss for the case of two labels.

Jatentaki
  • 11,804
  • 4
  • 41
  • 37
1

In pytorch 1.8.1, I think the right way to do is fill the front part of the target with labels and pad the rest part of the target with -1. It is the same as the MultiLabelMarginLoss, and I got that from the example of MultiLabelMarginLoss.

Orange Chen
  • 11
  • 2