0
  • Is there a method in "torch.nn.Functional" which can allow me to get the Cross Entropy Loss for a Multi-class Classification with Integer Labels (1 integer label/class per instance)?
  • Should I convert both lists into a FloatTensor?
import torch
import torch.nn.functional as F

Right now, my

predictions = [1, 2, 2, 5, 3]
actual_targets = [1, 2, 6, 5, 7]

have integers as elements

Then I convert from list to tensors:

predictions = torch.tensor(predictions)
actual_targets = torch.tensor(actual_targets)

I have tried some other posts which mentioned converting tensors to 'Long' if we have integer labels, but I encountered the same error message.

predictions = torch.LongTensor(predictions)
predictions.type(torch.LongTensor)
actual_targets = torch.LongTensor(actual_targets)
actual_targets.type(torch.LongTensor)

F.cross_entropy(predictions, targets)


Error message:
---> 13 F.cross_entropy(predictions, targets)

File /opt/conda/lib/python3.10/site-packages/torch/nn/functional.py:3029, in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction, label_smoothing)
   3027 if size_average is not None or reduce is not None:
   3028     reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 3029 return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)

RuntimeError: Expected floating point type for target with class probabilities, got Long
Kai
  • 17
  • 7

1 Answers1

1

First of all, how do you get your predictions? The Cross Entropy Loss function takes a vector of logits with size equal to the numebr of classes. This vector is a set of real-valued numbers that represent the unnormalized log probabilities of the possible classes. In addition to this vector, the Cross Entropy Loss function takes the target class. Argmax function can be used to get the predicted class from the logits vector.

In your case however, the predictions are already in the form of a list of classes so you should take a step back and use the raw logits for further calculations.

In your question you asked for multi-class classification. By this, you probably mean that you want to predict multiple features where each feature can be one of multiple target classes. This can also be referred to as multi-label classification. There are multiple approaches to achieve this. The simplest one is to use multiple prediction heads, one for each feature. Let's say you have 3 features. For each feature, you can calculate the cross entropy from the logits vector using the F.cross_entropy function:

# Loss for feature 1, example using random numbers
logits = torch.tensor([[ 0.4175,  0.1158,  1.5293,  0.5842, -1.1428]]) # logits for feature 1
target = torch.tensor([2]) # target label
loss_1 = F.cross_entropy(logits, target)

You can combine the losses by simply adding them together:

loss = loss_1 + loss_2 + loss_3
MarGenDo
  • 727
  • 1
  • 8
  • 17