I'm having a a problem with using weights in my Loss function. I have a really imbalanced dataset with 7 classes, so I calculated the weight for each class and put it in a tensor.
The list I Tensor'd looks like this [0.8901, 0.3295, 0.9885, 0.8887, 0.9858, 0.9486, 0.9673].
I want to use those weights in my CrossEntropyLoss function, but when I then train on the data using ResNet18 (model = torch.hub.load('pytorch/vision:v0.10.0', 'resnet18').to(device)) I get an Error because apparently, the ResNet has 1000 channels (-> classes) in its final layer, so I am supposed to give 1000 weights in my tensor, which is nonsense.
So my main question is: How can I use ResNet with 7 weights in my CrossEntropyLoss?
Also an extra question: Say I don't use ResNet but my own CNN, which has 7 channels in its final layer. How does the CrossEntropyLoss function know which class to assign which weight to, since its a list and not a dictionary...?
What I tried:
class_weights = torch.Tensor([0.8901, 0.3295, 0.9885, 0.8887, 0.9858, 0.9486, 0.9673])
[...]
model = torch.hub.load('pytorch/vision:v0.10.0', 'resnet18').to(device)
criterion = nn.CrossEntropyLoss(weight=class_weights).to(device)
and when I started training, I got this error:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Input In [15], in <cell line: 1>()
1 if use_save==False:
----> 2 fn.trainNetwork(epochs=num_epochs, loader=train_loader, model=model, crt=criterion, opt=optimizer)
3 fn.saveModel(model, data_dir, "HAM10000")
4 else:
File ~/digitale-wirtschaft/src/functions/cnn_actions.py:26, in trainNetwork(epochs, loader, model, crt, opt, save)
24 # forward
25 scores = model(data)
---> 26 loss = crt(scores, targets)
27 losses.append(loss.item())
28 # backward
File /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1130, in Module._call_impl(self, *input, **kwargs)
1126 # If we don't have any hooks, we want to skip the rest of the logic in
1127 # this function, and just call forward.
1128 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1129 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1130 return forward_call(*input, **kwargs)
1131 # Do not call functions when jit is used
1132 full_backward_hooks, non_full_backward_hooks = [], []
File /opt/conda/lib/python3.10/site-packages/torch/nn/modules/loss.py:1164, in CrossEntropyLoss.forward(self, input, target)
1163 def forward(self, input: Tensor, target: Tensor) -> Tensor:
-> 1164 return F.cross_entropy(input, target, weight=self.weight,
1165 ignore_index=self.ignore_index, reduction=self.reduction,
1166 label_smoothing=self.label_smoothing)
File /opt/conda/lib/python3.10/site-packages/torch/nn/functional.py:3014, in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction, label_smoothing)
3012 if size_average is not None or reduce is not None:
3013 reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 3014 return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)
RuntimeError: weight tensor should be defined either for all 1000 classes or no classes but got weight tensor of shape: [7]
I also tried multiplying the list in the Tensor with my batch_size (=64), so:
class_weights = torch.Tensor([0.8901, 0.3295, 0.9885, 0.8887, 0.9858, 0.9486, 0.9673]*batch_size)
but that led to the same error, only with a different number for weight tensor shape (=448) in the last sentence.
