0

this is my first question here and I hope you can help me.

At the moment I'm training a binary classifier for medical images and my dataset is imbalanced with a ratio of roughly 0.8 (negative) to 0.2 (positive). My code is written with pytorch and pytorch lightning and I am using torchmetrics for evaluation.

So my question is if the torchmetrics AUROC is a good choice as evaluation metric for my model and if anyone has used it themselves for a similar dataset. The parameter average="weighted" should account for imbalance.

My implementation:

auroc_val = AUROC(num_classes=2, average="weighted")

auroc_val(prediction, target)

Torchmetrics link:

https://torchmetrics.readthedocs.io/en/latest/references/modules.html#auroc

Source code:

https://github.com/PyTorchLightning/metrics/blob/master/torchmetrics/classification/auroc.py#L27-L193

My understanding is that this implementation should work fine and the results look legit but I am asking because I already ran into some pitfalls when it comes to model evaluation.

Thanks in advance, comments on this are much appreciated!

user123
  • 1
  • This is not related to programming, and is more suitable for https://stats.stackexchange.com. On a side note, AUCroc can be biased towards the dominant class. AUCpr is a less biased representation (PR- Precision Recall curve). – jhso Jan 11 '22 at 22:57
  • Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. – Community Jan 21 '22 at 18:40

0 Answers0