I'm working on an imblance classification problem, regarding the evaluation on the test set I had the following questions
- Is it a fair practice to use balanced test set for evaluation ?
- To create balanced test set, I'm removing the negtive examples from the imbalanced test set (without using any oversampling methods).
for eg., I have 500 positive class examples and 900 negative class examples. To make test set balance, i removed 400 negative class samples. Then the number of positive and negative class examples are same, thus dataset is balanced.
I also observe that PR-AUC for positve classes is increased (compared to imbalanced test set) after balancing the test set, so is it fair to report PR-AUC vaalues with balanced test set ?
