I have a binary variable with two possible values, 0 or 1. It's an imbalanced problem, 98% of my data is 0 and 2% is 1. My objective is to give a list of 100 the instances of the test set which have more probabilities to be 1.
Until now, I was facing this with a binary lightgbm classifier using the average precision metric. The problem is that using this metric i'm focusing on predicting all the data and i'm only interested in predicting correctly the 100 instances with higher scores. Intuitively, if i used a metric to train that depended on the predicted score of the algorithm I could have less false positives in the first 100 instances. How can I do so? Is there any metric that can help me solve this issue?
I tried customizing metrics with lightgbm but it seems complicated.