1

I am carrying out an speaker recognition project, I have the model already trained and its precision is 90%, but I have a problem when I make inference, the model gives two probabilities, since it was trained for two interlocutors, but I want that when I detect an speaker who was not in the training set, tell me that he is an "unknown speaker", therefore, how can I choose the decision threshold according to the two probabilities that the model gives me?

This is a snippet of the code in question:

sample_df = pd.DataFrame(list_).transpose()
sample_pred =  model.predict(sample_df)[0] # Here the model returns the name of the 
                                           # predicted speaker

sample_prob = model.predict_proba(sample_df)[0] # Here I get a list of two items, the 
                                                # probabilities for each speaker 
print(sample_prob) # Output example: [0.46 0.54]

for k,j in enumerate(sample_prob):
    if j <= 0.6 and sample_prob[k+1] <= 0.6: # How to change dynamically according to the 
                                             # result of sample_prob, this threshold ?, 
                                             # for example I put 0.6.
        sample_pred= "unknown speaker"
        break
    else:
        break
desertnaut
  • 57,590
  • 26
  • 140
  • 166
lmarenast
  • 11
  • 1

1 Answers1

0

I think you have a binary classification task -> is it an "unknown speaker" or not. And, if I understood correctly, you want to optimise the threshold. In other words, you do not want to use 0.5. Since this is a classification task I would choose the threshold that maximises the f1-score on the validation set (not the test set because that would mean that you have data leakage):

thresholds = np.arange(0, 1, 0.01)
scores = [f1_score(y_test, predictions>t) for t in thresholds]
idx = np.argmax(scores)

Best score is given by thresholds[idx]

I see that you output 2 probabilities but you really need just one. From the first one you can easily infer the second one.

tzinie
  • 717
  • 5
  • 9