How much I can do with the evaluation tab in automl

Question

I'm trying AutoML Natural Language, and find the "evaluation" tab has much information. I want to know how much tuning I can really do with this tab. For example:

When I adjust the "score threshold", the "precision" and "recall" will change. But what I can do with it? Shall I retrain the model with a changed "score threshold"?
What will happen when i click "Train the New Model", if i didn't change the dataset, but adjusted the "score threshold"?

Setting the score threshold is a procedure external to the model training, and it neither affects it nor it is affected by it — desertnaut, Sep 13 '18 at 23:13

score 1 · Answer 1 · edited Sep 13 '18 at 23:16

Use the evaluation tab to select a relevant score threshold to apply to your problem. Because the API call to the model return a payload in the following format, with all the labels and their associated score, it is necessary to select a threshold to filter the results and display only the label(s) with score above this selected threshold.

    payload {
      classification {
        score: 0.999904990196
      }
      display_name: <label_1_name>
    }
    payload {
      classification {
        score: 9.50000030571e-05
      }
      display_name: <label_2_name>
    }

So, you don't retrain a new model with the selected threshold, because the threshold is not a training parameter.

Regarding the 2nd question: It will train the same model. In the UI, you can only "play" with the threshold to update the evaluation results, it does not save anything to the back-end. It just help you to select a relevant number for your application.

How much I can do with the evaluation tab in automl

1 Answers1