Is it now possible to tune classification model thresholds in tidymodels?

Question

Does tidymodels now provide a means to tune classification model thresholds? I believe this was slated as an upcoming feature in the Spring of 2020. I looked around the tidymodels website, but have not seen a mention the feature.

Take a look at the [probably](https://probably.tidymodels.org/) package, especially the vignette on how to optimize probability thresholds. This package is still fairly immature so we are interested in feedback on what does/does not work for you. — Julia Silge, Apr 14 '21 at 23:43

score 3 · Accepted Answer · answered May 11 '21 at 18:13

3

As Julia says, there is an indirect method to do this.

We plan on making it a fully tunable parameter (like other parameters) but a few things have pushed this back but it is near the top of our development list.

answered May 11 '21 at 18:13

topepo

13,534
3
39
52

score 1 · Answer 2 · answered Apr 04 '23 at 09:37

There's now an R package available, probably (as part of the tidymodels ecosystem) that solves this problem.

In essence, we can change classification thresholds now like this:

hard_pred_0.75 <- lending_test_pred %>%
  mutate(
    .pred = make_two_class_pred(
      estimate = .pred_good, 
      levels = levels(Class), 
      threshold = .75
    )
  )

See this example for (reproducible) details.

Is it now possible to tune classification model thresholds in tidymodels?

2 Answers2