How to convert logit shap values to probability

Question

I've been trying to convert shap values with the logit link to probabilities for all my features in a dataset. The model_output ='probability' parameter doesn't work for my catboost model. I want to convert the shap values for each feature so that when you sum them they equal the probability shown on the force plot. I've used a prewritten function to convert log odds to probabilities but the sum doesn't equal the same in the force plot. Any suggestions?

explainer = shap.TreeExplainer(catboost)
shap_values = explainer.shap_values(df_X_test)
shap.force_plot(explainer.expected_value, shap_values[0,:], df_X_test.iloc[0,:],link='logit')

score 1 · Answer 1 · edited Oct 09 '21 at 00:33

1

A workaround I have been using for instance-level explanations:

shap_values = (shap_values / sum(shap_values)) * probability

Where shap_values is a list, len(shap_values) = n_features, one Shapley value per feature and the probability is a double in [0, 1].

This will scale your Shapley values from whatever domain they are in, to the predicted probability.

edited Oct 09 '21 at 00:33

Flair

2,609
1
29
41

answered Oct 08 '21 at 13:21

StefanPopov

11
1
3

What is probability in this instance? where do I get the number from? – Nicholas Hansen-Feruch Dec 03 '22 at 05:24
Probability would be the output of your ML model. If **X** is your instance, and **f** is your model, then `probability = f(x)` – StefanPopov Dec 04 '22 at 09:51

How to convert logit shap values to probability

1 Answers1

Linked