Use 'predict_contrib' in LightGBM to get SHAP-values

Question

In the LightGBM documentation it is stated that one can set predict_contrib=True to predict the SHAP-values.

How do we extract the SHAP-values (apart from using the shap package)?

I have tried

model = LGBM(objective="binary",is_unbalance=True,predict_contrib=True)
model.fit(X_train,y_train)
pred_shap = opt_model.predict(X_train) #Does not get SHAP-values

which does not seem to work

Sergey Bushmanov · Accepted Answer · 2020-11-11T20:28:04.437

5

Shap values the LGBM way with pred_contrib=True:

from lightgbm.sklearn import LGBMClassifier
from sklearn.datasets import load_iris

X,y = load_iris(return_X_y=True)
lgbm = LGBMClassifier()
lgbm.fit(X,y)
lgbm_shap = lgbm.predict(X, pred_contrib=True)
# Shape of returned LGBM shap values: 4 features x 3 classes + 3 expected values over the training dataset
print(lgbm_shap.shape)
# 0th row of LGBM shap values for 0th feature
print(lgbm_shap[0,:4])

Output:

(150, 15)
[-0.0176954   0.50644615  5.56584344  3.43032313]

Shap values from shap:

import shap
explainer = shap.TreeExplainer(lgbm)
shap_values = explainer.shap_values(X)
# num of predicted classes
print(len(shap_values))
# shap values for 0th class for 0th row
print(shap_values[0][0])

Output:

3
array([-0.0176954 ,  0.50644615,  5.56584344,  3.43032313])

Looks the same to me.

edited Nov 11 '20 at 20:28

answered Nov 11 '20 at 20:15

Sergey Bushmanov

23,310
7
53
72

That is odd - I just get predictions when using predict that way. – CutePoison Nov 11 '20 at 20:19
Try to copy the suggested code snippet line by line and see where/how it's different from yours – Sergey Bushmanov Nov 11 '20 at 20:25
Aah.. found it! I have `predict_contrib` in my model-init and not in `predict` method – CutePoison Nov 11 '20 at 20:44
Might be why as well ! – CutePoison Nov 11 '20 at 20:45
It should be `pred_contrib=True` in `.predict()` method – Sergey Bushmanov Nov 11 '20 at 20:46
Yeah, I noticed from your former lead (: typical stupid mistake ! – CutePoison Nov 11 '20 at 20:53
Although the documentation states `predict_contrib` - or am I misreading that ? – CutePoison Nov 11 '20 at 20:56
I think you (and me) are reading right. I believe the documentation is misleading in this case (stale? typo in docs?). First, see in docs that `predict_contrib` should be applied in `predict` method. Second try `lgbm.predict?` and see the params it expects. Actually the best way to resolve the issues like this is to go through source code – Sergey Bushmanov Nov 11 '20 at 21:06
how does `predict_contrib ` work in lightgbm? – Joey Gao Mar 04 '21 at 13:03

score 3 · Answer 2 · answered Nov 21 '20 at 16:23

The confusion arises because of the duplication (inconsistent naming) of the controlling parameter in two different lightgbm APIs.

Each of the two major APIs uses its own spelling:

predict_contrib (C)
pred_contrib (python).

And the documentation favors the C version (the python API spelling is not even considered an alias...)

Use 'predict_contrib' in LightGBM to get SHAP-values

2 Answers2