3

I need to calculate features importance for my LightGBM Booster model. However, I cannot understand how are the values for feature importances obtained when using 'gain' type.

The docs say:

If "gain", result contains total gains of splits which use the feature.

I am using MAE objective and its initial value (absolute difference between mean value and each object) on the train sample equals 36.82; sum of all absolute errors equals 33200. However, the sum of all feature importances is about 139124, which is much greater. Could anyone share the formula, thanks.

jottbe
  • 4,228
  • 1
  • 15
  • 31
Akim
  • 139
  • 6
  • It has been a while, since you asked this question and I can't completely answer it, but I thought about this myself in my project and I think it is for two reasons, first it is the sum over all trees and they correct errors the previous trees made (they overshoot some times) and so the sum can be higher, because the FE doesn't distinguish the direction. The second is, that it uses the gratients (or something that represents the gradients) and not the actual values. The second point leads to very strange FE values in case of binary classification. – jottbe Dec 03 '20 at 11:16
  • This is, because it uses the values before applying the logistic function, that it internally uses to get the binary classification. – jottbe Dec 03 '20 at 11:17

0 Answers0