0

I’m currently studying XGBoost, and I learned that information gain in XGBoost is computed like this:

XGBoost information gain

1

What I’m curious is that, previously about information gain, I learned that it is computed (entropy of parent node - sum of entropy of child nodes), which is the opposite of XGBoost since XGBoost IG is computed as (score of left node + score of right node - score of parent node). I don’t understand why.

vimuth
  • 5,064
  • 33
  • 79
  • 116
em seoyk
  • 1
  • 1
  • This is Equation 7 in the [XGBoost KDD paper](https://www.kdd.org/kdd2016/papers/files/rfp0697-chenAemb.pdf). There are many possible ways you can choose a split when learning decision trees, and the authors motivate it as an impurity measure. – Alexander L. Hayes Dec 31 '22 at 19:03

0 Answers0