1

I am inspecting the following model:

decision tree plot

And I realized the leaf nodes are not on the same order of magnitude as the input data (.9 vs .09). Does this mean that once a leaf node is chosen it will multiply the input data by that leaf node to get a prediction?

It doens't really seem like that's what's going on, unless somehow the model for .predict is different than the model it plots.

test

testing it out manually gives me something way different. Instead, it seems to nearly predict the same value.

molbdnilo
  • 64,751
  • 3
  • 43
  • 82
MetaStack
  • 3,266
  • 4
  • 30
  • 67
  • 1
    Does this answer your question? [What does the value of 'leaf' in the following xgboost model tree diagram means?](https://stackoverflow.com/questions/40926340/what-does-the-value-of-leaf-in-the-following-xgboost-model-tree-diagram-means) (The accepted answer is more helpful than the highest voted answer.) – Nick ODell Dec 10 '21 at 23:49
  • @NickODell I was just reading that and, no, actually. the `return` result, is that the raw prediction? if so why don't I get 0.0992... when I run predict on that value? besides, that question is about classification, and I'm talking about regression... maybe the plot_tree from scikit-learn that I'm using doesn't handle the difference well? idk. it sounds much more like the first answer, that the leaf is expressing some probability, but I don't know what that means in a regression case. – MetaStack Dec 10 '21 at 23:51
  • 1
    Looking into this a bit further, XGBoost is creating multiple trees. So `plot_tree(xgb, num_trees=0)` shows you the first tree, `plot_tree(xgb, num_trees=1)` shows you the second one, etc. To see the number of trees that XGBoost is fitting, see [this question](https://stackoverflow.com/questions/50426248/how-to-know-the-number-of-tree-created-in-xgboost). – Nick ODell Dec 11 '21 at 00:39
  • @NickODell thanks! that makes way more sense now, I wish I could plot the resulting tree instead of investigating them one at a time, I don't know how they're weighted so turns out this does me very little good, but it does make sense that the leaf node is maybe a real prediction then... – MetaStack Dec 11 '21 at 00:43
  • Please read the [description](https://stackoverflow.com/tags/ml/info) of the ML tag. – molbdnilo Dec 11 '21 at 11:33

0 Answers0