2

Say I am using the titanic dataset, with the variable age only:

import pandas as pd

data = pd.read_csv('https://www.openml.org/data/get_csv/16826755/phpMYEkMl')[["age", "survived"]]
data = data.replace('?', np.nan)
data = data.fillna(0)
print(data)

the result:

         age  survived
0         29         1
1     0.9167         1
2          2         0
3         30         0
4         25         0
...      ...       ...
1304    14.5         0
1305       0         0
1306    26.5         0
1307      27         0
1308      29         0

[1309 rows x 2 columns]

Now I train a decision tree to predict survival from age:

from sklearn.tree import DecisionTreeClassifier
tree_model = DecisionTreeClassifier(max_depth=3)
tree_model.fit(data['age'].to_frame(),data["survived"])

And if I print the structure of the tree:

from sklearn import tree
print(tree.export_text(tree_model))

I obtain:

|--- feature_0 <= 0.08
|   |--- class: 0
|--- feature_0 >  0.08
|   |--- feature_0 <= 8.50
|   |   |--- feature_0 <= 1.50
|   |   |   |--- class: 1
|   |   |--- feature_0 >  1.50
|   |   |   |--- class: 1
|   |--- feature_0 >  8.50
|   |   |--- feature_0 <= 60.25
|   |   |   |--- class: 0
|   |   |--- feature_0 >  60.25
|   |   |   |--- class: 0

These means that the final division for every node is:

0-0.08 ; 0.08-1.50; 1.50-8.50 ; 8.50-60; >60

My question is, how can I capture those limits in an array that looks like this:

[-np.inf, 0.08, 1.5, 8.5, 60, np.inf]

Thank you!

Mattravel
  • 1,358
  • 1
  • 15
Sole Galli
  • 827
  • 6
  • 21

1 Answers1

3

The decision classifier, in this case tree_model has an attribute called tree_ which allows access to low level attributes.

print(tree_model.tree_.threshold)

array([ 0.08335, -2.     ,  8.5    ,  1.5    , -2.     , -2.     ,
       60.25   , -2.     , -2.     ])

print(tree_model.tree_.feature)

array([ 0, -2,  0,  0, -2, -2,  0, -2, -2], dtype=int64)

The arrays feature and threshold only apply to split nodes. The values for leaf nodes in these arrays are therefore arbitrary.

To get the division/threshold of a feature, you can filter the threshold using the feature array.

threshold = tree_model.tree_.threshold
feature = tree_model.tree_.feature
feature_threshold = threshold[feature == 0]
thresholds = sorted(feature_threshold)
print(thresholds)

[0.08335000276565552, 1.5, 8.5, 60.25]

To have np.inf, you need to add it yourself.

thresholds = [-np.inf] + thresholds + [np.inf]
print(thresholds)

[-inf, 0.08335000276565552, 1.5, 8.5, 60.25, inf]

Reference: Understanding the decision tree structure.

wavingtide
  • 1,032
  • 4
  • 19