0

I had a problem to classify inputs which have more than one label. So problem is multi-label classification. I used scikit-learn Decision Tree classifiers to do this and it gives pretty good results at initial stages. But, I am wondering how is it working under the hood and How the split is done in Decision Tree for multi-label classification? The important question is about how one model which is initialized once can be trained with two different class of labels at the same time? How the Decision Tree model will solve out the optimization task for both different sets of labels?

Urvish
  • 643
  • 3
  • 10
  • 19
  • it works the same as for a 'normal' classification problem – PV8 Sep 03 '19 at 12:58
  • @PV8 I know it gives the more or less same result but the question is about how one model which is initialized once can be trained on two different class of labels at the same time? – Urvish Sep 03 '19 at 15:50

1 Answers1

0

Under the hood, each node in your decision tree has the same labels as the root node, however, the probability of each label is different. When you run model.predict(), the model gives the prediction as the label with the highest probability. You can use model.predict_proba() to see the probability for each label separately. You can use this code to get the probabilities correctly:

all_probs=pd.DataFrame(model.predict_proba(X_test),columns=model.classes_)
Roee Anuar
  • 3,071
  • 1
  • 19
  • 33
  • Anur my question is how it works? What will be the optimization objective of the model when there are two sets of labels? – Urvish Sep 03 '19 at 15:54
  • It depends on the branching metric you chose for your model. For example if you work with gain ratio - the mutual information of each attribute with the class attribute is compared, and the attribute with the highest gain is selected. – Roee Anuar Sep 03 '19 at 15:56
  • can you provide some more depth thoughts on this? Some resource to learn will help as well. – Urvish Sep 03 '19 at 16:03
  • 1
    try this http://people.cs.pitt.edu/~milos/courses/cs2750-Spring2014/Lectures/Class13.pdf and this http://www.ise.bgu.ac.il/faculty/liorr/hbchap9.pdf – Roee Anuar Sep 03 '19 at 16:52