0

I have the following code for creating and training a sklearn.ensemble.GradientBoostingClassifier

class myMonitor:
    def __call__(self, i, estimator, locals):
        proba = estimator.predict_proba(Xp2)
        myloss = calculateMyLoss(proba, yp2) # calculateMyLoss is defined 
                                             # further on
        print("Calculated MYLOSS: ",myloss)
        return False

... #some more code

model = GradientBoostingClassifier(verbose=2, learning_rate = learningRate, n_estimators=numberOfIterations, max_depth=maxDepth, subsample = theSubsample, min_samples_leaf = minLeafSamples, max_features=maxFeatures)
model.fit(Xp1, yIntegersp1, monitor = myMonitor())

However, when I run this code I get the error:

    model.fit(Xp1, yIntegersp1, monitor = myMonitor())
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/ensemble/gradient_boosting.py", line 980, in fit
    begin_at_stage, monitor)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/ensemble/gradient_boosting.py", line 1058, in _fit_stages
    early_stopping = monitor(i, self, locals())
  File "OTTOSolverGBM.py", line 44, in __call__
    proba = estimator.predict_proba(Xp2)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/ensemble/gradient_boosting.py", line 1376, in predict_proba
    score = self.decision_function(X)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/ensemble/gradient_boosting.py", line 1102, in decision_function
    score = self._decision_function(X)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/ensemble/gradient_boosting.py", line 1082, in _decision_function
    predict_stages(self.estimators_, X, self.learning_rate, score)
  File "sklearn/ensemble/_gradient_boosting.pyx", line 115, in sklearn.ensemble._gradient_boosting.predict_stages (sklearn/ensemble/_gradient_boosting.c:2502)
AttributeError: 'NoneType' object has no attribute 'tree_'

Why can't I use the same estimator (which is not None, I checked) to calculate the class probabilities during the run? is there a way to accomplish what I want (i.e. check the model on validation data at every iteration of the fitting procedure)?

WhiteTiger
  • 758
  • 1
  • 7
  • 21

2 Answers2

1

Your estimator is self. Try

def __call__(self, i, locals)
    proba = self.predict_proba(Xp2)
lanenok
  • 2,699
  • 17
  • 24
0

You could probably do something based on partial_fit similar to this example on forests. For analysis after training, have a look at this example on gradient boosting.

Andreas Mueller
  • 27,470
  • 8
  • 62
  • 74