XGBoost python custom objective function

Question

According to documentation, http://xgboost.readthedocs.io/en/latest/python/python_api.html if we want to define custom objective function, it should have signature

objective(y_true, y_pred) -> grad, hess

where

hess: array_like of shape [n_samples]
The value of the second derivative for each sample point

But, if we have loss function, depending on N variables, we should have NxN matrix of second derivatives, but our hess's shape is only Nx1. Should we exlude "cross-variable" derivatives? Or what else?

https://gist.github.com/ytsaig/a596feec53d2a024ac69f5ae5a83d8f7 here's a multiclass example that probably answers your question. Yes, they have to be excluded, only d^2F / d^2Ni is relevant, not the crossvariable (d^2F / dNidNj) — Johan Rensink, May 11 '18 at 12:17
Oh and here is XGBoosts logreg example: https://github.com/dmlc/xgboost/blob/master/demo/guide-python/custom_objective.py — Johan Rensink, May 11 '18 at 12:19

score 1 · Answer 1 · answered May 04 '18 at 10:54

I think, the derivative that you have to take is with respect to the score that is returned by the Booster. Thus, it results in one value per training example (=sample) leading to a [n_samples] array. The score is what goes into your objective function, i.e. x in (x-m)**2 in MSE or 1/(1+exp(-x)) in the logistic function.

XGBoost python custom objective function

1 Answers1