0

I try to use Squared Normalized Error as my objective function for XGBoostRegressor using documentation hints here: https://xgboost.readthedocs.io/en/latest/tutorials/custom_metric_obj.html. My objective function equation is:

(prediction - observation) / standard_deviation(observations)

While trying to develop it I encountered the following issues:

  1. I am wondering if such objective function is allowed, since standard deviation contains information about all observations (labels) while loss is calculated for each training example individually.
  2. If my approach is correct, I am wondering how to calculate hessian and gradient of this objective function. I analyzed the squared error loss function here: Creating a Custom Objective Function in for XGBoost.XGBRegressor, but failed to understand why x=(predictions - observations) is treated as one parameter. In other words, why do we use loss function as x^2 instead of (x-y)^2? x and y correspond to predictions and observations respectively.

EDIT: I use XGBoost for the task of photovoltaic (PV) yield forecasting and I make predictions for multiple systems using one model. I would like to have low percentage error for all systems, despite their size. However, squared error makes training focus on the largest systems, as their error is naturally the largest. I changed the objective function to:

(prediction - observation) / system_size

and made system_size a global variable, as adding new input variables to gradient and hessian functions is not allowed. The code compiles without errors, but predictions are within very small range. Gradient can be divided by system_sizes, as dividing by constant does not change the derivative. Code I managed to develop so far:

def gradient_sne(predt: np.ndarray, dtrain: DMatrix) -> np.ndarray:
    #Compute the gradient squared normalized error.
    y = dtrain.get_label()
    return 2*(predt - y)/system_sizes

def hessian_sne(predt: np.ndarray, dtrain: DMatrix) -> np.ndarray:
    #Compute the hessian for squared error
    y = dtrain.get_label()
    return 0*y + 2

def custom_sne(y_pred, y_true):
    #squared error objective. A simplified version of MSNE used as
    #objective function.
    grad = gradient_sne(y_pred, y_true)
    hess = hessian_sne(y_pred, y_true)
    return grad, hess

# Customized metric
def nrmse(predt: np.ndarray, dtrain: DMatrix):
    ''' Root mean squared normalized error metric. '''
    y = dtrain.get_label()
    predt[predt < 0] = 0 # all negative predictions are zero
    std_dev = np.std(y)
    elements = np.power(((y - predt) / std_dev), 2)
    return 'RMSNE', float(np.sqrt(np.sum(elements) / len(y)))

I use python 3.7.5 and xgboost 1.0.2. I would appreciate your help very much.

danvic
  • 1
  • 2

0 Answers0