2

I have this function to set up a descending learning rate:

def learning_rate_005_decay_power_099(current_iter):
    base_learning_rate = 0.05
    lr = base_learning_rate  * np.power(.99, current_iter)
    return lr if lr > 1e-3 else 1e-3

Now i want to pass it to the callback parameter in the train API of xgboost. I'm trying this:

watchlist = [(xg_train, 'train'), (xg_test, 'test')]

num_round = params['num_round']
xgclassifier = xgb.train(params, xg_train, num_round, watchlist, early_stopping_rounds=early_stopping, callbacks = [xgb.callback.reset_learning_rate(learning_rate_005_decay_power_099)]);

But i'm getting this error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-142-398cd3e1ab95> in <module>
     82 
     83             num_round = params['num_round']
---> 84             xgclassifier = xgb.train(params, xg_train, num_round, watchlist, early_stopping_rounds=early_stopping, callbacks = [xgb.callback.reset_learning_rate(learning_rate_005_decay_power_099)]);
     85             xgb_rounds.append(xgclassifier.best_iteration)
     86 

~/.local/lib/python3.6/site-packages/xgboost/training.py in train(params, dtrain, num_boost_round, evals, obj, feval, maximize, early_stopping_rounds, evals_result, verbose_eval, xgb_model, callbacks, learning_rates)
    214                            evals=evals,
    215                            obj=obj, feval=feval,
--> 216                            xgb_model=xgb_model, callbacks=callbacks)
    217 
    218 

~/.local/lib/python3.6/site-packages/xgboost/training.py in _train_internal(params, dtrain, num_boost_round, evals, obj, feval, xgb_model, callbacks)
     68                            end_iteration=num_boost_round,
     69                            rank=rank,
---> 70                            evaluation_result_list=None))
     71         # Distributed code: need to resume to this point.
     72         # Skip the first update if it is a recovery step.

~/.local/lib/python3.6/site-packages/xgboost/callback.py in callback(env)
    137         if context == 'train':
    138             bst, i, n = env.model, env.iteration, env.end_iteration
--> 139             bst.set_param('learning_rate', get_learning_rate(i, n, learning_rates))
    140         elif context == 'cv':
    141             i, n = env.iteration, env.end_iteration

~/.local/lib/python3.6/site-packages/xgboost/callback.py in get_learning_rate(i, n, learning_rates)
    128             new_learning_rate = learning_rates[i]
    129         else:
--> 130             new_learning_rate = learning_rates(i, n)
    131         return new_learning_rate
    132 

TypeError: learning_rate_005_decay_power_099() takes 1 positional argument but 2 were given

If i try this with lightgbm using exactly the same working routine, it works nice:

xg_train = lgb.Dataset(X_train, label=y_train, silent = True)
            xg_test = lgb.Dataset(X_test, label=y_test, silent = True)

            watchlist = [xg_train, xg_test]
            names = ['train', 'test']

            xgclassifier = lgb.train(params, xg_train, num_round, valid_sets = watchlist, valid_names = names, early_stopping_rounds= early_stopping,
                                     callbacks=[lgb.reset_parameter(learning_rate=learning_rate_005_decay_power_099)]);

How can this operation be set correctly please? And also, what makes different the 2 calls in xgboost and lightgbm?

Thank you very much in advance

Miguel 2488
  • 1,410
  • 1
  • 20
  • 41

1 Answers1

0

I think it requires a function with 2 parameters:

f(boosting_round, num_boost_round)

You gave value for parameter boosting_round only.

https://xgboost.readthedocs.io/en/latest/python/python_api.html#xgboost.callback.reset_learning_rate

guyko
  • 17
  • 2