2

I used the Scikit-learn API for XGBoost (in python). My accuracy was ~ 68%. I used the same parameter set and used the Learning API for XGBoost; my accuracy was ~ 60%. My understanding is that Scikit-learn API is a wrapper around Learning API and thus they should give me the same results. I do not understand why I am getting different results from these two APIs.

    cores=16
    random_state=0

    params = {
        'n_estimators': 100, 
        'learning_rate': 0.1,
        'max_depth': 3,
        'min_child_weight': 1.0,
        'subsample': 1.0,
        'gamma': 0.0,
        'tree_method':'gpu_exact',
        'colsample_bytree': 1.0,
        'alpha' : 0.0,
        'lambda': 1.0,
        'nthread': cores,
        'objective': 'binary:logistic',
        'booster': 'gbtree',
        'seed': random_state,
        'eta':0.1,
        'silent': 1
    }

    model = XGBClassifier(**params)
    r = model.fit(X_train,y_train)
    print(model)

    # make predictions for test data
    y_pred = model.predict(X_test)
    predictions = [round(value) for value in y_pred]

    # evaluate predictions
    accuracy = accuracy_score(y_test, predictions)
    print("Accuracy: %.2f%%" % (accuracy * 100.0))

Results:

XGBClassifier(alpha=0.0, base_score=0.5, booster='gbtree',
       colsample_bylevel=1, colsample_bytree=1.0, eta=0.1, gamma=0.0,
       lambda=1.0, learning_rate=0.1, max_delta_step=0, max_depth=3,
       min_child_weight=1.0, missing=None, n_estimators=100, n_jobs=1,
       nthread=16, objective='binary:logistic', random_state=0,
       reg_alpha=0, reg_lambda=1, scale_pos_weight=1, seed=0, silent=1,
       subsample=1.0, tree_method='gpu_exact')

Accuracy: 68.32%

    dtrain = xgb.DMatrix(X_train, label=y_train)
    dvalid = xgb.DMatrix(X_test, label=y_test)

    # fit model no training data
    model = xgb.train(params=params,dtrain=dtrain)

    # make predictions for test data
    y_pred = model.predict(dvalid)
    predictions = [round(value) for value in y_pred]

    # evaluate predictions
    accuracy = accuracy_score(y_test, predictions)
    print("Accuracy: %.2f%%" % (accuracy * 100.0))

Results:

Accuracy: 60.25%

Sukrit Mukhopadhyay
  • 21
  • 3
  • These algorithms are non deterministic. You have no guarantee that they will give you exactly the same performance. – Rafal Janik May 03 '19 at 21:53

1 Answers1

3

I believe the difference is because you have not specified the number of boosting rounds in the standard xgboost API (xgb.train()). As a result it is using the default of 10.

'n_estimators' is an sklearn specific terminology.

Also, contrary to the comment given above, this particular algorithm is expected to be deterministic when run multiple times on the same system.

rmitchell
  • 41
  • 3