0

I am performing a GridSearch on the parameters below for an XGB Classifier. It's simple enough, but when I run grid_search.fit(X_train, y_train) it returns ValueError: Invalid parameter xgb for estimator

I completed a xgb.get_params() to confirm the correct parameters were used and they all check out - see xgb.get_params() below. I've even went so far as to copy/paste the params/value into the code to see if they would work and I still receive the same error

Code:

preprocessing = Pipeline(steps=[('ct', ColumnTransformer(transformers=[
                                ('cat', cat_transformer, cat_features),
                                ('num', num_transformer, num_features)],
                                remainder = 'passthrough')),
                                ('svd', TruncatedSVD(n_components=8)),
                                ('feature_selection', selector)])
params = {"xgb__eta": [0.1], 
          "xgb__gamma": [0],
          "xgb__max_depth":[3],
          "xgb__min_child_weight": [1],
          "xgb__lambda": [1],
}

pipe = Pipeline(steps=[('preprocessing', preprocessing), ('classifier', XGBClassifier())])

grid_search = GridSearchCV(pipe, params, cv=10, scoring='roc_auc')

grid_search.fit(X_train, y_train)

xgb.get_params()

{'base_score': 0.5,
 'booster': 'gbtree',
 'colsample_bylevel': 1,
 'colsample_bynode': 1,
 'colsample_bytree': 1,
 'gamma': 0,
 'learning_rate': 0.1,
 'max_delta_step': 0,
 'max_depth': 3,
 'min_child_weight': 1,
 'missing': None,
 'n_estimators': 100,
 'n_jobs': 1,
 'nthread': None,
 'objective': 'binary:logistic',
 'random_state': 0,
 'reg_alpha': 0,
 'reg_lambda': 1,
 'scale_pos_weight': 1,
 'seed': None,
 'silent': None,
 'subsample': 1,
 'verbosity': 1}
```````

1 Answers1

1

You are referring to your classifier as xgb when that variable isn't declared anywhere.

The correct way to specify the params is to use classifier__ prefix since that's how you refer to XGBoostClassifier in your pipeline. Thus, you should change the params keys to:

params = {"classifier__eta": [0.1], 
          "classifier__gamma": [0],
          "classifier__max_depth":[3],
          "classifier__min_child_weight": [1],
          "classifier__lambda": [1],
}

or, inversely, change classifier to xgb in:

pipe = Pipeline(steps=[('preprocessing', preprocessing), ('xgb', XGBClassifier())])
LemurPwned
  • 710
  • 10
  • 19