Python's Xgoost: ValueError('feature_names may not contain [, ] or <')

Question

Python's implementation of XGBClassifier does not accept the characters [, ] or <' as features names.

If that occurs, it raises the following:

ValueError('feature_names may not contain [, ] or <')

It would seem that the obvious solution would be to pass the equivalent numpy arrays, and get rid of the column names altogether, but if they haven't done it that must be for a reason.

What use does XGBoost have for the feature names, and what is the downside of simply passing it Numpy Arrays instead of Pandas DataFrames?

Edit: this is not a question about workarounds (those are obvious and in the question), but about why it is implemented this way

Answer 1

I know it's late but writing this answer here for other folks who might face this. Here is what I found after facing this issue: This error typically happens if your column names have the symbols [ or ] or <. Here is an example:

import pandas as pd
import numpy as np
from xgboost.sklearn import XGBRegressor

# test input data with string, int, and symbol-included columns 
df = pd.DataFrame({'0': np.random.randint(0, 2, size=100),
                   '[test1]': np.random.uniform(0, 1, size=100),
                   'test2': np.random.uniform(0, 1, size=100),
                  3: np.random.uniform(0, 1, size=100)})

target = df.iloc[:, 0]
predictors = df.iloc[:, 1:]

# basic xgb model
xgb0 = XGBRegressor(objective= 'reg:linear')
xgb0.fit(predictors, target)

The code above will throw an error:

ValueError: feature_names may not contain [, ] or <

But if you remove those square brackets from '[test1]' then it works fine. Below is a generic way of removing [, ] or < from your column names:

import re
import pandas as pd
import numpy as np
from xgboost.sklearn import XGBRegressor
regex = re.compile(r"\[|\]|<", re.IGNORECASE)

# test input data with string, int, and symbol-included columns 
df = pd.DataFrame({'0': np.random.randint(0, 2, size=100),
                   '[test1]': np.random.uniform(0, 1, size=100),
                   'test2': np.random.uniform(0, 1, size=100),
                  3: np.random.uniform(0, 1, size=100)})

df.columns = [regex.sub("_", col) if any(x in str(col) for x in set(('[', ']', '<'))) else col for col in df.columns.values]

target = df.iloc[:, 0]
predictors = df.iloc[:, 1:]

# basic xgb model
xgb0 = XGBRegressor(objective= 'reg:linear')
xgb0.fit(predictors, target)

For more read this code line form xgboost core.py: xgboost/core.py. That's the check failing which the error is thrown.

Answer 2

This is another regex solution.

import re

regex = re.compile(r"\[|\]|<", re.IGNORECASE)

X_train.columns = [regex.sub("_", col) if any(x in str(col) for x in set(('[', ']', '<'))) else col for col in X_train.columns.values]

Answer 3

Yet another solution:

X.columns = X.columns.str.translate("".maketrans({"[":"{", "]":"}","<":"^"}))

If you're interested in seeing which are the culprits:

X.columns[X.columns.str.contains("[\[\]<]")]

Answer 4

Just useto_numpy() to generate a numpy array:

from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier

X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=0.25, random_state=42)
clf = XGBClassifier(random_state=42)

###### Here
clf.fit(X_train.to_numpy(), y_train.to_numpy())

Answer 5

Here is the simplest solution....

Just use str.replace('arg1','arg2') in columns of your data. arg1-> define symbol u want to change. [^a-zA-Z0-9] this list describes all the symbols arg2 -> define symbol u want to replace it with

Example:

X_train.columns = X_train.columns.str.replace('[^a-zA-Z0-9]', '_')

**It worked Fine while fitting XGBRegressor models **