Recursive Feature Elimination. This algorithm implements backwards selection of predictors based on predictor importance ranking. The predictors are ranked and the less important ones are sequentially eliminated prior to modelling. The goal is to find a subset of predictors that can be used to produce an accurate model.
Questions tagged [rfe]
161 questions
2
votes
1 answer
Why are my grid_scores_ from a RFECV, different from the score of the same cross validated model with the optimal features from the RFECV?
I'm using sklearn's RFECV to come to the optimal set of features for my classification problem. I have X with 217 numerical features, for a binary label y. I determine the optimal set of features like so:
min_features_to_select = 3
cv =…
Ablu_68
- 73
- 4
2
votes
1 answer
Apply MinMaxScaler() to RFECV() with a pipeline
I'm trying to do feature selection and I'm using RFECV for it and LogisticRegression. To do this, I need to scale the data because the regression will not converge otherwise.
However, I think if I scaled the full data first it would be biased…
amestrian
- 546
- 3
- 12
2
votes
2 answers
issue using imbalanced dataset with logloss and RFECV
I am using imbalanced dataset(54:38:7%) with RFECV for feature selection like this:
# making a multi logloss metric
from sklearn.metrics import log_loss, make_scorer
log_loss_rfe = make_scorer(score_func=log_loss, greater_is_better=False)
#…
Naveen Reddy Marthala
- 2,622
- 4
- 35
- 67
2
votes
1 answer
how to pass fit parameters of estimator to RFECV's fit?
I have a dataframe with some features and i would like to select important ones using RFECV. But, when I tried to send xgboost's fit parameters to RFECV's fit method like this:
# initialising xgboost
xgb_rfe =…
Naveen Reddy Marthala
- 2,622
- 4
- 35
- 67
2
votes
0 answers
Python 3 is slower than Python 2 in RFE
I'm new at python sklearn. Since I moved to Python 3, my code starts to take more time to run RFECV... It passed from 5h to almost 15h to run several RFE CV.
I found this link, but I guess that the case is different from mine: Why is Python 3 is…
Hrpereira
- 21
- 5
2
votes
1 answer
sklearn RFE with logistic regression
I am trying to make a logistic regression model with RFE feature selection.
weights = {0:1, 1:5}
model = LogisticRegression(solver='lbfgs', max_iter=5000, class_weight=weights)
rfe = RFE(model, 25)
rfe_model = rfe.fit(X_train,…
Rosa
- 155
- 10
2
votes
1 answer
RFECV for classification giving KeyError: 'weight'
Recursive feature elimination with cross-validation (RFECV) is not working, getting KeyError: 'weight'
. As I see it is not able to calculate coefficients, hence the weights are missing.
I have estimator as XGBClassifier.
Jayahe
- 127
- 8
2
votes
1 answer
Scikit Learn RFECV ValueError: continuous is not supported
I am trying to use scikit learn RFECV for feature selection in a given dataset using the code below:
import pandas as pd
from sklearn.ensemble import RandomForestRegressor
import matplotlib.pyplot as plt
from sklearn.model_selection import…
adamk95
- 35
- 4
2
votes
1 answer
How to use Recursive Feature elimination?
I am new to ML and have been trying Feature selection with RFE approach. My dataset has 5K records and its binary classification problem. This is the code that I am following based on a tutorial online
#no of features
nof_list=np.arange(1,13) …
The Great
- 7,215
- 7
- 40
- 128
2
votes
1 answer
How to use 'max_features' in Gridsearch when combining with RFECV?
Thank you for answering in advance. This is my first post and I am relatively new to python, so I apologize if I have formatted something terribly.
I am trying to combine recursive feature elimination and grid search in sklearn to determine the best…
krkaufma
- 39
- 7
2
votes
1 answer
How to insert selected variables by RFE into machine learning model in r?
I want to use recursive feature elimination method to select the top features and then put them into machine learning models. I write the code of RFE as
library(mlbench)
library(caret)
control <- rfeControl(functions=rfFuncs, method="cv",…
Helia
- 228
- 1
- 10
2
votes
2 answers
retrieve selected variables from caret recursive feature elimination (rfe) results
In my working project, I use rfe function from caret package to do recursive feature elimination. I use a toy example to illustrate my point.
library(mlbench)
library(caret)
data(PimaIndiansDiabetes)
rfFuncs$summary <- twoClassSummary
control <-…
zesla
- 11,155
- 16
- 82
- 147
2
votes
2 answers
Extract Optimal Features from Recursive Feature Elimination (RFE)
I have a dataset consisting of categorical and numerical data with 124 features. In order to reduce its dimensionality I want to remove irrelevant features. However, to run the dataset against a feature selection algorithm I one hot encoded it with…
Adam
- 94
- 1
- 12
2
votes
1 answer
In R, caret package RFE function selects more features than allowed in size
I have a simple code that uses rfe to perform feature selection on different time periods of my data. I use the following rfeControl and rfe function calls:
control <- rfeControl(functions=rfFuncs, method="cv", number=10)
results <-…
Danica Fernandes
- 103
- 1
- 8
2
votes
1 answer
Setting n_features_to_select RFE as percentage in pipeline
I have a pipeline like so:
lin_reg_pipeline = Pipeline([
('polynomial_features', PolynomialFeatures()),
('normalize_polynomial_features', StandardScaler()),
('feature_selection', RFE(LinearRegression(), verbose=1)),
('lin_reg',…
Kevin
- 23
- 5