Is there a way to display regression coefficients in a pandas data frame for categorical independent variables?

Question

I have built a multiple linear regression model and I found the coefficients using model.coef_. I want to make a pandas data frame which displays each of the factors and its coefficient.

pd.DataFrame(model.coef_, x.columns, columns = ['coef']).sort_values(by = 'coef', ascending = False) works only for the numerical independent variables. I have two columns of categorical variables and I have encoded them both.

Suppose the two values of a column are 'male' and 'female', I want to display the individual coefficients like

	coef
Male	0.2
Female	0.3

Is it possible to do this?

alparslan mimaroğlu · Accepted Answer · 2021-08-18T11:22:18.347

2

You can set_index after

(pd.DataFrame({'coef':model.coef_, 'category':x.columns})
.sort_values(by = 'coef', ascending = False)
.set_index('category'))

edited Aug 18 '21 at 11:22

answered Aug 18 '21 at 11:02

alparslan mimaroğlu

1,450
1
10
20

I am getting an error. `AttributeError: columns not found` – Sona Aug 18 '21 at 11:21
Is there a dataframe called x that has columns? Based on your question there seems to be one. – alparslan mimaroğlu Aug 18 '21 at 11:23
yes, x is the data frame consisting of the independent variables. 6 of them are numerical and 2 are categorical. – Sona Aug 18 '21 at 11:35
Then can you try again? There was a `]` in the comment but I don't think that was the problem. – alparslan mimaroğlu Aug 18 '21 at 11:49
1

Is it possible that you overwrote what x is. what do you get when you call `type(x)` – alparslan mimaroğlu Aug 18 '21 at 12:18
1

`scipy.sparse.csr.csr_matrix` I think it was changed while encoding. This is my code `ct = ColumnTransformer(transformers=[('encoder', OneHotEncoder(), [0,1])], remainder='passthrough') X = ct.fit_transform(X)` – Sona Aug 18 '21 at 16:51
1

It works after converting back to a pandas frame and flattening the coefficients to a 1D numpy array. Thanks :) – Sona Aug 19 '21 at 04:47

Is there a way to display regression coefficients in a pandas data frame for categorical independent variables?

1 Answers1