Drop automatically similar column effectively

Question

Here's the extreme simplification of my data

Column1  Column2  Column3  Column4
    yue       78      758      uyf
    tue       78      785      uyf

My Expected output

Column2  Column3
     78      758
     78      785

What I did currently is using df.describe() and after that look all parameter who has 0 in standard deviation and drop column manually like df[['Column2', 'Column3']] in this case, but currently I have nore than 200 columns

Regards

Does this answer your question? [Drop columns with low standard deviation in Pandas Dataframe](https://stackoverflow.com/questions/31799187/drop-columns-with-low-standard-deviation-in-pandas-dataframe) — Shubham Periwal, Apr 30 '21 at 08:50

score 1 · Accepted Answer · edited Apr 30 '21 at 09:25

1

#returns columns with a standard deviation not equal to zero

df[df.columns[df.std()!=0]]

edited Apr 30 '21 at 09:25

axtck

3,707
2
10
26

answered Apr 30 '21 at 08:59

MNgwira

26
1
2

score 0 · Answer 2 · answered Apr 30 '21 at 10:39

Do you want this ? Idea is to iterate over each column and check the number of unique values. If the unique value is only 1, then add that particular column to use_cols list.

use_cols = [cols for cols in df.columns if df[cols].nunique() ==1]
df= df[use_cols]

Drop automatically similar column effectively

2 Answers2