I want to drop columns from DataFrame
whose all values are null. Suppose I have a DataFrame -
col1 | col2 | col3 | col4
----------------------------
1 null C1 null
2 null null null
null null null null
3 null C2 null
Column col2
and col4
have all null
values, so I have to drop those columns from DataFrame whose all values are null. So the final DataFrame would look like the following-
col1 | col3
--------------
1 C1
2 null
null null
3 C2
The operation should take close to O(1).