1

I am working with the dataset of size (500, 33).

In particular the data set contains 9 features say

[X_High, X_medium, X_low, Y_High, Y_medium, Y_low, Z_High, Z_medium, Z_low]

Both visually & after correlation matrix calculation I observed that

[X_High, Y_High, Z_High] & [ X_medium, Y_medium, Z_medium ] & [X_low, Y_low, Z_low] are highly correlated (above 85%).

I would like to perform a Clustering algorithm (say K means or GMM or DBSCAN).

In that case,

Is it necessary to remove the correlated features for Unsupervised learning ? Whether removing correlation or modifying features creates any impact ?

Mari
  • 698
  • 1
  • 8
  • 27

2 Answers2

4

My assumption here is that you're asking this question because in cases of linear modeling, highly collinear variables can cause issues.

The short answer is no, you don't need to remove highly correlated variables from clustering for collinearity concerns. Clustering doesn't rely on linear assumptions, and so collinearity wouldn't cause issues.

That doesn't mean that using a bunch of highly correlated variables is a good thing. Your features may be overly redundant and you may be using more data than you need to reach the same patterns. With your data size/feature set that's probably not an issue, but for large data you could leverage the correlated variables via PCA/dimensionality reduction to reduce your computation overhead.

Julian Drago
  • 719
  • 9
  • 23
  • If I had a very small dataset, say, (35,45).In that case removing the features is a good choice ? – Mari Dec 27 '19 at 00:46
  • I want to be clear that removing features is not quite the same as the PCA/dimensionality reduction technique I suggested (in case you're new to dimensionality reduction). Outright removal of features is a complicated topic - what reasons do you have to say these features don't matter? Would removing them somehow bias your model or cause issues with its assumptions? This is a large topic that can't be fully answered here, but it's a tradeoff between your data size, computation time, and what you know about your data. – Julian Drago Dec 28 '19 at 17:34
  • What if you have a feature which is highly uncorrelated with any of the other features? Would this feature be a potential candidate to be removed for unsupervised learning, since there is a lack of data linkage? – Tan Pengshi Alvin Nov 10 '21 at 08:11
0

Removal of features in unsupervised learning is not a complicated matter. You should include features you want to analyze and remove features you don't want to analyze. Including too many features makes inference much more difficult. I can typically do inference on around 10 to 20 features at the most. More than that and you have a mess of a diagram to explain to someone. If you don't need to do inference, then you could consider adding more features but it is still not advisable due to blowing up your vector space.

The objective way to determine if 20 features vs. 100 features improved your segmentation model is to utilize supervised learning to validate the segments. This is one approach towards validating your unsupervised method.

addi wei
  • 71
  • 1
  • 3