4

I would like to cut my SciPy's dendrogram into a number of clusters on multiple threshold values.

I've tried using fcluster but it can cut only on one threshold value.

(Here is a piece of code which I have taken from another question for example.)

import pandas

data = pandas.DataFrame({
'total_runs': {0: 2.489857755536053, 1: 1.2877651950650333, 2: 0.8898850111727028, 3: 0.77750321282732704, 4: 0.72593099987615461, 5: 0.70064977003207007, 6:0.68217502514600825,7: 0.67963194285399975, 8: 0.64238326692987524, 9:0.6102581538587678, 10: 0.52588765899448564, 11: 0.44813665774322564, 12: 0.30434031343774476, 13: 0.26151929543260161, 14: 0.18623657993534984, 15: 0.17494230269731209,16: 0.14023670906519603, 17: 0.096817318756050832, 18:0.085822227670014059, 19: 0.042178447746868117, 20: -0.073494398270518693,21: -0.13699665903273103, 22: -0.13733324345373216, 23: -0.31112299949731331, 24: -0.42369178918768974, 25: -0.54826542322710636,26: -0.56090603814914863, 27: -0.63252372328438811, 28: -0.68787316140457322,29: -1.1981351436422796, 30: -1.944118415387774,31: -2.1899746357945964, 32: -2.9077222144449961}, 
'total_salaries': {0: 3.5998991340231234,1: 1.6158435140488829, 2: 0.87501176080187315, 3: 0.57584734201367749, 4: 0.54559862861592978, 5: 0.85178295446270169,6: 0.18345463930386757, 7: 0.81380836410678736, 8: 0.43412670908952178, 9: 0.29560433676606418, 10: 1.0636736398252848, 11: 0.08930130612600648, 12: -0.20839133305170349, 13: 0.33676911316165403, 14: -0.12404710480916628, 15: 0.82454221267393346,16: -0.34510456295395986, 17: -0.17162157282367937, 18: -0.064803261585569982, 19: -0.22807757277294818, 20: -0.61709008778669083,21: -0.42506873158089231, 22: -0.42637946918743924, 23: -0.53516500398181921, 24: -0.68219830809296633, 25: -1.0051418692474947,26: -1.0900316082184143, 27: -0.82421065378673986, 28: 0.095758053930450004, 29: -0.91540963929213015, 30: -1.3296449323844519,31: -1.5512503530547552, 32: -1.6573856443389405}
})


from scipy.spatial.distance import pdist
from scipy.cluster.hierarchy import linkage, dendrogram

distanceMatrix = pdist(data)
dend = dendrogram(linkage(distanceMatrix, method='complete'), 
       color_threshold=4, 
       leaf_font_size=10,
       labels = df.teamID.tolist())

Dendrogram

So for above dendrogram I would like to make a cut at 3 for Green cluster but for Blue & Red cluster cut should be made at 5 (so both of them comes in a single cluster).

mux032
  • 65
  • 2
  • 8

1 Answers1

5

The method fcluster can do this with monocrit parameter, which allows you to pinpoint exactly where to cut on the dendrogram. You want to make cuts at positions -1 and -3, where -1 is the top of the tree and -3 is the third node (where blue meets green) counting from the top down. This is how: 

Z = linkage(distanceMatrix, method='complete')
monocrit = np.zeros((Z.shape[0], ))
monocrit[[-1, -3]] = 1
fc = fcluster(Z, 0, criterion='monocrit', monocrit=monocrit)

The flat clusters will be formed by performing separation only at nodes with values strictly greater than the threshold (which is 0).

To illustrate this, I first redo the dendrogram with numbered leaves:

dend = dendrogram(Z, color_threshold=4, leaf_font_size=10, labels = range(33))

dendrogram

and then print the flat clusters:

for k in range(1, 4):
    print(np.where(fc == k))

They are 

(array([30, 31, 32]),)
(array([12, 14, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]),)
(array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 13, 15]),)

So, green is split in two and red and blue are together.