How to get a list of all leaves under a node in a dendrogram?

Question

I made a dendrogram using scipy.cluster.hierarchy.dendrogram, using the following generated data:

a = np.random.multivariate_normal([10, 0], [[3, 1], [1, 4]], size=[100,]) b = np.random.multivariate_normal([0, 20], [[3, 1], [1, 4]], size=[50,]) c = np.random.multivariate_normal([8, 2], [[3, 1], [1, 4]], size=[80,]) X = np.concatenate((a, b, c),)

creating the linkage function:

from scipy.cluster.hierarchy import dendrogram, linkage Z = linkage(X, 'ward')

and then:

dendrogram( Z, truncate_mode='lastp', # show only the last p merged clusters p=5, # show only the last p merged clusters show_leaf_counts=False, # otherwise numbers in brackets are counts leaf_rotation=90., leaf_font_size=12., show_contracted=True, # to get a distribution impression in truncated branches )

Now, I have overall 230 observations in my data that were splitted to p=5 clusters. I want to have, for each cluster, a list of all row indices of all observations that are in it. In addition, I'd like to know the structure of the hierarchy above those 5 clusters.

Thanks!

Answer 1

I am a newbie to clustering and dendrogram. So welcome to point out error if there is any.

# put X in a dataframe
df = pd.DataFrame()
df['col1']=X[:,0]
df['col2']=X[:,1]

index=[]
for i in range(len(X)):
    elem = 'A' + str(i)
    index.append(elem)

df['index'] = index
print(df.shape)
df.head()

Z = linkage(X, 'ward')

dendrogram(
Z,
truncate_mode='lastp',  # show only the last p merged clusters
p=5,  # show only the last p merged clusters
show_leaf_counts=True,  # otherwise numbers in brackets are counts
leaf_rotation=90.,
leaf_font_size=12.,
show_contracted=True,  # to get a distribution impression in truncated branches
);
plt.show()

# retrieve elements in each cluster
label = fcluster(Z, 5, criterion='maxclust')

df_clst = pd.DataFrame()
df_clst['index']  = df['index']
df_clst['label']  = label

# print them
for i in range(5):
   elements = df_clst[df_clst['label']==i+1]['index'].tolist()  
   size = len(elements)
   print('\n Cluster {}: N = {}  {}'.format(i+1, size, elements))