2

According to the this published page BCubed precision and recall, thus F1-Measure calculation is the best technique for evaluating clustering performance. See Amigó, Enrique, et al. "A comparison of extrinsic clustering evaluation metrics based on formal constraints." Information retrieval 12.4 (2009): 461-486.

It shows BCubed calculation as seen below image

enter image description here

So as far as i understood we calculate precision and recall for the each item and then take average of their sum?

However my understanding is not matching their given evaluation as can be seen at the image below

enter image description here

According to the image above cluster homogeneity example - left side, I calculate Precision of BCubed as below but not matching

black : 4/4
gray: 4/7
Other three each one : 1/7

so average precision is : (4/4 + 4/6 + 1/7 + 1/7 + 1/7) / 5

However this is not matching with their result in image which is 0.59

BCubed precision of an item is the proportion of items in its cluster which have the item’s category (including itself). The overall BCubed precision is the averaged precision of all items in the distribution. Since the average is calculated over items, it is not necessary to apply any weighting according to the size of clusters or categories. The BCubed recall is analogous, replacing “cluster” with “category”.

sfjac
  • 7,119
  • 5
  • 45
  • 69
Furkan Gözükara
  • 22,964
  • 77
  • 205
  • 342

1 Answers1

2

There are 14 items, not 5.

(4*4/4+1/3+2*2/3+3*1/7+4*4/7)/14 = 0.5986394557823128

But these are toy examples. I prefer Adjusted Rand Index over BCubed, and its much wider known and accepted.

Has QUIT--Anony-Mousse
  • 76,138
  • 12
  • 138
  • 194
  • ok i see now. also where is this Adjusted Rand Index over BCubed is accepted? from the articles published i see that people uses averaged F1 measure which is Metrics Based On Set Matching i suppose am i incorrect? – Furkan Gözükara Apr 06 '16 at 13:02
  • the f measure defined here is the most commonly used one in the literature : http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.474.6500&rep=rep1&type=pdf – Furkan Gözükara Apr 06 '16 at 13:50