Computing F-measure for clustering

Question

Can anyone help me to calculate F-measure collectively ? I know how to calculate recall and precision, but don't know for a given algorithm how to calculate one F-measure value.

As an exemple, suppose my algorithm creates m clusters, but I know there are n clusters for the same data (as created by another benchmark algorithm).

I found one pdf but it is not useful since the collective value I got is greater than 1. Reference of pdf is F Measure explained. Specifically I have read some research paper, in which the author compares two algorithms on the basis of F-measure, they got collectively values between 0 and 1. if you read the pdf mentioned above carefully, the formula is F(C,K) = ∑ | ci | / N * max {F(ci,kj)}
where ci is reference cluster & kj is cluster created by other algorithm, here i is running from 1 to n & j is running from 1 to m.Let say |c1|=218 here as per pdf N=m*n let say m=12 and n=10, and we got max F(c1,kj) for j=2. Definitely F(c1,k2) is between 0 and 1. but the resultant value calculated by above formula we will get value above 1.

Can you post your recall and precision ? IINM if they're between 0 and 1, their mean (see your ref) should be <1 (check your formula). If they're not, your algorithm is most likely wrong. — Nikana Reklawyks, Oct 06 '12 at 00:29
Yes, please elaborate how you obtain precision and recall. They must be in 0 to 1 to make sense. Otherwise, F1 will also go out of bounds. Do you maybe have *overlapping* clusters? Then it won't work; and I'm not aware of a good extension that does allow evaluating overlapping clusters. — Has QUIT--Anony-Mousse, Oct 06 '12 at 07:18
I think this was meant: http://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-clustering-1.html — Andreas Mueller, Aug 04 '15 at 21:46

Has QUIT--Anony-Mousse · Answer 1 · 2012-10-06T07:16:51.290

The term f-measure itself is underspecified. It's the harmonic mean, usually of precision and recall. Actually you should even say F1-score if you mean the unweighted version, because you can put different weight on the two input values. But without saying which two values are averaged (not in the sense of the arithmetic mean!) this doesn't say much.

https://en.wikipedia.org/wiki/F1_score

Note that the values must be in the 0-1 value range. Otherwise, you have an error earlier on.

In cluster analysis, the common approach is to apply the F1-Measure to the precision and recall of pairs, often referred to as "pair counting f-measure". But you could compute the same mean on other values, too.

Pair-counting has the nice property that it doesn't directly compare clusters, so the result is well defined when one result has m cluster, the other has n clusters. However, pair counting needs strict partitions. When elements are not clustered or assigned to more than one cluster, the pair-counting measures can easily go out of the range 0-1.

E. Achtert, S. Goldhofer, H.-P. Kriegel, E. Schubert, A. Zimek
Evaluation of Clusterings Metrics and Visual Support
Int. Conf. Data Engineering (ICDE 2012)
http://www.computer.org/portal/web/csdl/doi/10.1109/ICDE.2012.128

Discusses some of these metrics (including Rand index and such) and gives a simple explanation of the "pair counting F-measure".

Regardless of this post's interest, I think an answer to this question should at least make said **particular mean** explicit — Nikana Reklawyks, Oct 06 '12 at 00:32
I was too lazy to verify via Wikipedia that it is the harmonic mean, and not the geometric. I tend to mix up things. — Has QUIT--Anony-Mousse, Oct 06 '12 at 06:59
@Anony-Mousse Thanks a lot to author for the above post. it was quite useful to me. Thanks a lot — mahesh cs, Oct 16 '12 at 12:39

score 8 · Accepted Answer · edited Nov 19 '20 at 19:42

The paper Characterization and evaluation of similarity measures for pairs of clusterings by Darius Pfitzner, Richard Leibbrandt and David Powers contains a lot of useful information regarding this subject, including the following example:

Given the set,

           D = {1, 2, 3, 4, 5, 6}

and the partitions,

           P = {1, 2, 3}, {4, 5}, {6}, and
           Q = {1, 2, 4}, {3, 5, 6}

where P is set created by our algorithm and Q is set created by standard algorithm we known

           PairsP = {(1, 2), (1, 3), (2, 3), (4, 5)},
           PairsQ = {(1, 2), (1, 4), (2, 4), (3, 5), (3, 6), (5, 6)}, and
           PairsD = {(1, 2), (1, 3), (1, 4), (1, 5), (1, 6), (2, 3), (2, 4),
                      (2, 5), (2, 6), (3, 4), (3, 5), (3, 6), (4, 5), (4, 6), (5, 6)}

so,

           a = | PairsP intersection PairsQ | = |(1, 2)| = 1
           b = | PairsP- PairsQ | = |(1, 3)(2, 3)(4, 5)| = 3
           c = | PairsQ- PairsP  | = |(1, 4)(2, 4)(3, 5)(3, 6)(5, 6)| = 5

     F-measure= 2a/(2a+b+c)

Note: There is an error in the publication on page 364 where a, b, c, and d are computed and the result of b and c are actually switched incorrectly. This switch would throw off the results of some other measures. Obviously, the F-measure is unaffected.

score 2 · Answer 3 · answered Aug 17 '13 at 17:35

The N in your formula, F(C,K) = ∑ | ci | / N * max {F(ci,kj)}, is the sum of the |ci| over all i i.e. it is the total number of elements. You are perhaps mistaking it to be the number of clusters and therefore are getting an answer greater than one. If you make the change, your answer will be between 1 and 0.

Computing F-measure for clustering

3 Answers3

Linked