0

I have a 3 by 3 confusion matrix: 

+---------+------+--------+-----+
|         | Good | Medium | Bad |
+---------+------+--------+-----+
| Class 1 | 314  | 176    | 95  |
+---------+------+--------+-----+
| Class 2 | 184  | 275    | 126 |
+---------+------+--------+-----+
| Class 3 | 87   | 134    | 364 |
+---------+------+--------+-----+

I would now like to state something about that elements of class 1 are more likely to be identified as 'good' compared to elements of class 3. Similarly, I want to compare class 2 with class 3 and class 1 with class 2 as well. What would be a correct statistical approach/test to do this?

Astarno
  • 193
  • 1
  • 9
  • You can try using odds ratio as an statistical inference. For more, see [here](https://en.wikipedia.org/wiki/Odds_ratio#Statistical_inference) – null May 16 '20 at 18:43
  • Does that work on a 3x3 table/matrix though? – Astarno May 16 '20 at 18:45
  • It may able to give you some inference, but overall this is an ANOVA case. The true implementation for you is definetly ANOVA. – null May 16 '20 at 18:55

1 Answers1

0

If you are interested in it more likely to be good, you can collapse the other two categories into not good, and do a pairwise fisher.test for the comparison:

from scipy.stats import fisher_exact
import itertools
import pandas as pd
df = pd.DataFrame(index=['Class1','Class2','Class3'],
                  data={'Good':[314,184,87],'Medium':[176,275,134],'Bad':[95,126,364]})

df['not_gd'] = df['Medium'] + df['Bad']

def fn(comp,df):
    mat = df.loc[comp][['Good','not_gd']]
    ft = fisher_exact(mat)
    return {'comp':" vs ".join(comp),'OR':ft[0],'p':ft[1]}

pd.DataFrame([fn(list(i),df) for i in itertools.combinations(df.index.to_list(),2)])


    comp                OR          p
0   Class1 vs Class2    2.525148    1.828720e-14
1   Class1 vs Class3    6.632396    5.891070e-46
2   Class2 vs Class3    2.626537    1.959776e-11

It is possible to setup a logistic regression, but you will most likely get similar results like above.. and might be harder to do the post-hoc in python (needa use R).

StupidWolf
  • 45,075
  • 17
  • 40
  • 72