How to create an error function comparing two matrices?

Question

I have two matrices in MATLAB. Each one is filled with 1 and 0 at different positions. I want to compare each element:

If there is a 1 match, I want it to record as True Positive.
If there is a 0 match, I want it to record as True Negative.
If one says 1 and the other says 0, I want to record as False Positive.
If one says 0 and the other says 1, I want to record as False Negative.

I tried just comparing the two matrices:

idx = A == B

But, that gives me a simple match, not telling me when there is a True Positive or Negative, etc.

Is there any specific function I could use, or any alternative?

Wolfie · Accepted Answer · 2019-11-18T15:47:58.743

You could just add the matrices in a prescribed way....

a = [1 0 1 0
     1 1 0 0
     0 0 1 1];

b = [1 0 0 0
     0 0 0 1
     0 0 1 0];

C = a + 2*b; 
% For pairs [a,b] we expect
% [0,0]: C = 0, true negative
% [1,0]: C = 1, false positive
% [0,1]: C = 2, false negative
% [1,1]: C = 3, true positive
% C = 
%   [ 3 0 1 0
%     1 1 0 2
%     0 0 3 1 ]

If you have the Statistics and Machine Learning toolbox and you only want a summary, you might just need the function confusionmat.

From the docs:

C = confusionmat(group,grouphat) returns the confusion matrix C determined by the known and predicted groups in group and grouphat. [...]. C is a square matrix with size equal to the total number of distinct elements in group and grouphat. C(i,j) is a count of observations known to be in group i but predicted to be in group j.

For example:

a = [1 0 1 0
     1 1 0 0
     0 0 1 1];

b = [1 0 0 0
     0 0 0 1
     0 0 1 0];

C = confusionmat( a(:), b(:) );
% C = 
%    [ 5    1
%      4    2]
% So for each pair [a,b], we have 5*[0,0], 2*[1,1], 4*[1,0], 1*[0,1]

A similar function for those with the Neural Network Toolbox instead would be confusion.

score 2 · Answer 2 · answered Nov 18 '19 at 13:26

2

You could just use bitwise operators to produce the four different values:

bitor(bitshift(uint8(b),1),uint8(a))

Produces an array with

0 : True Negative
1 : False Negative (a is true but b is false)
2 : False Positive (a is false but b is true)
3 : True Positive

answered Nov 18 '19 at 13:26

L. Scott Johnson

4,213
2
17
28

1

Nice idea, this can be achieved with simply `a + 2*b`, see [here](https://stackoverflow.com/a/58914888/3978545). – Wolfie Nov 18 '19 at 13:54
Indeed, but I like my bit operations to be explicitly uint8 (or whatever) for clarity. – L. Scott Johnson Nov 18 '19 at 14:07
Oh, and I see that's exactly your answer already: I didn't notice it was the same because I'm not used to bitops being "hidden" under regular algbraic ops. – L. Scott Johnson Nov 18 '19 at 14:09
My original answer was just to summarise using the confusion matrix, I added the algebraic option after reading your great suggestion, +1 for the inspo! – Wolfie Nov 18 '19 at 15:47

score 1 · Answer 3 · answered Nov 18 '19 at 12:32

One naive approach would be four comparisons, case by case:

% Set up some artificial data
ground_truth = randi(2, 5) - 1
compare = randi(2, 5) - 1

% Determine true positives, false positives, etc.
tp = ground_truth & compare
fp = ~ground_truth & compare
tn = ~ground_truth & ~compare
fn = ground_truth & ~compare

Output:

ground_truth =
   1   0   1   0   0
   0   1   1   0   1
   1   1   0   1   0
   0   1   0   1   1
   0   0   0   1   0

compare =
   0   1   1   0   1
   0   1   1   1   0
   1   1   0   0   1
   1   1   1   0   0
   1   1   1   1   1

tp =
  0  0  1  0  0
  0  1  1  0  0
  1  1  0  0  0
  0  1  0  0  0
  0  0  0  1  0

fp =
  0  1  0  0  1
  0  0  0  1  0
  0  0  0  0  1
  1  0  1  0  0
  1  1  1  0  1

tn =
  0  0  0  1  0
  1  0  0  0  0
  0  0  1  0  0
  0  0  0  0  0
  0  0  0  0  0

fn =
  1  0  0  0  0
  0  0  0  0  1
  0  0  0  1  0
  0  0  0  1  1
  0  0  0  0  0

That works, because 0 and 1 (or any positive value) are alternative representations for true and false.

To keep your main code clean, set up a separate function, say my_stats.m

function [tp, fp, tn, fn] = my_stats(ground_truth, compare)

  % Determine true positives, false positives, etc.
  tp = ground_truth & compare;
  fp = ~ground_truth & compare;
  tn = ~ground_truth & ~compare;
  fn = ground_truth & ~compare;

end

and call it in your main code:

% Set up some artificial data
ground_truth = randi(2, 5) - 1
compare = randi(2, 5) - 1

[tp, fp, tn, fn] = my_stats(ground_truth, compare)

Hope that helps!

score 0 · Answer 4 · answered Nov 18 '19 at 14:45

0

I found that I can use the find method and set two conditions, then just find the numbers of the element in each variable


TruePositive = length(find(A==B & A==1)) 
TrueNegative = length(find(A==B & A==0)) 
FalsePositive = length(find(A~=B & A==1))
FalseNegative = length(find(A~=B & A==0))

The confusionmatrix() method suggested by @Wolfie is also really neat, especially if you use the confusionchart() which provides a nice visualisation.

answered Nov 18 '19 at 14:45

user1234

111
9

3

Using `nnz(A==B & A==1)` or `sum(A==B & A==1)` (or any of your other condiditions) will generally be much faster than combining `length` and `find`, because in this case you don't care about the indices returned by `find` so just "counting the ones" which `nnz` and `sum` both do would be simpler. – Wolfie Nov 18 '19 at 15:46

How to create an error function comparing two matrices?

4 Answers4