numerical issue when computing complementary projection

Question

I am using matlab to compute the following problem:

A, B are two m by n orthogonal matrices, A'*A = I and B'*B = I where I is the identity matrix. And m is much bigger than n. I am computing C = (2B*B'-I)*A, which is also a orthogonal matrix.

But in matlab, the precision of number leads to the following issue:

max(max(abs(A'*A-I))) = e0 > 0
max(max(abs(B'*B-I))) = e0 > 0

And the computed C has max(max(abs(C'*C-I))) = e1 > e0.

If I repeat the above process by computing D = (2C*C'-I)*B, this error bound lift up for D, with more and more such kind of iterations, this error explodes.

Is there a way to compute without increasing this error bound?

Thanks!

Answer 1

As a general rule, the more computations and transformations your perform, the more you have to start worrying about floating point error.

To mitigate the effects, there are a couple of possibilities:

1) Use software like Maple or Mathematica that support exact arithmetic, which will mostly eliminate the problem at the cost of greater computational burden. My understanding is that Matlab does not support exact arithmetic (but I could be wrong on this...)

2) Use a higher precision than Double to mitigate the floating point error. For example, the FEX submission Multiple Precision Toolbox for MATLAB claims to support arbitrarily large precision formats. I've never used it myself though, so I can't comment on how well it works.

3) Depending on the relative magnitudes of the numbers in your matrices, there may be some numerical tricks or transformations that you can exploit to reduce the floating point error. This is a big topic, but in general the idea is to try and scale your problem so that numbers are all as close to each other in order of magnitude as possible.

There may be some other possibilities that I'm not aware of. I'm certainly no expert in this area. I'll be interested to see what other respondents suggest.

Oh, one other thing, I just stumbled upon a closely related SO question.