5

It's mathematically known that inverting a positive definite matrix via Cholesky decomposition is faster than just using np.linalg.inv(X). However, when I experimented with both and it turns out Cholesky decomposition's performance is worse!

# Inversion through Cholesky
p = X.shape[0]
Ip = np.eye(p)
%timeit scipy.linalg.cho_solve(scipy.linalg.cho_factor(X,lower=True), Ip)

The slowest run took 17.96 times longer than the fastest. This could mean that an intermediate result is being cached.
10000 loops, best of 3: 107 µs per loop


# Simple inversion
%timeit np.linalg.inv(X)

The slowest run took 58.81 times longer than the fastest. This could mean that an intermediate result is being cached.
10000 loops, best of 3: 25.9 µs per loop

The latter took shorter. Why is this? In R, chol2inv(chol(X)) is usually faster than solve(X).

Daeyoung
  • 194
  • 7
  • What is `p`? I just tried it with `p=1000`, and Cholesky was faster. – Warren Weckesser Sep 19 '16 at 15:05
  • @WarrenWeckesser I tried with 100... is it only faster as it grows bigger? – Daeyoung Sep 19 '16 at 15:06
  • 2
    That could be. Perhaps the Cholesky method has a bit more overhead. You can already see that you have to make two function calls instead of one. In fact, to be fair, you should also include the creation of `Ip` in the timing of the Cholesky method. – Warren Weckesser Sep 19 '16 at 15:11

3 Answers3

2

I run a comparison on 1000x1000 matrices, and the Inversion through Cholesky was roughly twice as fast.

Liran Katzir
  • 119
  • 1
  • 5
2

Perhaps your matrix is too small. I just tested matrix inversion for a $2\times2$ matrix in Matlab using Cholesky decomposition followed by LU decomposition. 999999 repeats take 5 seconds using Cholesky and only takes 3.4 seconds using LU. Indeed the algorithm of Cholesky followed by back substitution has a smaller big O result, but the result is an asymptotic result and only applies for big matrices.

#LU decomposition
tic
for i=1:999999
    (V_i(:,:,2)+[0 1e-10;0 0])\eye(2);
end
toc
Elapsed time is 3.430676 seconds.

#Cholesky
tic
for i=1:999999
    (V_i(:,:,2))\eye(2);
end
toc
Elapsed time is 4.824175 seconds.
user31575
  • 31
  • 1
  • 4
0

np.linalg.inv uses parallel processing. Check your cpu spike when you run both functions. if you are using parallel processing to run multiple inv or Cholesky you will find Cholesky to be faster.

Rudra Roy
  • 51
  • 1
  • 10