Calculating SVD using multiple cores in R

Question

I want to run svd() in R on a large sparse matrix (17k x 2m), and I have access to a cluster. Is there a straightforward way to calculate SVD in R using multiple cores?

The RScaLAPACK package (http://www.inside-r.org/packages/cran/RScaLAPACK) would seem to make this possible, but it no longer appears to be actively supported (http://cran.r-project.org/web/packages/RScaLAPACK/) and I assume there's a reason for that.

score 2 · Answer 1 · answered Apr 06 '14 at 06:46

2

rARPACK is the package you need. Works like a charm (even with matrix much larger than your specification). Superfast because it parallelizes via C and C++.

answered Apr 06 '14 at 06:46

Praveen Kumar

73
9

score 2 · Answer 2 · answered May 02 '14 at 21:08

2

rARPACK is one choice, but be sure that you have an optimized multicore BLAS library, since all the parallel computing part is not in rARPACK itself but rather in BLAS.

Also, be careful that rARPACK only calculates PARTIAL SVD, meaning it only calculates the largest k singular values and associated singular vectors. If you do need the full SVD, you may still turn to use svd().

answered May 02 '14 at 21:08

yixuan

774
3
6

Pretty much all BLAS are decent to start with, even precompiled ones (eg Ubuntu's default), and tuning them requires a lot of time. – isomorphismes Apr 01 '15 at 00:28

score 1 · Answer 3 · answered Jul 14 '21 at 16:17

1

You may also consider bigstatsr::bigSVD(). I've tested it and it tends to be faster than my GPU when I was working on large methylation datasets. It's not parallel, but I found it's performance really is pretty remarkable.

answered Jul 14 '21 at 16:17

James Dalgleish

163
15

Calculating SVD using multiple cores in R

3 Answers3