0

I have a general question about scalapack and system resolution. I am using DGESV (Fortran) on a dense matrix. From what size would it make sense to switch to Scalapack? From 10 000 X 10 000 ? 100 000 to 100 000? Is there a general answer to that or is it case by case?

Thanks for insights

Laetis
  • 1,337
  • 3
  • 16
  • 28
  • Scalapack is for distributed parallelism, Lapack for serial computing or threading. Your question really does not make sense. How often does your computation need to use the linear algebra routine? Once? Once every timesteps? Thousand times per second? Does your code use MPI? Is it ready to be rewritten to MPI if not? How complex is the rest of the code? – Vladimir F Героям слава Sep 08 '21 at 09:11
  • I'm a little confused as DGESV as for linear equation solves, not matrix diagonalization. Do you mean dgeev? – Ian Bush Sep 08 '21 at 09:20
  • 2
    Simplest answer is when you run out of memory on a single node - if you use less memory than that use a threaded LAPACK/BLAS and take advantage of parallelism with probably minimal changes to your code. However if you need more memory than that, or as Vladimir says one node is just not fast enough, then you will have to use Scalapack, which will probably mean a fair bit of coding. – Ian Bush Sep 08 '21 at 09:20
  • Also note if it is time you are interested matrix diagonalizations don't scale that well. Also if it is diagonalizations you want you might look at [ELPA](https://elpa.mpcdf.mpg.de/) which has a simpler interface (IMO) than ScaLAPACK, and some people report it to scale better. – Ian Bush Sep 08 '21 at 09:22
  • Thanks for you reply. It was really more to have a general idea than on a specific case. Let's say my code is as this example: http://www.netlib.org/scalapack/slug/node27.html#SECTION04220000000000000000 . when does it worth to do all that changes instead of just using dgesv. Of course I can try on my server, but I was wondering if there were a general order of magnitude. – Laetis Sep 09 '21 at 06:37
  • So what does that example do? Just one linear solve and that's it? It makes a HUGE difference in whether your whole program is mostly solving one linear system or if that is just one of many complicated steps. It just changes everything, really. Moreover, it really sounds like a topic for https://scicomp.stackexchange.com/, not for Stack Overflow. – Vladimir F Героям слава Sep 09 '21 at 10:03
  • Also how often do you plan to run that code? How many nodes do you have? All this can change the answer completely. If you want to run the code just a few times, there is little point to spend a month of coding it a bit faster. If you have just one nodes where you can run it threaded, there is no point using Scalapack at all. Wheter it is worth or not depends on so many factors, most of them individual, and the size of the matrix is one of the *less important* ones. – Vladimir F Героям слава Sep 09 '21 at 10:07
  • 1
    Bit really, the Computational Science Stackexchange is a much better venue for such a question, there is very little programming aspect in your question, if any at all. Note that if you have just a few nodes, it may get more from using a GPU instead using some dedicated libraries. – Vladimir F Героям слава Sep 09 '21 at 10:12

0 Answers0