0

I want to create different number of MKL threads for each MPICH process.

For example, lets say I have 4 MPICH processes. What I want is:

Process 1=4 MKL thread, Process 2=3 MKL thread, Process 3=5 MKL thread, etc.

I know export MKL_NUM_THREADS=4; export MKL_DOMAIN_NUM_THREADS="MKL_ALL=1, MKL_BLAS=4", but I don't get what I have to do in my specific case.

asheeshr
  • 4,088
  • 6
  • 31
  • 50
Gevni
  • 43
  • 2
  • 10
  • Set the number of threads in program by calling `mkl_set_num_threads()` based on the process rank (if that's what you want). – Hristo Iliev Jan 25 '13 at 16:11

1 Answers1

1

If what you want to achieve is that different ranks in an MPI job use different number of MKL threads, you can do it in two different ways.

You can set the number of threads in code according to the process rank, e.g.

#define NUM_PROCS 4

int threads_per_proc[NUM_PROCS] = { 4, 3, 5, 5 };
int rank;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
// ...
// Signal an error if rank >= 4
// ...
mkl_set_num_threads(threads_per_proc[rank]);

You can also use the MPMD launch mode of mpiexec (or mpirun), but that would be a bit cumbersome:

mpiexec -env MKL_NUM_THREADS 4 -n 1 ./program : \
        -env MKL_NUM_THREADS 3 -n 1 ./program : \
        -env MKL_NUM_THREADS 5 -n 2 ./program

I have omitted the options to set MKL_DOMAIN_NUM_THREADS for brevity. This launches one copy of the MPI program ./program with MKL_NUM_THREADS set to 4 (which copy becomes rank 0); one copy with MKL_NUM_THREADS set to 3 (which copy becomes rank 1); two copies with MKL_NUM_THREADS set to 5 (these copies become ranks 2 and 3).

Hristo Iliev
  • 72,659
  • 12
  • 135
  • 186
  • When i try to compile this code it give me an error : undefined reference to `MKL_Set_Num_Threads'. I think i need to link it with MKL. I don't get which command i use to link it with MKL. I tried to read MKL manual but didn't understand how I can link? – Gevni Jan 25 '13 at 18:06
  • If you use a recent version of the Intel compiler, you can simply pass the `-mkl` option. For older versions or for other compilers (e.g. GCC), it depends. I believe the user manual has a very extensive section with lots of examples on how to link with MKL. – Hristo Iliev Jan 25 '13 at 20:03