Configuring MKL with R

Question

I'm having a problem to trigger threads with R using the MKL library. I am currently using Ubuntu 18.04.2 LTS.

Linux pedro-HP-EliteOne-800-G1-AiO 4.18.0-15-generic #16~18.04.1-Ubuntu SMP Thu Feb 7 14:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

r-open$ screenfetch 
                          ./+o+-       pedro@pedro-HP-EliteOne-800-G1-AiO
                  yyyyy- -yyyyyy+      OS: Ubuntu 18.04 bionic
               ://+//////-yyyyyyo      Kernel: x86_64 Linux 4.18.0-15-generic
           .++ .:/++++++/-.+sss/`      Uptime: 19h 27m
         .:++o:  /++++++++/:--:/-      Packages: 1834
        o:+o+:++.`..```.-/oo+++++/     Shell: bash
       .:+o:+o/.          `+sssoo+/    Resolution: 1920x1080
  .++/+:+oo+o:`             /sssooo.   DE: GNOME 
 /+++//+:`oo+o               /::--:.   WM: GNOME Shell
 \+/+o+++`o++o               ++////.   WM Theme: Adwaita
  .++.o+++oo+:`             /dddhhh.   GTK Theme: Ambiance [GTK2/3]
       .+.o+oo:.          `oddhhhh+    Icon Theme: ubuntu-mono-dark
        \+.++o+o``-````.:ohdhhhhh+     Font: Ubuntu 11
         `:o+++ `ohhhhhhhhyo++os:      CPU: Intel Core i5-4590S @ 4x 3.7GHz [27.8°C]
           .o:`.syhhhhhhh/.oo++o`      GPU: intel
               /osyyyyyyo++ooo+++/     RAM: 2659MiB / 7867MiB
                   ````` +oo+++o\:    
                          `oo++.

Note: My machine has 4 cores.

Initially I downloaded the file l_mkl_2019.2.187.tgz on the intel website and did the installation for /home/pedro/. Later I downloaded the R-3.5.2.tar.gz file to compile R with multithreaded MKL support. I performed the compilation by doing:

source /home/pedro/intel/mkl/bin/mklvars.sh intel64
cd /home/pedro/Downloads/R-3.5.2 && ./configure --enable-R-shlib --enable-threads=posix --with-lapack --with-blas="-fopenmp -m64 -I$MKLROOT/include -L$MKLROOT/lib/intel64 -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -lpthread -lm"
make
make install

Apparently the compilation occurred without any problems. Using the sessionInfo() command I had the following result:

> sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.2 LTS

Matrix products: default
BLAS/LAPACK: /home/pedro/intel/compilers_and_libraries_2019.2.187/linux/mkl/lib/intel64_lin/libmkl_gf_lp64.so

locale:
 [1] LC_CTYPE=pt_BR.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=pt_BR.UTF-8        LC_COLLATE=pt_BR.UTF-8    
 [5] LC_MONETARY=pt_BR.UTF-8    LC_MESSAGES=pt_BR.UTF-8   
 [7] LC_PAPER=pt_BR.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=pt_BR.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_3.5.2

Basically I'm trying to parallelize a Monte Carlo simulation using the parallel :: mclapply function. The fact is that serially (1 core) I have better results using the MKL but when trying to parallelize, I realize that the threads are not fired in the 4 cores.

What I did using the function parallel::mclapply was:

parallel::mclapply(1:8, FUN = function(x) func_metodos(), mc.cores = 4)), wherein func_metodos() is a function that I implemented in R and want to repeat 8 times.

Note: I would like a solution other than using the Microsoft R Open. I do not like to use something that Microsoft puts its finger on.

Best regards.

Yes, I can run the examples from the parallel package. However, apparently the threads are not being distributed to more than one core. I say this by looking at the Gnome **CPU History tool**. — Pedro Rafael, Feb 20 '19 at 11:52
Regardless of whether **R** is using **MKL**, I believe that the **parallel** package examples would work, however, using only one core. — Pedro Rafael, Feb 20 '19 at 18:59

score 4 · Answer 1 · answered Feb 20 '19 at 13:38

4

I covered that in some length in

in this blog post
based on a script in this repo.

This can be fully automated as I demonstrated there. I can add more color but I got to run now...

answered Feb 20 '19 at 13:38

Dirk Eddelbuettel

360,940
56
644
725

Okay, I did the steps from https://github.com/eddelbuettel/mkl4deb. I previously had not done echo "MKL_THREADING_LAYER=GNU" >> /etc/environment. However, in my simulation, I get a great reduction in simulation time using BLAS + Lapack and using MKL the simulations get worse. This leads me to believe that something is still wrong. However, in my last test it does not compile the R. I just installed it using apt-get install r-base. Maybe that's it. I will test compiling the R. – Pedro Rafael Feb 20 '19 at 14:20
1

No that is a misunderstanding. You do *not* need to recompile R for this – Dirk Eddelbuettel Feb 20 '19 at 14:22
It's good to know that there's no need to compile the R for the perfect MKL support. I'll try to find out what might be happening. I know there are problems where paralleling may not be a good solution. However, I strongly believe that this is not the case since I get an improvement of almost 50% paralleling using Blas + Lapack. Most likely it was to observe something similar using the Intel MKL library. I was surprised to see a worsening. – Pedro Rafael Feb 20 '19 at 14:26
It seems to me that using `MKL_THREADING_LAYER=GNU` causes **MKL** to use **GNU OpenMP**, and somehow I can not explain why, slower than using **Blas**, **Atlas** or **OpenBlas**. I tried changing the **MKL** versions and I was not successful either. – Pedro Rafael Feb 20 '19 at 20:22
When experimented around the time I wrote the blog post, I did see MKL dominating OpenBLAS but not by much. But I wrote, and committed to GitHub, is generic and has worked for a number of people. I do not know what may be different with your setup -- but something likely is. – Dirk Eddelbuettel Feb 20 '19 at 20:31

Configuring MKL with R

1 Answers1