3

I have built a custom package with some functions written in RcppEigen. I also have Microsoft R open with Intel MKL enabled. How could I link the R package to the Intel MKL feature?

Setup 1:

Below are procedures that I have tried to link the package with MKL in the normal R, but failed:

The Eigen documents says I need:

1. #define EIGEN_USE_MKL_ALL
2. link your program to MKL libraries

(the MKL linking advisor)

Based on 2, in my file Makevars

PKG_CXXFLAGS = -I/opt/intel/mkl/include 
PKG_LIBS = ${LAPACK_LIBS} ${BLAS_LIBS} ${FLIBS}  -L/opt/intel/mkl/lib/intel64 -Wl,--no-as-needed -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core -lgomp -lpthread -lm -ldl

I got the errors when compiling the package:

Error in dyn.load(dllfile) : 
unable to load shared object '/home/path/RPackageName.so':
libmkl_intel_lp64.so: cannot open shared object file: No such file or directory

Update based on Ralf's comment: in the Makevars file add the option <,-rpath,'path'>, the errors are gone.

PKG_CXXFLAGS = -DMKL_LP64 -m64  -I/opt/intel/mkl/include 
PKG_LIBS =  ${LAPACK_LIBS} ${BLAS_LIBS} ${FLIBS}  -L/opt/intel/mkl/lib/intel64 -Wl,--no-as-needed,-rpath,'/opt/intel/mkl/lib/intel64' -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core -lgomp -lpthread -lm -ldl

The package compiled successfully, but with these messages below, which I do not understand.

/home/shen/R/x86_64-pc-linux-gnu-
library/3.4/RcppEigen/include/Eigen/src/Core/Assign_MKL.h: In 
instantiation of ‘static void Eigen::internal::Assignment<DstXprType, 
Eigen::CwiseUnaryOp<Eigen::internal::scalar_log_op<double>, 
SrcXprNested>, Eigen::internal::assign_op<double, double>, 
Eigen::internal::Dense2Dense, typename 
Eigen::internal::enable_if<Eigen::internal::vml_assign_traits<Dst, 
Src>::EnableVml>::type>::run(DstXprType&, const SrcXprType&, const 
Eigen::internal::assign_op<double, double>&) [with DstXprType = 
Eigen::Matrix<double, -1, 1>; SrcXprNested = const 
Eigen::ArrayWrapper<const Eigen::Matrix<double, -1, -1> >; 
Eigen::internal::Assignment<DstXprType, 
Eigen::CwiseUnaryOp<Eigen::internal::scalar_log_op<double>, 
SrcXprNested>, Eigen::internal::assign_op<double, double>, 
Eigen::internal::Dense2Dense, typename 
Eigen::internal::enable_if<Eigen::internal::vml_assign_traits<Dst, 
Src>::EnableVml>::type>::SrcXprType = 
Eigen::CwiseUnaryOp<Eigen::internal::scalar_log_op<double>, const 
Eigen::ArrayWrapper<const Eigen::Matrix<double, -1, -1> > >]’:
/home/shen/R/x86_64-pc-linux-gnu-
library/3.4/RcppEigen/include/Eigen/src/Core/AssignEvaluator.h:836:49:   
required from ‘void Eigen::internal::call_assignment_no_alias(Dst&, 
const Src&, const Func&) [with Dst = Eigen::Matrix<double, -1, 1>; Src 
= Eigen::CwiseUnaryOp<Eigen::internal::scalar_log_op<double>, const 
Eigen::ArrayWrapper<const Eigen::Matrix<double, -1, -1> > >; Func = 
Eigen::internal::assign_op<double, double>]’

Setup 2 I compiled directly in MRO without any special arguments in the file Makevars, got the errors below:

/home/shen/R/x86_64-pc-linux-gnu-
library/3.3/RcppEigen/include/Eigen/src/Core/util/MKL_support.h:57:21: 
fatal error: mkl.h: No such file or directory

PS: My experiences with the Microsoft R open are that they could accelerate the normal R scripts and functions in Armadillo without doing anything. Just run them normally in the Microsoft R Open.

vtshen
  • 202
  • 1
  • 11
  • What is your setup? Are you using MS R open, which includes MKL? Or are you using ‚standard‘ R linked against MKL? Or ‚standard‘ R with a separate installation of MKL? – Ralf Stubner Dec 31 '17 at 13:05
  • Hi Ralf, I have two setups. 1st setup is standard R without linking to MKL, but MKL is installed separately. I tried in the 1st setup first, the procedures are in the question, but failed. So I turned to another approach, that is the 2nd setup, which is the MS R open, including MKL. But I do not know how to do that. – vtshen Dec 31 '17 at 17:48
  • In the first setup, from which the error message originated, you have to tell the runtime linker the location of the MKL libs. Either via ‚-rpath‘ during compilation or LD_LIBRARY_PATH at runtime. In the second case I would expect that no special arguments are necessary for compilation and linking, since MRO uses MKL already and should provide the right arguments in ${LAPACK_LIBS} and ${BLAS_LIBS}. – Ralf Stubner Dec 31 '17 at 19:46
  • Hi Ralf, with the addition of the option <,rpath,'path'>, I was able to compile the package successfully. However, when I was running to check the performance, CPU is only 100%, not what I expected after adding the feature MKL. Maybe it is my Eigen code, but I do not know how to check the reason why MKL is not performing. – vtshen Jan 01 '18 at 17:25
  • For the 2nd setup, I compiled it directly in MRO, without special arguments, but failed, the error was 'cannot find the file mkl.h' – vtshen Jan 01 '18 at 17:27
  • Did you find a solution in the end? – Tom Wenseleers May 07 '18 at 15:31
  • @TomWenseleers In which setup are you interested? Linux or Windows? MRO, CRAN R with MKL as BLAS/LAPACK or CRAN R with other BLAS/LAPACK but additional MKL? – Ralf Stubner May 08 '18 at 12:52
  • MRO, CRAN R with MKL as BLAS/LAPACK on either Windows or Linux... – Tom Wenseleers May 08 '18 at 13:20
  • @TomWenseleers Not yet, just waiting ... – vtshen May 11 '18 at 04:35

1 Answers1

2

Partial answer to record my findings. Maybe others can build on this.

Linux with R linked against MKL

For this setup I used a Debian stable machine with R 3.5.0 installed. I installed MKL via this script. For compilation I used an add-hoc plugin with the compilation flags provided in the updated question plus -Wno-ignored-attributes to silence some unrelated warnings:

library(Rcpp)

registerPlugin(
  name = "mkl",
  plugin = function(x) {
    list(
      includes = "#define EIGEN_USE_MKL_ALL",
      env = list(PKG_CXXFLAGS = "-DMKL_LP64 -m64  -I/opt/intel/mkl/include -Wno-ignored-attributes",
                 PKG_LIBS = "${LAPACK_LIBS} ${BLAS_LIBS} ${FLIBS}  -L/opt/intel/mkl/lib/intel64 -Wl,--no-as-needed,-rpath,'/opt/intel/mkl/lib/intel64' -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core -lgomp -lpthread -lm -ldl")
    )
  }
)

cppFunction('
Eigen::VectorXd mkl_sin(Eigen::VectorXd x) {
  return x.array().sin();
}
', plugins = "mkl", depends = "RcppEigen")
mkl_sin((1:10)/10)
#>  [1] 0.09983342 0.19866933 0.29552021 0.38941834 0.47942554 0.56464247] 0.64421769 0.71735609 0.78332691 0.84147098

This works without any warnings, so I conclude that it works when R is linked with MKL as external BLAS/LAPACK.

Linux with MRO

For this setup I used a Ubuntu based Docker image:

FROM ubuntu:16.04

RUN apt-get update \
 && apt-get install --yes --no-install-recommends \
    apt-transport-https \
    build-essential \
    ca-certificates \
    curl \
    gfortran \
 && curl -O https://packages.microsoft.com/config/ubuntu/16.04/packages-microsoft-prod.deb \
 && dpkg -i packages-microsoft-prod.deb \
 && apt-get update \
 && apt-get install --yes --no-install-recommends \
    microsoft-r-open-foreachiterators-3.4.3 \
    microsoft-r-open-mkl-3.4.3 \
    microsoft-r-open-mro-3.4.3 \
 && Rscript -e 'install.packages("RcppEigen")'

The problem with MRO is that it does not include the MKL headers and only part of the MKL library:

ls /opt/microsoft/ropen/3.4.3/lib64/R/lib/libmkl_*
/opt/microsoft/ropen/3.4.3/lib64/R/lib/libmkl_core.so
/opt/microsoft/ropen/3.4.3/lib64/R/lib/libmkl_gf_ilp64.so
/opt/microsoft/ropen/3.4.3/lib64/R/lib/libmkl_gf_lp64.so
/opt/microsoft/ropen/3.4.3/lib64/R/lib/libmkl_gnu_thread.so
/opt/microsoft/ropen/3.4.3/lib64/R/lib/libmkl_vml_def.so
/opt/microsoft/ropen/3.4.3/lib64/R/lib/libmkl_vml_mc3.so

In particular libmkl_intel_lp64.so, which is requested in the linking command above, is missing. It is therefore not possible to use the above recipe with MRO. It might work to install MKL in addition to MRO and link with that, but I have not tested this.

However, MKL will be used whenever Eigen falls back to BLAS/LAPACK methods. This will speed up operations when compared to an R build using reference BLAS/LAPACK (default for Windows).

Ralf Stubner
  • 26,263
  • 3
  • 40
  • 75