1

I have a question regarding the Array operations in Eigen (basically matrix element-wise operations).

Are such operations (+,-,*,/) parallelized in Eigen (when using OpenMP)? The doc does not specify it (c.f. here), however such operations would be expected to be parallelized since it would be pretty straightforward I guess.

Example:

MatrixXd A = MatrixXd::Zero(100,100);
MatrixXd B = MatrixXd::Ones(100,100);

MatrixXd C = A.array() + B.array(); // element-wise addition
MatrixXd D = A.array() / B.array(); // element-wise division

It would be great if it was parallelized. I have a lots of these element-wise operations in my code, and it would be heavier to redefine all of these with OpenMP.

Thanks in advance

Odin
  • 633
  • 4
  • 11

1 Answers1

0

The Eigen web site lists the few cases that take advantage of multithreading.

Currently, the following algorithms can make use of multi-threading:

general dense matrix - matrix products
PartialPivLU
row-major-sparse * dense vector/matrix products
ConjugateGradient with Lower|Upper as the UpLo template parameter.
BiCGSTAB with a row-major sparse matrix format.
LeastSquaresConjugateGradient

This does not exclude SIMD operations, so those will still be used.

Avi Ginsburg
  • 10,323
  • 3
  • 29
  • 56
  • Indeed, I had a look at this page (c.f. my post). I don't understand what you mean by "This does not exclude SIMD operations, so those will still be used.", do you mean that SIMD operations as array operations support multi-threading in Eigen? Thanks – Odin Apr 20 '17 at 11:04
  • He probably means that you will still have SIMD (vector) operations within each thread – Cantfindname Apr 20 '17 at 14:50
  • Ok, thanks. At least, if it is parallelized on the rows or the columns, it will be more efficient, than purely sequential computations I guess ? – Odin Apr 20 '17 at 15:20
  • I meant that you will have SIMD (vector) operations within the *one* thread (unless you manually introduce more). It will be vectorized column wise, as that's the storage order for `MatrixXd`. You can group the operations together to minimize the number of temporaries, as well. – Avi Ginsburg Apr 20 '17 at 15:25
  • Thanks for your answer. Ok, I understand. Vectorized operations will speed up the computations on arrays. However, it would be even more efficient if these array operations would be multi-threaded. I use several cores in other part of my code, these supplementary cores are unused during array operations, which is a waste of resources. I guess, I will have to redefine array operations by myself to benefit from multi-threading. I will get in touch with the Eigen team to see if this is planned or if I can contribute. Thanks again – Odin Apr 21 '17 at 08:02
  • You can write your expressions as blocks using OMP's thread index for separating them. See for [example](http://stackoverflow.com/questions/41845724/eigen-openmp-no-parallelisation-due-to-false-sharing-and-thread-overhead/41847107#41847107). – Avi Ginsburg Apr 21 '17 at 08:35