1

I've been using the SBM algorithm, implemented in the graph-tool package. I need to process a huge amount of data and need to run it in parallel.

I know that OpenMP is activated by default in this package and used in the specific and compatible algorithms, but the documentation doesn't specify which algorithms.

I've tried openmp_enabled() or openmp_set_num_threads() and also export OMP_NUM_THREADS=16. Everything seems fine but when I check the running processes, it's not paralleled.

Do you have any experience with implementing SBM parallelized?

Saeed Choobani
  • 21
  • 5

1 Answers1

2

Although graph-tool uses OpenMP, not every algorithm is implemented in parallel, simply because this cannot be done in some cases. The SBM inference algorithm implemented in graph-tool is based on MCMC, which cannot be parallelized in general. Because of this, enabling OpenMP will have no effect.

Tiago Peixoto
  • 5,149
  • 2
  • 28
  • 28
  • Thank you, dear Tiago. It would be amazing if there was a shortlist of available algorithms that can be parallelized in graph-tools. – Saeed Choobani Dec 06 '20 at 17:11
  • @Tiago, I'm confused. Isn't the `(multiflip_)mcmc_sweep()` function parallelized via OpenMP? When I've used `minimize_blockmodel_dl()` in the past, I notice my CPU cores oscillating between full utilization (all cores) to single-core while the function runs, indicating that at least some of the computation is parallelized. Wasn't that due to [this line](https://git.skewed.de/count0/graph-tool/-/blob/15a33b72ca2cc440885cf145fb888c7f205862ed/src/graph/inference/blockmodel/graph_blockmodel_multiflip_mcmc.cc#L110)? – Stuart Berg Dec 07 '20 at 17:32
  • 1
    @StuartBerg That line has to do with the *parallel tempering* algorithm (which runs in parallel), not the regular MCMC code (which does not)! The function minimize_blockmodel_dl() also has a block merging phase which *does* run in parallel, but that is a minor part. – Tiago Peixoto Dec 07 '20 at 21:01
  • @SaeedChoobani For all functions in graph-tool (singular) that are implemented in parallel, this is mentioned in the documentation. If this is not mentioned, then it's not the case. – Tiago Peixoto Dec 07 '20 at 21:03