1

I am currently using the future package in R for some heavy parallel processing tasks.

When I examined the CPU usage while the script was running, I noticed that each parallel section is using only 2.3% of the CPU power on the machine (see below). Is there a way to increase the usage to a higher number (say 5% or 10%)?

screenshot

Sorry if I missed anything obvious from the package documentation.

johnlee14
  • 65
  • 4
  • You should maybe tell us more about what yo actually do, and even provide some code. The best way to parallelize is really case-specific. Usually, when CPU usage is low, it means that you're not really computing most of the time, but rather exporting data to clusters or reading from disk. Also, the first question would be, do you get 100% usually when disabling parallelization? – F. Privé Jan 31 '21 at 06:58

1 Answers1

0

Your script (or any process) will only use what is needed. While I am not familiar with the exact workings of future, unless you can define a treshold for CPU-usage there are no direct ways of increasing this arbitrarily.

If your script is still slow then you need to look at other (or additional ways) of speeding it up. You should also look into if the overhead of paralellisation is causing unnecessary workloads, perhaps try with less cores/workers and see if that increases the CPU-usage, and evaluate that against a benchmark (e.g. time to completion).

mhovd
  • 3,724
  • 2
  • 21
  • 47