2

I have a python script which I run on my old quad-core laptop through multiprocessing. The simulation cannot be parallelised, I just run different instances of it on the different cores.

I was considering renting some more powerful cpus on google compute engine. Will I be able to use the same python multiprocessing script just with more cores available?

the script just calls a pool and then apply_async many times

tidus95
  • 359
  • 2
  • 14

2 Answers2

1

Multiprocessing works the same way on Google's VMs as on your local box (roughly...). In order to use all cores available on a box, do not pass processes argument to multiprocesing.Pool, like:

pool = multiprocessing.Pool(processes=None)

This will make Python create a Pool with multiprocessing.cpu_count() number of processes, cpu_count usually being a number of cores.

Marcin
  • 4,080
  • 1
  • 27
  • 54
0

Yes it should be possible. If you need access to multiple cores on the same instance, as an example the App Engine flexible runtime, which uses Compute Engine as the underlying VM, allows you to configure the number of cores available

dany L
  • 2,456
  • 6
  • 12