1

I am currently developing a python script that uses the multiprocessing module (more specifically, multiprocessing.Pool) to split the workload of populating some files with data held within large (several GBs) data structures in memory. At first I was concerned that scaling this up would be a problem if processes copy the entire process state (including the large data structures) across to each sub-process after forking the main process. However, reading up on how to reduce this memory footprint led me to the information that recent versions of Linux support copy-on-write when forking a process, meaning that the total physical memory footprint is significantly reduced.

However, I am unsure whether the version of Linux that I am using is sufficiently recent to have this functionality. Is there a relatively simple way I can find out?

Rob Streeting
  • 1,675
  • 3
  • 16
  • 27

0 Answers0