How do you measure the memory footprint of a set of forked processes?

Question

Say I've got a process using 200MB of memory, and it fork()s:

python -c "import os; data='x'*200000000; os.fork(); raw_input()"

Programs like 'top' will show each process using 200MB, with very little SHRd memory, so it appears as if the processes are using 400MB in total. However, because fork() implements copy-on-write (COW) for the process's memory pages, the reality is that the processes are only using 200MB in total.

Why doesn't top show what fraction of the memory is COW? Is there a way to make it do so? Or is there another command I can use instead?

Note: 'top' on OSX seems to have an RSHRD column that does what I'd expect. My question is for Linux.

score 5 · Accepted Answer · edited Apr 13 '17 at 12:37

You can get that sort of information from the /proc/<pid>/smaps file for each process in the form of the Pss entry (short for "Proportional share size").

In the above example with 200MB "shared" between two processes, each process would show 100MB in the PSS entry for that mapping, i.e. the memory is distributed evenly between the processes that share it (until it gets copied and unshared by either process).

Here's an extract from running something like what you posted:

$ top
...
30986 me        20   0  790m 769m 2200 S    0  4.8   0:00.48 python3.2
30987 me        20   0  790m 767m  224 S    0  4.8   0:00.00 python3.2
$ cat /proc/30986/smaps
...
0119a000-015b7000 rw-p 00000000 00:00 0                                  [heap]
Size:               4212 kB
Rss:                3924 kB
Pss:                1992 kB
...
7fa06b020000-7fa09ab11000 rw-p 00000000 00:00 0 
Size:             781252 kB
Rss:              781252 kB
Pss:              390626 kB
...
7fff8e45a000-7fff8e489000 rw-p 00000000 00:00 0                          [stack]
Size:                192 kB
Rss:                 160 kB
Pss:                  82 kB
...

(There's a lot of stuff in those files, including mappings for shared libraries that are potentially shared between many processes, and thus each process gets only a small part accounted in its PSS entry for those.)

Here's a nice article about this: ELC: How much memory are applications really using?

I don't know of a common top-like tool to display this information, and I don't think ps has options to show that either unfortunately. The article points to a repository with python scripts called pagemap by Matt Mackall that you could use or adapt though.

_{Shameless plug: you'll find a few posts on Unix & Linux about PSS and the smaps file if you're interested in that.}

How do you measure the memory footprint of a set of forked processes?

1 Answers1