How to measure memory without copy-on-write pages?

Question

How can one measure total RSS memory occupied by a set of (forked) processes, without double-counting shared and copy-on-write pages?

Answer 1

I ended up writing my own utility for this: https://gist.github.com/Eugeny/04ccfe8accf4bc74b0ca

I ran it against init (pid 1) for testing and the total reported roughly equals the physical ram usage (as per htop), so I guess it's somewhat correct.

Example use:

~ » pstree -ap 15897
zsh,15897
  └─sudo,9783 make rundev
      └─make,9784 rundev
          └─sh,9785 -c cd ajenti-panel && ./ajenti-panel -v --autologin --plugins ../plugins --dev
              └─python ./ajenti,9786
                  ├─./ajenti-panel ,9834                              
                  ├─./ajenti-panel ,9795                     
                  └─{python ./ajenti},9796

~ » sudo ./memuse.py 15897
PID                 Commandline                          Frames (+unique)           VMEM
 - 15897            (/usr/bin/zsh                  ):      1776  +1776           7104 KB
  -  9783           (sudo make rundev              ):       608  +408            2432 KB
   -  9784          (make rundev                   ):       261  +98             1044 KB
    -  9785         (/bin/sh -c cd ajenti-panel && ):       166  +48              664 KB
     -  9786        (python ./ajenti-panel -v --aut):      9279  +8977          37116 KB
      -  9795       (./ajenti-panel worker [restric):      7637  +1334          30548 KB
      -  9834       (./ajenti-panel worker [session):      8972  +2639          35888 KB
----------------------------------------------------------------------------------------
TOTAL:                                                    15280                 61120 KB

Answer 2

There is no clearly defined way of determining in any tool I know of which processes share which maps without iterating through all the mappings and comparing addresses.

However, linux does offer reasonable estimate known as the Proportional set size. This is reported in /proc/[pid]>/maps.

This value is the size of the mapping divided by the number of siblings/parent processes with the same mapping open.

So, with a program that has a 1MiB mapping open, plus a 1MiB shared with 4 other processes, the proportional set size is 1MiB + (1Mib / 4) or 1.250 MiB. The RSS in this case would be 2MiB.

There is a patch for htop floating around which will use the PSS to calculate a 'good estimate' of actual memory in use.