Can the total of all rss values of active processes (state is S or O) be ever much greater than total physical memory + swap?
By physical memory, I refer to the result of prtdiag | grep Memory
And swap size, I refer to the result of swap -l.
Easily.
Physical memory that is shared between processes - either IPCS shared memory or some of the mmap()'d physical memory used by shared objects - will be counted in the RSS of every process that maps that memory into its address space. There's no easy way to account for this when the starting point for counting memory usage is enumerating processes - you'd have to do a virtual-to-physical mapping for all the virtual pages for each process then eliminate duplicates while hoping the memory mappings being examined didn't change while totals were being calculated.
A good example of this is an Oracle database server with a large SGA - which is implemented on Solaris using IPCS shared memory. If 20 processes map a 32 GB shared memory SGA, that would contribute 640 GB to the sum of the processes' RSS, even though only 32 GB of RAM is used.
