Under Windows, what is a processes "working set"?

Question

To date I haven't had a satisfactory answer to this question, but I am constantly gob-smacked by the obscene number under the "Working Set" column in Sysinternals Process Explorer. Can someone define this in an easy-to-understand manner?

Answer 1

The working set can be defined as :

Working Set Bytes = Sizeof(RAM) – (Available Bytes + Pool Nonpaged Bytes + Pool Paged Resident Bytes + System Cache Resident Bytes + System Code Resident Bytes + System Driver Resident Bytes)

wifth the following definitions :

Pool Nonpaged Bytes: these represent allocations directed to the nonpaged pool, which is a set virtual memory pages that always remain resident in RAM. (These are nonpageable bytes.) Device drivers and the OS use the nonpaged pool to store data structures that must stay in physical memory and can never be paged out to disk. (For example, the TCP/IP driver must allocate some amount of nonpaged memory for every TCP/IP connection that is active on the computer for data structures that are required during processing of network adaptor interrupts when page faults cannot be tolerated.)

Pool Paged Resident Bytes: Most virtual memory pages that are acquired in the Operating System range of virtual addresses can be paged out. The Pool Paged Resident Bytes represent memory locations from the pageable pool that currently reside in RAM. System Cache Resident Bytes: the system’s file cache occupies a reserved range of virtual memory addresses, some of which may currently reside in RAM. (Cached file segments can also be non-resident, in which case they must be fetched from disk when they are referenced by executing processes.)

System Cache Resident Bytes : represents segments of the file cache that are currently resident in RAM.

System Code Resident Bytes: memory locations associated with system code that is currently resident in RAM.

System Driver Resident Bytes: memory locations associated with device driver code that is currently resident in RAM.

Answer 2

Here is my go at an easy-to-understand explanation.

A process working set is the amount of physical RAM it is currently using (referencing), in total, to do its work.

It can be a little confusing because it counts "shared" space in the RAM as well. For example, if process1 needs to run some system dll, the OS will read it into RAM. If process2 also needs to run that system dll, the OS doesn't need to read it into RAM, as it is already there. HOWEVER, the amount of RAM used to store that system dll code will show up under the working set of both processes, as they are both using that code to do their work.

So when you see a process with a huge working set, there may be a fair amount of that RAM it is claiming to use that is actually being shared with other processes, like the OS.

Answer 3

See MSDN?

"The working set of a program is a collection of those pages in its virtual address space that have been recently referenced".

So in short: ranges of addresses (pages) that are currently being directly used by the app.

Answer 4

The working set is the set of pages of RAM a process needs at any given time. It can vary with time, grow, or shrink. Imagine a spreadsheet: most of the time, you add or change values near the end of the sheet, but every night you run a report working with the entire sheet. Since this is stored in RAM, the working set will be small during the day, and grow to cover the entire dataset when the report is run.

This is important because if your program's working set is bigger than RAM, you'll start thrashing--continually swapping out pages of RAM on disk. The above example may be fine or unavoidable if you only access each cell once, but if you repeatedly need everything, you're effectively limited to the speed of swap. Cheapest solution: add more RAM. More complicated solution: rewrite your program to be cache aware (access (m*n) arrays with a stride of 1, 2, 3, 4, ... m+1, m+2, rather than 0*m+0, 1*m+0, 2*m+0... n*m+0, 0*m+1... n*m+n.

Answer 5

The process working set is those pages of the process's virtual address space that are currently in-memory (and marked as belonging to that process's working set). Thus it is also the amount of physical pages (or pretty close) currently allocated to the process

These are usually the in-use and most-recently used pages, on the theory that these are also the pages that it will most likely need in the future, but not necessarily. It is quite possible for a process working set to include many pages that it has not used recently, or that it may never need in the future (i.e. "bloat").

This is a general tendency in most virtual memory OS's because there is no reason not to give a process more physical memory (and thus, working set) if no other process needs it.

Note that WS (working set) is not the amount of physical space that a program needs, it is the amount the it currently has.

Answer 6

Here is a visualized representation of the scientific definition of the working set model by Peter Denning (which not necessarily would match what operating systems and especially the Windows Task Manager would report to you).

Suppose you have a set of processes: {P1, P2, P3, P4}

And suppose you have the following list of page references by the processes over time:

    T-5  T-4  T-3  T-2  T-1
---------------------------
P1:  2,   3,   5,   4,   5
P2:  2,   3,   4,   1,   2
P3:  2,   3,   1,   4,   5
P4:  4,   5,   5,   2,   5 

Lets also suppose you have a delta (dt) of 4.

Your working set is the set of all unique page references from the time T-delta to T.

The working set for each process at time T is as follows:

P1: {3,4,5}
P2: {1,2,3,4}
P3: {1,3,4,5}
P4: {2,5}