Memory Management in Windows – Part 3: Physical Memory

Just like virtual memory, processes also have a limited amount of physical memory, referred to as the process’ Working Set. There are also System and Session Working Sets, which contain subsets of pageable code and data scoped to those spaces. While it is possible for a process to specify a minimum and maximum working set size, these limits are almost always ignored in practice. The default minimum and maximum per process is 50 and 345 pages, respectively. You can self-impose a hard cap on maximum working set if you’d like to do that for some reason. Without these self-imposed restrictions, working sets will grow or shrink beyond these limits as demand requires and resources allow.

When memory is getting low, working sets are trimmed by the working set manager, taking into account a number of factors including minimum working set sizes and the last time pages were accessed. Page faults under these conditions that require another page from the working set are handled by replacing pages on the working set and paging out old ones, rather than allocating new pages. It is possible to manually trigger the trimming of a process’ working set at anytime by setting the maximum working set size while the program is running. When memory is plentiful, the working set manager instead spends its time calculating how many pages could be removed from working sets should the need ever arise.

On a system-wide scale, the total memory available for allocation is referred to as the “system commit limit.” This comprises the sum of all physical memory available to the system plus the size of any page files. It is possible to run Windows with no page file at all (though this is almost universally discouraged), in which case the commit limit is simply the amount of physical memory available to Windows. Note that it is likely that this number does not coincide exactly with the amount of physical memory installed, as certain hardware reserves memory for itself independent of the operating system.

Any memory allocated against the system commit limit is considered the “system commit charge” and represents everything that must be kept either in RAM, or in the page file (so basically, anything that isn’t a file on disk other than the page file). When the system commit limit is reached, the system will attempt to increase the size of the page file. If this does not succeed (or, seemingly, if it can’t be carried out fast enough), memory allocations will fail. Each process also has a process page file quota, which tracks its contributions to the system commit charge. It’s worth noting that the commit charge and process page file quotas reflect the maximum theoretical, rather than actual, usage. Windows will not allocate any memory that it could not actually provide if necessary, even if many of those allocations have not and may never take place.

As you can imagine, pages added to a process’ working set are not chosen at random (well, they kind of are, but ASLR is another topic). Windows keeps track of every physical page of memory in the Page Frame Number database. These pages can be in one of nine states: Free, Zeroed, Modified, Modified No-Write, Standby, Transition, Active, Rom, or Bad. Active, or Valid, pages are either part of a working set or in use by some other means and typically have a valid PTE pointing to them. Transition pages are currently undergoing I/O, not part of a working set, and not on a page list. Modified no-write is a special case of the modified state where the page won’t be written to disk. This state is only used by file system drivers in specific scenarios. The other six states’ pages are each tracked in their own linked list for quick access by the memory manager.

Every page in the system starts out on the free page list, and returns there when it is no longer in use. These free pages are zeroed by the zero page thread and placed on the zero page list. Memory is (typically) pulled from the zero page list into a working set. When memory is trimmed from a working set, it either goes onto the modified or standby list. Modified pages have been changed in some way since they were last written to disk, and therefore must have their contents saved by the modified page writer before becoming standby pages. Standby pages can be reused immediately by whatever was using them previously, because their content has not changed since the last time it was written to disk. Rom pages are read-only, and bad pages have failed a consistency check should not be used. Most new allocations happen from the zero page list. Kernel mode processes are permitted to pull directly from the free page list in some cases, so long as the memory’s content has been overwritten before it makes it to user mode.