Evaluating Memory and Cache Usage

Investigating User-Mode Memory Leaks

A memory leak occurs when applications allocate memory for use but do not free allocated memory when finished. As a result, available memory is used up over time, often causing the system to stop functioning properly. Therefore, it is important to investigate the causes of all memory leaks, particularly on server computers. This section describes how to identify memory leaks, including ones that affect the critical nonpaged memory pool, and where to find tools and information related to working with memory leaks.

Note

It is sometimes possible to mistake an increase in system load for a memory leak. To distinguish between these conditions, observe the Memory and Process counters over a number of days. If you see the system first reach a steady state, then attain a level of increased load (usually achieved during some peak portion of the day), and then fall again, it is likely you are seeing variations in load rather than a leak. On network computers, look at user sessions and throughput rates such as transferred bytes per second to eliminate workload as a factor.

Identifying a Memory Leak

The symptoms of a memory leak include the following:

A gradually worsening response time.
The appearance of an error message, shown in Figure 6.7, indicating that the system is low on virtual memory. (Another message box might precede this, indicating that virtual memory has been exceeded and that the system has increased the paging file size automatically.)

Enlarge figure

Figure 6.7 Out of Virtual Memory Error Message

The appearance of error messages indicating that system services have stopped.

If you suspect that a particular application or service is causing a memory leak, investigate the memory use of your applications using the following counters:

Memory\Available Bytes reports available bytes; its value tends to fall during a memory leak.
Memory\Committed Bytes reports the private bytes committed to processes; its value tends to rise during a memory leak.
Process(process_name)\Private Bytes reports bytes allocated exclusively for a specific process; its value tends to rise for a leaking process.
Process(process_name)\Working Set reports the shared and private bytes allocated to a process; its value tends to rise for a leaking process.
Process(process_name)\Page Faults/sec reports the total number of faults (hard and soft faults) caused by a process; its value tends to rise for a leaking process.
Process(process_name)\Page File Bytes reports the size of the paging file; its value tends to rise during a memory leak.
Process(process_name)\Handle Count reports the number of handles that an application opened for objects it creates. Handles are used by programs to identify resources they must access. The value of this counter tends to rise during a memory leak; however, you cannot rule out a leak simply because this counter's value is stable.

Monitor these counters over a period ranging from two hours to a few days. Logging is recommended, both because of the overhead of monitoring multiple instances of the Process counters and because leaks tend to manifest themselves slowly.

In addition, to isolate the problem and avoid unnecessary overhead, monitor from a remote computer, if possible. Network activity or interaction with other computers can interfere with the results.

Memory Leaks and the Nonpaged Pool

Although any leak is serious, memory leaks are of particular concern when they involve the nonpaged pool. Many system services allocate memory from the nonpaged pool because they need to reference it when processing an interrupt and cannot take a page fault at that time. To identify whether or not a leak affects the nonpaged pool, include the following counters in your monitoring:

Memory\Pool Nonpaged Bytes
Memory\Pool Nonpaged Allocs
Process(process_name)\Pool Nonpaged Bytes

Note

Because the internal counters used by Task Manager, Process Monitor, and System Monitor to measure the size of the nonpaged pool for each process are not precise, it is recommended that you monitor changes in the overall pool size over time (a few days, for example), rather than rely on the absolute, instantaneous values reported for each process. The counter values are estimates that count duplicate object handles as well as space for the object. Also, because the process pool size counts are rounded to page size, pool space is overestimated when a process uses only part of a page. In contrast, total pool size counts are precise. Therefore, the sum of pool sizes for each process might not equal the value for the whole system.

The counters on the Memory object monitor the total size of the nonpaged pool and the number of allocations of pool space for the whole system. The counter on the Process object monitors nonpaged pool space allocated to each process.

To use System Monitor to monitor the nonpaged pool for leaks, follow these steps:

Record the size of the nonpaged pool when the system starts. Then log the Memory and Process objects for several days; a 10-minute update interval is sufficient.
Review the log for changes in size of the nonpaged pool. You should be able to associate any increases in the size of the pool, as indicated by Memory\Pool Nonpaged Bytes, with the start of a process, as indicated by Process\% Processor Time. Also look at individual Process object counters such as Process\Handle Count, Process\Private Bytes, Process\Nonpaged Pool Bytes, Process\Paged Pool Bytes, and Process\Threads. During a memory leak, you might also see rising values for these counters.
When processes are stopped, you should see a decrease in pool size. Any growth in the nonpaged pool is considered abnormal, and you need to distinguish which process is causing the change in pool size.

You might also want to monitor the number of active threads before and after running the process (use the Performance tab in Task Manager or the Objects\Threads or Process(_Total)\Thread Count counters). A process that is leaking memory might be creating a large number of threads; these appear when the process starts and disappear when the process stops.
Watch the value of Memory\Pool Nonpaged Bytes for an increase of ten percent or more from its value at system startup to see if a serious leak is developing.

The following additional tools provide information about the paged and nonpaged memory pools as listed in Table 6.3. These tools collect their data from the same sources.

Table 6.3 Tools That Provide Information About Memory Pools

Tool Name	Description	Location
Memsnap (memsnap.exe)¹	Records system memory usage to a log file.	Windows 2000 Support Tools
Process Monitor (pmon.exe)¹	Provides total and per process values for nonpaged and paged pool memory. Also monitors the committed memory values shown in the Pmon display for increases; the process with the leak should have an increasing value reported under Commit Charge.	Windows 2000 Support Tools
¹ These tools are useful because they show allocations on a per-process basis.

For information about installing and using the Windows 2000 Support Tools and Support Tools Help, see the file Sreadme.doc in the \Support\Tools folder of the Windows 2000 operating system CD.

For a quick demonstration of a memory leak, start LeakyApp, a test tool on the Windows 2000 Resource Kit companion CD, and observe the values of the monitored counters. Notice the steady increase in the following counters: Memory\Pages/sec, Process(LeakyApp)\Working Set, and Process(LeakyApp)\Private Bytes.

Figure 6.8 illustrates counter activity during a memory leak generated by the LeakyApp tool.

Enlarge figure

Figure 6.8 Process Memory Activity During a Memory Leak

Although the memory leak illustrated in Figure 6.9 has a systemic effect, the problem can be tracked to a single cause—the leaking application. If you have an application that exhibits similar behavior, it is recommended that you either modify it (if you have access to the source code) or replace it with another program.

Developer tools for analyzing and tuning memory usage by applications are available on the Windows 2000 Resource Kit companion CD. For more information about developer tools, see the MSDN link on the Web Resources page at http://windows.microsoft.com/windows2000/reskit/webresources.

The following tools optimize memory-intensive applications.

Application Monitor (ApiMon) on the Windows 2000 Resource Kit companion CD monitors page faults caused by an application and reports them by Microsoft® Win32® function calls.
The Working Set Tuner (WST) on the SDK analyzes the patterns of function calls in your application code and generates an improved function ordering to reduce physical memory usage. One objective of tuning your working set is to arrive at the smallest possible working set without causing page faults.
Virtual Address Dump (Vadump) on the SDK creates a list that contains information about the memory usage of a specified process, including address size, total committed memory for the image, the executable, each dynamic-link library (DLL), and heap usage.
The nonpaged pool size and the paged pool size are set by default during Windows 2000 Setup based on your memory configuration. The maximum nonpaged pool size is 256 MB. The maximum paged pool size is approximately 470 MB. The actual size varies depending on your configuration. More physical memory results in less paged pool because the virtual address space must instead be used to contain more critical memory-management structures. The pool sizes are defined in the registry in the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Memory Management subkey.

To extend the file system cache working set from 512 MB to 960 MB, set the value of the PagedPoolSize registry entry to 192000000, set SystemPages to 0, and make sure that the system is optimized for file sharing with LargeSystemCache set to 1. To make the maximum virtual address space available to the paged pool, set the PagedPoolSize registry entry to – 1, provided your system is not using the /3GB Boot.ini switch.

Typically you do not need to set the NonPagedPoolSize entry because on systems with more than 1.2 GB of memory, the system automatically defaults to the maximum nonpaged pool size. If you need to set the NonPagedPoolSize value, set it to the value you want (in bytes); do not set it to – 1.