January 1999

Code for this article: Jan99Nerd.exe (92KB)

James Finnegan is a developer at Communica, Inc., a Massachusetts-based company specializing in system software design. He can be reached via http://www.windows.to or jimf@nema.com.

Welcome to Jim's Nerditorium. Within these pages you can look forward to a smattering of highly relevant system-level tips, tricks, and techniques to illustrate practical, real world uses of operating system resources usually found within the periphery of the mainstream documentation. What does that mean?       The aim of this column is to promote Windows NT® and Windows® 98 system-level development. As I showed in my March 1998 article in MSJ, "Pop Open a Privileged Set of APIs with Windows NT Kernel Mode Drivers," kernel mode development is not just for driver writers. It is for anyone who needs to delve in to areas that are usually restricted or prohibited by today's modern operating systems. In my opinion, especially when dealing with Windows NT, having a core understanding of how things work is critical. It is seldom argued that the ubiquitous success of Windows 9x is largely due to its stunning compatibility and ability to unify diverse operating environments practically flawlessly. What did it was a true understanding of how things work—largely by driver writers. It's time to promote this attitude to the operating systems of the future.       A number of things attract me to this type of development, and it has nothing to do with the hordes of gorgeous supermodels who think that kernel-mode development is just oh so cool. System-level development has a certain level of timelessness about it. Although the facades of operating systems are in flux, their core remains tried and true. Matt Pietrek has often referred to MSJ as an encyclopedia that you purchase in installments. Much of its content is referred to by developers for years on end. I'm hoping that the information you find here is something you'll find yourself referring to for a long time.       Don't get me wrong; I'm not here to teach you how to hack, crack, circumvent, spindle, or otherwise mutilate the operating system. Rather, I've personally found that the difference between mediocre software and great software is the exploration and mastery of a computer's capabilities by the software's author. Of course, this surfs on the hairy edge of exploitation, and you must wield this power with caution. Now, let's rock! Windows NT Process Monitoring       Matt Pietrek's Under the Hood column has often treated you to many of the underutilized APIs that permit you to view processes and threads within Windows NT. From a user-mode point of view, these APIs provide a fairly powerful ability to see "things" on a global basis. Otherwise, Win32® applications are restricted to their own little sandbox in which other things happening within a system are hidden from view.

Figure 1 Viewing Processes

Although these APIs are powerful, they do not permit you to view systemwide operations as they happen. They let you take a static snapshot of the world as it currently exists for examination. Clearly, this may not be appropriate under certain circumstances. Within this column I'll show how to utilize some little-known kernel-mode APIs to view process, thread, and image loading as it happens.       If you look at the Windows NT Task Manager (TaskMgr.exe), in the Processes tab (see Figure 1) you'll see a list of current user-mode processes running within your system. You'll also notice that the listbox is periodically cleared and repopulated with current data, creating a flickering effect. Taskmgr.exe utilizes a mix of the Process Status API (PSAPI.DLL) and the Virtual DOS Machine Debug API (VDMDBG.DLL) to present a uniform list of 16-bit and Win32-based applications (including MS-DOS® boxes, represented by NTVDM.EXE).

Figure 2 CheezMan

To demonstrate the limitations of the aforementioned APIs, I've implemented a lightweight version of the Windows NT Task Manager called CheezMan (for Cheezy Task Manager, see Figure 2). CheezMan uses a combination of PSAPI.DLL and VDMDBG.DLL to present a result similar to TaskMgr.exe. If you examine the source code of CheezMan (see Figure 3), you'll see that a timer is set up to periodically enumerate and display a list of Win32 processes and 16-bit Windows tasks at the current moment in time. This is done within the EnumerateProcesses function through the PSAPI EnumProcesses API call:

// Get a list of all current PIDs running if(!EnumProcesses( aProcesses, sizeof(aProcesses), &cbNeeded)) return; // Get the count of PIDs in the array cProcesses = cbNeeded / sizeof(DWORD);

For each process ID that is returned in the array, my function PrintProcessName is called. PrintProcessName obtains that process name by opening a handle to the process first:

// Get a handle to the passed-in process ID HANDLE hProcess = OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_VM_READ, FALSE, processID );

PrintProcessName then enumerates all modules within a process with EnumProcessModules, and gets the name of the first process in the list by calling GetModuleBaseName. The first module is always the executable's name:

if(EnumProcessModules(hProcess, &hMod, sizeof(hMod), &cbNeeded)) { GetModuleBaseName(hProcess, hMod, szProcessName, sizeof(szProcessName)); }

If the process name is NTVDM.EXE, PrintProcessName will then try to enumerate each 16-bit Windows task by calling VDMEnumTaskWOWEx (located in VDMDBG.DLL).

// If the process name is NTVDM.EXE, try walking it to // see if it is a WOW box. The callback will not be // called if NTVDM is not WOW if(!stricmp(szProcessName,"ntvdm.exe")) { // VDMEnumTaskWOWEx() is NOT documented in the VDMDBG.HLP // file—but it is in VDMDBG.H. Explanations there make // its use clear (and documented!) Basically, you need // modname and filename as additional parameters in the // callback. VDMEnumTaskWOWEx(processID, EnumerateWin16Processes, 0); }

As noted in the source code, if NTVDM.EXE is really running an MS-DOS-based application, rather than hosting a WOW box, VDMEnumTaskWOWEx blissfully refuses to call your callback function (referenced in the second parameter). Therefore, no additional check is required on your part to determine if the VDM is really a WOW session.       The callback function used by VDMEnumTaskWOWEx receives a plethora of parameters, as can be seen in the prototype of EnumerateWin16Processes:

BOOL WINAPI EnumerateWin16Processes(DWORD dwThreadId, WORD hMod16, WORD hTask16, PSZ pszModName, PSZ pszFileName, LPARAM lpUserDefined)

As you can see, VDMDBG.DLL leaves little to the imagination regarding the identity of a specific 16-bit task by passing the Win32 thread ID, 16-bit task and module handles, as well as the module and file names of the task. This callback is called for each task that is present in the WOW box, making enumeration of the module names child's play.       Running CheezMan gives you a fairly good view of the world as a whole with very little code and effort. All of this is done courtesy of documented user-mode APIs. Very cool.       The limitations of CheezMan (and TaskMgr) are obvious. When run, the flickering effect of the continual refreshing of the listbox is pathetic (although TaskMgr is far more graceful than my grotesque CheezMan). In addition, taking periodic snapshots of the system is inefficient and leaves quite a bit of room for things to fall through the cracks. Processes and threads can easily come and go right in between polling.       Clearly, what you need is an API like NotifyRegister in the 16-bit Windows ToolHelp.DLL. With NotifyRegister, you pass a callback function that gets called as interesting things happen, such as the loading and unloading of tasks and DLLs. Doing this within 16-bit Windows is straightforward, since all applications are sharing one big happy address space, and ToolHelp has intrinsic knowledge of the internal structure of Windows task and module lists.       No such user-mode API exists in Win32, where you can watch the action happen globally, from within a single user-mode app. But things that seem impossible in user mode are usually trivially easy in kernel mode. Kernel-Mode Party Time       As I demonstrated in my March 1998 MSJ article, kernel-mode drivers are not just for devices. In fact, I like to think of them more as system-level DLLs. In this light, using a driver as a surrogate to perform far-reaching tasks on your behalf becomes easy. But all of the unrestricted access in the world does you no good unless you know what you're looking for. Fortunately, there's a plethora of kernel-mode APIs at your disposal to do many non-hardware-related things. These APIs were probably initially created to solve some specific problem with hardware support. But there's nothing restricting you from utilizing them for your own use.       An example of this is the Process Structure API. These functions, which have been historically buried in the fringes of the DDK documentation, permit you to see the process, thread, and image loading action as it happens (see Figure 4 for the complete API). The Process Structure API's documentation has recently been expanded within the Windows 2000 DDK to include functions that are of interest here. But like many other useful DDK APIs, these APIs have existed since Windows NT 4.0 (albeit undocumented). Although I'd like to give you permission to utilize these APIs within Windows NT 4.0, be aware that their use under any operating system earlier than Windows 2000 is not recommended.       Inclusion of these functions in the Windows 2000 kernel-mode API is largely for drivers that need to track processes and threads for the purpose of creating or maintaining separate contexts for individual processes and threads (such as process-local file handles within a file system). However, there's nothing restricting you from snooping in on this activity and reporting the results back to user-mode.       The more interesting of the Process Structure APIs (PsSetCreateProcessNotifyRoutine, PsSetCreateThreadNotifyRoutine, and PsSetLoadImageNotifyRoutine) offer quite a powerful set of notification capabilities to kernel-mode drivers. I'll show how to utilize these APIs and retrieve the results within my user-mode application, ProcView.exe. ProcView is dependent on ProcView.sys to monitor these kernel-mode events and notify the user-mode app when something interesting has happened. Although ProcView could be far more complex and feature-rich, I've left off the fancy tricks to ensure the clarity of the APIs used here. ProcView Kernel-Mode Driver       The PsSetCreateProcessNotifyRoutine and PsSetCreateThreadNotifyRoutine APIs permit you to register a callback function that is invoked each time a process or thread is created or deleted systemwide. The PsSetLoadImageNotifyRoutine API allows you to examine the image loading of an executable or DLL as well as on a systemwide basis. Each of these APIs permits chaining for up to eight kernel-mode callback functions. With the current beta release of Windows 2000, you cannot unregister a callback function once it is set up. This means that a driver that utilizes these functions must remain loaded and running until Windows 2000 is shut down.       Use of these APIs is straightforward. Each takes a pointer to your defined callback function (see Figure 5 for the function prototypes). PsSetCreateProcessNotifyRoutine takes an additional boolean parameter called Remove. As is the case with so many unutilized API parameters, you must pass zero here. However, based on the parameter's name, you could imagine the shape of things to come in future operating systems.       As seen in ProcView.c (see Figure 6), the callback functions are set up all at once within DriverEntry (for more information about the general structure of a kernel-mode driver, see my March 1998 article). Since callbacks cannot be unregistered, there is no point to do this elsewhere, such as within the context of an IOCTL.

// Set up callback routines... ntStatus = PsSetCreateProcessNotifyRoutine( ProcViewProcessCallback, 0); ntStatus = PsSetCreateThreadNotifyRoutine( ProcViewThreadCallback); ntStatus = PsSetLoadImageNotifyRoutine( ProcViewImageCallback);

Each of these callbacks simply takes the passed-in parameters and stores the values in the device object's device extension. As you can see, obtaining this data is not an issue. But what do you do with it once you have it? In the case of internally tracking this activity wholly within a device driver, maintaining a list of currently running processes is fairly straightforward. However, the intention here is to perform some user-mode snooping. How do you tell user-mode that your kernel-mode code has seen something interesting?       Win32-based apps traditionally utilize events to signal that interesting things have occurred. This permits you to create some efficient code that gets signaled asynchronously, only being notified when further intervention is required. Doing this wholly within user mode is trivial, but how do you do this between your two disparate kernel-mode and user-mode components?       The Windows 2000 I/O Manager has provisions for creating named kernel-mode objects that can be attached to and monitored by user-mode processes. Within DriverEntry, I created three event objects, one each for process, thread, and image notification:

// Create events for user-mode processes to monitor RtlInitUnicodeString(&uszProcessEventString, L"\\BaseNamedObjects\\ProcViewProcessEvent"); extension->ProcessEvent = IoCreateNotificationEvent (&uszProcessEventString, &extension->ProcessHandle); RtlInitUnicodeString(&uszThreadEventString, L"\\BaseNamedObjects\\ProcViewThreadEvent"); extension->ThreadEvent = IoCreateNotificationEvent (uszThreadEventString, &extension->ThreadHandle); RtlInitUnicodeString(&uszImageEventString, L"\\BaseNamedObjects\\ProcViewImageEvent"); extension->ImageEvent = IoCreateNotificationEvent (&uszImageEventString, &extension->ImageHandle);

Within main.c of ProcView.exe, these kernel-mode events are attached to by referencing their names within the Win32 OpenEvent API call:

EventArray[0] = OpenEvent(SYNCHRONIZE, FALSE, "ProcViewProcessEvent"); EventArray[1] = OpenEvent(SYNCHRONIZE, FALSE, "ProcViewThreadEvent"); EventArray[2] = OpenEvent(SYNCHRONIZE, FALSE, "ProcViewImageEvent");

Once these event handles are obtained within user-mode, they can be utilized and referenced just like any other user-mode created event handle.       Now that you see how the event handle glue is created between kernel and user-mode, signaling that your kernel-mode driver has seen something interesting is as simple as signaling the event handle:

// Pulse the event so any user-mode apps listening will // know something interesting has happened. Sadly, // user-mode apps can't reset a KM event, which is // why we're pulsing it here... KeSetEvent (extension->ProcessEvent, 0, FALSE); KeClearEvent (extension->ProcessEvent);       From user mode, ProcView.exe waits on the attached event handles until one of them has been signaled: WaitForSingleObject(ProcessEvent, INFINITE);

Unfortunately, user-mode Win32-based apps do not have the ability to alter the state of a kernel mode event, which means that it cannot be manually reset from your user-mode app. This is why the event handle is "pulsed" in kernel-mode, rather than being set in kernel-mode and reset in user mode, which would be ideal.       Obtaining the collected kernel-mode data simply requires you to issue an IOCTL to your driver:

bReturnCode = DeviceIoControl(hDriver, IOCTL_PROCVIEW_GET_PROCINFO, 0, 0, &CallbackInfo, sizeof(CallbackInfo), &dwBytesReturned, &ov);

Once this data is retrieved, the process and thread IDs can be accessed just like the IDs that are retrieved from other user-mode APIs (like CheezMan earlier). In fact, ProcView.exe utilizes functions similar to those in CheezMan to display meaningful data (like the process's name), drawing upon the untranslated data obtained from kernel-mode. Easy, huh? Conclusion       Although you probably feel like I'm beating you over the head with it, I need to say it again: kernel-mode is not just for devices! In fact, kernel-mode and user-mode are not all that different. Objects such as the event handles I've shown you can be referenced on both sides of the fence. In addition, collected data can be used and referenced without abandon. Blurring the dividing line that usually exists between user and kernel mode will help you create those great apps. Have a suggestion for Nerditorium? Send it to Jim Finnegan at jimf@nema.com.

From the January 1999 issue of Microsoft Systems Journal.