HOWTO: Use VDMDBG Functions on Windows NT

• Microsoft Win32 Application Programming Interface (API) included with:

  - Microsoft Windows NT 4.0

SUMMARY

Even when programming in a 32-bit environment, there are times when working with 16-bit applications is necessary. In Windows NT, 16-bit applications run within a Virtual DOS Machine (VDM). VDMDBG.dll contains many functions that are useful for working with 16-bit applications. This library is part of the Microsoft Platform SDK.

VDMDBG functions provide a good way to enumerate, create, and terminate 16- bit processes (tasks) within a VDM. This article describes how to use these functions on Windows NT.

MORE INFORMATION

When you use any of the VDMDBG functions, you need to link VDMDBG.lib with your project.

The VDMDBG functions mentioned below are only a subset of the available functions. However, the functions that are not mentioned are only relevant to a debugger.

Enumerating VDMs

The VDMEnumProcessWOW() function provides a simple way to enumerate all VDMs running 16-bit Windows tasks. These VDMs contain the WowExec.exe task. DOS VDMs are not enumerated.

The declaration for this function is as follows:

   INT WINAPI VDMEnumProcessWOW( PROCESSENUMPROC fp, LPARAM lparam );

PROCESSENUMPROC is declared as follows:

   typedef BOOL ( WINAPI *PROCESSENUMPROC )(
      DWORD dwProcessId,
      DWORD dwAttributes,
      LPARAM lpUserDefined
   );

Enumerating 16-bit Windows Tasks

You can use VDMEnumTaskWOW() and VDMEnumTaskWOWEx() to enumerate tasks within a particular VDM. The difference between the two is that VDMEnumTaskWOWEx() provides more information to the callback function. You should only use VDMs returned by VDMEnumProcessWOW() with these task enumeration functions. Using DOS VDMs does not make sense because every DOS application runs in its own VDM.

The declarations are:

   INT WINAPI VDMEnumTaskWOW( DWORD dwProcessId, TASKENUMPROC fp,
                              LPARAM lparam );

   INT WINAPI VDMEnumTaskWOWEx( DWORD dwProcessId, TASKENUMPROCEX fp,
                                LPARAM lparam );

TASKENUMPROC and TASKENUMPROCEX are defined as follows:

   typedef BOOL ( WINAPI *TASKENUMPROC )(
      DWORD dwThreadId,
      WORD hMod16,
      WORD hTask16,
      LPARAM lpUserDefined
   );

   typedef BOOL ( WINAPI *TASKENUMPROCEX )(
      DWORD dwThreadId,
      WORD hMod16,
      WORD hTask16,
      PSZ pszModName,
      PSZ pszFileName,
      LPARAM lpUserDefined
   );

Example of Enumeration

   // Enumerate all 16-bit tasks on the system.
   #include <windows.h>
   #include <stdio.h>
   #include <vdmdbg.h>

   BOOL WINAPI ProcessEnumProc( DWORD, DWORD, LPARAM );
   BOOL WINAPI TaskEnumProcEx( DWORD, WORD, WORD, PSZ, PSZ, LPARAM );

   void main()
   {
      // Enumerate VDMs.
      VDMEnumProcessWOW(
         (PROCESSENUMPROC)ProcessEnumProc,
         (LPARAM)NULL
      );

   }

   BOOL WINAPI ProcessEnumProc( DWORD dwProcessId, DWORD dwAttrib,
      LPARAM t )
   {
      printf("\nProcess ID: %d\n", dwProcessId);

      // Use process ID of VDM to enumerate through its tasks.
      VDMEnumTaskWOWEx(
         dwProcessId,
         (TASKENUMPROCEX)TaskEnumProcEx,
         (LPARAM)NULL
      );

      // Keep enumerating.
      return FALSE;
   }

   BOOL WINAPI TaskEnumProcEx( DWORD dwThreadId, WORD hMod16, WORD hTask16,
      PSZ pszModName, PSZ pszFileName, LPARAM lParam )
   {
      //print task's information
      printf("Thread ID: %d\n", dwThreadId);
      printf("Module handle: %d\n", hMod16);
      printf("Task handle: %d\n", hTask16);
      printf("Module Name: %s\n", pszModName);
      printf("File Name: %s\n", pszFileName);

      // Keep enumerating.
      return FALSE;
   }

Creating 16-bit Tasks

VDMStartTaskInWOW() creates a task within a VDM. The declaration is as follows:

   BOOL WINAPI VDMStartTaskInWOW( DWORD dwProcessId, LPSTR lpCommandLine,
                                  WORD wShow );

Terminating 16-bit Tasks

To terminate a task within a VDM call VDMTerminateTaskWOW(). The declaration is as follows:

   BOOL WINAPI VDMTerminateTaskWOW( DWORD dwProcessId, WORD htask );

This method is a rough equivalent to TerminateProcess() in Win32. It should be avoided, if possible. It does not give the task a chance to cleanly exit, so data may be lost. Unlike Win32, the WowExec is not guaranteed to clean up after a terminated task. This can leave the VDM corrupt and unusable. To terminate the task cleanly, send a WM_CLOSE to its top-level window.

Note Regarding 16-bit DOS Applications

None of the VDMDBG functions work with 16-bit DOS applications. To enumerate DOS VDMs, you need to use another method. First, you could use VDMEnumProcessWOW() to make a list of all Win16 VDMs, and then enumerate all instances of NTVDM.exe using some other scheme (such as PSAPI). Any NTVDM.exe from the full enumeration that was not in the Win16 list is a DOS VDM. You can create and terminate 16-bit DOS applications with CreateProcess() and TerminateProcess().

REFERENCES

For additional information, please see the following article in the Microsoft Knowledge Base:

   ARTICLE-ID: Q175030
   TITLE     : HOWTO: Enumerate Applications in Win32

Keywords          : BseDebug BseProcThrd
Version           : WINNT:4.0
Platform          : winnt
Issue type        : kbhowto