Some functions that are essential to debugging are actually part of the process, thread, or exception-handling architectures.
The CreateProcess function enables a debugger to start a process and debug it. The dwCreationFlags parameter to CreateProcess can be used to indicate debugging operations:
Value | Meaning |
DEBUG_PROCESS | If this flag is set, the calling process is treated as a debugger, and the new process is a process being debugged. The kernel notifies the debugger of all debug events that occur in the process being debugged. If this flag is not set and the calling process is being debugged, then the new process becomes another process being debugged by the calling process' debugger. If this bit is not set and the calling process is not a process being debugged, no debugging-related actions occur. |
DEBUG_ONLY_THIS_PROCESS | If this flag is set, the calling process is treated as a debugger, and the new process is a process being debugged. If the new process creates additional processes, the kernel will not notify the debugger of any debug events that occur in the additional processes. |
A debugger can debug a process and all of the process' descendants by creating a process with the DEBUG_PROCESS flag. If the process being debugged creates processes without the DEBUG_PROCESS flag, the debugger can debug these processes, too.
A debugger can debug a process and none of its descendants by creating a process with the DEBUG_PROCESS and DEBUG_ONLY_THIS_PROCESS flags.
A debugger can debug a debugger by creating a process with the DEBUG_PROCESS flag. The new process (the debugger being debugged) must then create a process with the DEBUG_PROCESS flag.
The OpenProcess function enables a debugger to obtain a handle to an existing process. Typically, debuggers open a process with the PROCESS_VM_READ and PROCESS_VM_WRITE flags. This enables the debugger to read and write to the process' virtual memory by using the ReadProcessMemory and WriteProcessMemory functions.
The CreateThread function creates a new thread object for a process. Debuggers typically need to examine or change the contents of a thread's registers. To accomplish this, a debugger must obtain a handle to the thread object by using the DuplicateHandle function and specifying the appropriate access to the thread (THREAD_GET_CONTEXT and/or THREAD_SET_CONTEXT).
Any process with appropriate access to a thread can examine the thread's registers by using the GetThreadContext function and set the contents of the thread's registers by using the SetThreadContext function.
A process can also obtain THREAD_SUSPEND_RESUME access to a thread object. This enables a debugger to control a thread's execution with the SuspendThread and ResumeThread functions.
When an exception occurs in a process that is being debugged, the kernel notifies the debugger that an exception occurred. This is known as first chance notification. The kernel suspends all threads in the process being debugged.
If the debugger does not handle the exception, the kernel attempts to locate an appropriate exception handler. If the kernel cannot locate an appropriate exception handler, the kernel again notifies the debugger that the exception occurred. This is known as last chance notification. If the debugger does not handle the exception after the last chance notification, the kernel terminates the process being debugged.
For more information about exception handling, see Chapter 24, “Structured Exception Handling.”