Architectural Overview
When COM remote debugging is enabled, there are a total of six notifications that occur in the round-trip of one COM RPC call: three on the client side and three on the server side. The overall sequence of events is as follows.
Suppose the client has an interface pointer pFoo of type IFoo* which happens to be a proxy for another object in a remote server process.
interface IFoo : IUnknown {
HRESULT Func();
};
IFoo *pFoo;
When the client invokes pFoo->Func(), it executes code in the interface proxy. This code is responsible for marshaling the arguments into a buffer, calling the server, and unmarshaling the return values. To do so, it draws on the services of an IRpcChannelBuffer instance with which it was initialized by the COM Library.
To get the buffer, the interface proxy calls IRpcChannelBuffer::GetBuffer(), passing in (among other things) the requested size for the buffer. Before actually allocating the buffer, the GetBuffer() implementation (normally20.) checks to see if debugging is enabled per DllDebugObjectRPCHook(). If so, then the channel calls DebugORPCClientGetBufferSize() (see below for details) to inform the debugger that an COM RPC call is about to take place and to ask the debugger how many bytes of information it would like to transmit to the remote server debugger. The channel then, unbeknownst to the interface proxy, allocates a buffer with this many additional bytes in it.
The interface proxy marshals the incoming arguments in the usual way into the buffer that it received, then calls IRpcChannelBuffer::SendReceive(). Immediately on function entry, the channel again checks to see if debugging is enabled. If so, then it calls DebugORPCClientFillBuffer() passing in the pointer to (the debugger's part of) the marshaling buffer. The debugger will write some information into the buffer, but this need be of no concern to the channel implementation other than that it is to ferry the contents of the buffer to the server debugger. Once DebugORPCClientFillBuffer() returns, the channel implementation of SendReceive() proceeds as in the normal case.
We now switch context in our explanation here to the server-side RPC channel. Suppose that it has received an incoming call request and has done what it normally does just up to the point where it is about to call IRpcStubBuffer::Invoke(), which when will cause the arguments to be unmarshaled, and so forth. Just before calling Invoke(), if there was any debugger information (for example, it exists in the incoming request and is of non-zero size) in the incoming request or if debugging is presently already enabled per DllDebugObjectRPCHook() (irrespective of the presence or size of the debug information), then the channel is to call DebugORPCServerNotify().21. The act of calling this function may in fact start a new debugger if needed and attach it to this (the server) process; however, this need not be of concern to the channel implementation. Having made the request, the channel proceeds to call Invoke() as in the normal case.
The implementation of Invoke() will unmarshal the incoming arguments, then call the appropriate method on the server object. When the server object returns, Invoke() marshals the return values for transmission back to the client. As on the client side, the marshaling process begins by calling IRpcChannelBuffer::GetBuffer() to get a marshaling buffer. As on the client side, the server side channel GetBuffer() implementation when being debugged (per the present setting of DllDebugObjectRPCHook(), not per the presence of the incoming debug information) asks the debugger how many bytes it wishes to transmit back to the client debugger. The channel allocates the buffer accordingly and returns it to the Invoke() implementation who marshals the return values into it, then returns to its caller.
The caller of IRpcStubBuffer::Invoke() then checks to see if he is presently being debugged. If so, then he at this time calls DebugORPCServerFillBuffer(), passing in the pointer to the debug-buffer that was allocated in the (last, should there erroneously be more than one) call to GetBuffer() made inside Invoke(); should no such call exist, and thus there is no such buffer, NULL is passed.22. The bytes written into the buffer (if any) by the debugger are ferried to the client side.
We now switch our explanatory context back to the client side. Eventually the client channel either receives a reply from the server containing the marshaled return values (and possibly debug information), receives an error indication from the server RPC infrastructure, or decides to stop waiting. That is, eventually the client channel decides that it is about to return from IRpcChannel::SendReceive(). Immediately before doing so, it checks to see if it is either already presently being debugged or if in the reply it received any (non-zero sized) information from the server debugger. If so, then it calls DebugORPCClientNotify(), passing in the server-debugger's information, if it has any; doing so may start and attach the debugger if needed. The channel then returns from SendReceive().