To allow objects to reside in apartments that are distinct from the client’s apartment, COM allows interfaces to be exported from one apartment and imported into another. To make an object’s interfaces visible outside the object’s apartment is to export the interface. To make an external interface visible inside an apartment is to import an interface. When an interface is imported, the resultant interface pointer refers to a proxy that can be legally accessed by any thread in the importing apartment.4 It is the job of the proxy to pass control back to the object’s apartment, ensuring that all method calls execute in the correct apartment. This passing of control from one apartment to another is referred to as method remoting and is how all cross-thread, cross-process, and cross-host communications occur in COM.
By default, method remoting uses the COM Object RPC (ORPC) communications protocol. COM ORPC is a lightweight protocol that is layered over MS-RPC, a DCE derivative. MS-RPC is a protocol-independent communications mechanism that can be extended to support new transport protocols (via dynamically loadable transport DLLs) and new authentication packages (via dynamically loadable Security Support Provider DLLs). COM uses the most efficient transport protocol available based on the proximity and types of the importing and exporting apartments. When communicating off host, COM prefers the User Datagram Protocol (UDP), although most common network protocols are supported.5 When communicating locally, COM uses one of several transports, each of which is optimized for a particular apartment type.
COM allows interface pointers to be passed across apartment boundaries using a technique called marshaling. Marshaling an interface pointer simply transforms the interface pointer into a transmissible byte stream whose contents uniquely identify the object and its owning apartment. This byte stream is the marshaled state of the interface pointer and allows any apartment to import the interface pointer and make method calls on the object. Note that because COM deals exclusively with interface pointers, not the objects themselves, this marshaled state does not represent the state of the object but rather the serialized state of an apartment-independent reference to the object. These marshaled object references simply contain connection establishment information that is completely independent of the object’s state.
Normally, interface pointers are marshaled implicitly as part of the normal operation of COM. When an in-process activation request is made for a class with an incompatible threading model, COM implicitly marshals the interface from the object’s apartment and unmarshals a proxy in the client’s apartment. When an out-of-process or off-host activation request is made, COM also marshals the resultant pointer from the apartment of the object and unmarshals a proxy for the client. When method calls are performed on proxies, any interface pointers that are passed as method parameters will be marshaled in order to make the object references available in both the client’s and the object’s apartments. Occasionally, it is necessary to marshal interfaces explicitly from one apartment to another outside the context of an activation request or a method call. To support this, COM provides a low-level API function, CoMarshalInterface, to marshal interface pointers explicitly.
CoMarshalInterface takes an interface pointer on input and writes the serialized representation of the pointer to a caller-provided byte stream. This byte stream can then be passed to another apartment, where the CoUnmarshalInterface API function uses the byte stream to return an interface pointer that is semantically equivalent to the original object yet can be legally accessed in the apartment that executes the CoUnmarshalInterface call. When calling CoMarshalInterface, the caller must indicate how far away the importing apartment is expected to be. COM defines an enumeration for expressing this distance:
typedef enum tagMSHCTX {
MSHCTX_INPROC = 4, // in-process/same host
MSHCTX_LOCAL = 0, // out-of-process/same host
MSHCTX_NOSHAREDMEM = 1, // 16/32 bit/same host
MSHCTX_DIFFERENTMACHINE = 2, // off-host
} MSHCTX;It is legal to specify a greater distance than required, but it is more efficient to use the correct MSHCTX when possible. CoMarshalInterface also allows the caller to specify the marshaling semantics using the following marshal flags:
typedef enum tagMSHLFLAGS {
MSHLFLAGS_NORMAL, // marshal once, unmarshal once
MSHLFLAGS_TABLESTRONG, // marshal once, unmarshal many
MSHLFLAGS_TABLEWEAK, // marshal once, unmarshal many
MSHLFLAGS_NOPING = 4, // suppress dist. garbage collection
} MSHLFLAGS;Normal marshaling (sometimes called call marshaling) indicates that the marshaled object reference must be unmarshaled only once, and if additional proxies are needed, additional calls to CoMarshalInterface are required. Table marshaling indicates that the marshaled object reference may be unmarshaled zero or more times without requiring additional calls to CoMarshalInterface. The details of table marshaling are described later in this chapter.
To allow interface pointers to be marshaled to a variety of media, CoMarshalInterface serializes the interface pointer through a caller-supplied interface of type IStream. The IStream interface models an arbitrary I/O device and exposes Read and Write methods. CoMarshalInterface simply calls the Write method on the caller-provided IStream interface without concern for where the actual bytes will be stored. Callers can get an IStream wrapper to raw memory by calling the API function CreateStreamOnHGlobal:
HRESULT CreateStreamOnHGlobal(
[in] HGLOBAL hglobal, // pass null to autoalloc
[in] BOOL bFreeMemoryOnRelease,
[out] IStream **ppStm);Given the semantics of IStream, the following code fragment:
void UseRawMemoryToPrintString(void) {
void *pv = 0;
// alloc memory
pv = malloc(13);
if (pv != 0) {
// write a string to the underlying memory
memcpy(pv, "Hello, World", 13);
printf((const char*)pv);
// free all resources
free (pv);
}
}is equivalent to this code fragment that uses an IStream interface instead of memcpy:
void UseStreamToPrintString(void) {
IStream *pStm = 0;
// alloc memory and wrap behind an IStream interface
HRESULT hr = CreateStreamOnHGlobal(0, TRUE, &pStm);
if (SUCCEEDED(hr)) {
// write a string to the underlying memory
hr = pStm->Write("Hello, World", 13, 0);
assert(SUCCEEDED(hr));
// suck out the memory
HGLOBAL hglobal = 0;
hr = GetHGlobalFromStream(pStm, &hglobal);
assert(SUCCEEDED(hr));
printf((const char*)GlobalLock(hr));
// free all resources
GlobalUnlock(hglobal);
pStm->Release();
}
}The API function GetHGlobalFromStream allows the caller to extract a handle to the memory that was allocated in CreateStreamOnHGlobal. The use of HGLOBALs is historic and in no way implies shared memory.
Armed with an understanding of each of its parameter types, the CoMarshalInterface API function is fairly simple:
HRESULT CoMarshalInterface(
[in] IStream *pStm, // where to write marshaled state
[in] REFIID riid, // type of ptr being marshaled
[in,iid_is(riid)] IUnknown *pItf,// pointer being marshaled
[in] DWORD dwDestCtx, // MSHCTX for destination apt.
[in] void *pvDestCtx, // reserved, must be zero
[in] DWORD dwMshlFlags // normal vs. table marshal
);The following code marshals an interface pointer into a block of memory suitable for transmitting to any apartment on the network:
HRESULT WritePtr(IRacer *pRacer, HGLOBAL& rhglobal) {
IStream *pStm = 0; rhglobal = 0;
// alloc and wrap block of memory
HRESULT hr = CreateStreamOnHGlobal(0, FALSE, &pStm);
if (SUCCEEDED(hr)) {
// write marshaled object reference to memory
hr = CoMarshalInterface(pStm, IID_IRacer, pRacer,
MSHCTX_DIFFERENTMACHINE, 0,
MSHLFLAGS_NORMAL);
// extract handle to underlying memory
if (SUCCEEDED(hr))
hr = GetHGlobalFromStream(pStm, &rhglobal);
pStm->Release();
}
return hr;
}Figure 5.1 illustrates the relationship between the interface pointer and the underlying memory that contains the marshaled object reference. After calling CoMarshalInterface, the object’s apartment is now ready to receive a connection request from a foreign apartment. Because the MSHCTX_DIFFERENTMACHINE flag was used, the importing apartment can potentially be on a different host machine.
To decode the marshaled object reference created in the previous code fragment into a valid interface pointer, the importing apartment will need to call the CoUnmarshalInterface API function:
HRESULT CoUnmarshalInterface(
[in] IStream *pStm, // where to read marshaled state
[in] REFIID riid, // type of ptr being unmarshaled
Figure 5.1 Marshaled Object Reference
[out,iid_is(riid)] void **ppv// where to put unmarshaled ptr
);CoUnmarshalInterface simply reads a serialized object reference and returns a pointer to the original object that can be legally accessed in the calling thread’s apartment. If the importing apartment is different from the apartment that originally exported the interface, then the resultant pointer will be a pointer to a proxy. If for some reason the CoUnmarshalInterface call is issued from the original apartment where the object resides, then a pointer to the actual object will be returned and no proxy will be created. The following code translates a marshaled object reference into a valid interface pointer:
HRESULT ReadPtr(HGLOBAL hglobal, IRacer * &rpRacer) {
IStream *pStm = 0; rpRacer = 0;
// wrap block of existing memory passed on input
HRESULT hr = CreateStreamOnHGlobal(hglobal, FALSE, &pStm);
if (SUCCEEDED(hr)) {
// get a pointer to the object that is legal in this apt.
hr = CoUnmarshalInterface(pStm,
IID_IRacer, (void**)&rpRacer,
pStm->Release();
}
return hr;
}The resultant proxy will implement each of the interfaces that the object exports by forwarding the method requests to the object’s apartment.
Prior to the Windows NT 4.0 release of COM, the format of a marshaled object reference was undocumented. To allow third parties to build COM-aware network products, the format was publicly documented in 1996 and submitted as an Internet draft standard. Figure 5.2 shows the format of a marshaled object reference. The header of the marshaled object reference begins with a distinguished signature (‘MEOW’)6 and a flags field indicates the marshaling technique used (e.g., standard marshaling, custom marshaling) as well as the IID of the interface contained in the reference. Assuming standard marshaling is used, a subheader of the object reference indicates how many external references this marshaled reference represents. This external reference count is part of COM’s distributed garbage collection protocol and does not directly correspond to the AddRef/Release reference count that the object may implement. The interesting elements of the object reference are the OXID/OID/IPID tuple that uniquely identify an interface pointer. Each apartment in the network is assigned a unique Object Exporter Identifier (OXID) at apartment creation time. This OXID is used to find network/IPC addressing information when a proxy first connects to the object. The Object Identifier (OID) uniquely identifies a COM identity in the network and is used by CoUnmarshalInterface to maintain the COM identity laws for proxies. The Interface Pointer Identifier (IPID) uniquely identifies an interface pointer in an apartment and is placed in the header of each subsequent method request. The IPID is used to efficiently dispatch ORPC requests to the correct interface pointer in the object’s apartment.
Figure 5.2 Standard Marshal Object Reference
Although OXIDs are interesting as logical identifiers, by themselves they are useless, as the proxy will need to use some IPC mechanism or network protocol to contact the object’s apartment. To translate OXIDs into fully qualified network or IPC addresses, each COM-enabled host machine provides an OXID Resolver (OR) service. Under Windows NT 4.0, the OR is implemented as part of the RPCSS service. When an apartment is first initialized, COM allocates an OXID and registers it with the local OR. This means that each OR knows of all running apartments on the local system. The OR also keeps track of the local IPC port for each apartment. When CoUnmarshalInterface needs to connect a new proxy to the object’s apartment, the local OR is consulted to resolve the OXID into a valid IPC or network address. If the unmarshal occurs on the same machine as the object’s apartment, then the OR simply looks up the OXID in its local OXID table and returns a local IPC address. If the unmarshal occurs on a host machine other than that of the object, the local OR first checks to see if it has encountered the OXID in the recent past by consulting a cache of recently resolved remote OXIDs. If the OXID has not been encountered recently, it forwards the request to the OR on the host machine of the object using RPC. Note that the marshaled object reference contains the object’s host address in a format suitable for a variety of network protocols, so the unmarshal-side OR knows where to forward the request.
To allow distributed OXID resolution, the OR service on each host machine listens for remote OXID resolution requests on a well-known endpoint (port 135 for TCP and UDP) for each supported network protocol. When an OXID resolution request is received from the network, the local OXID table is consulted. The resolution request will indicate which network protocols the client machine supports. If the requested apartment’s process has not yet started to use one of the requested protocols, the OR will contact the COM library in the object’s apartment using local IPC and cause the object’s process to start using the requested protocol. Once this occurs, the OR will note the new network address of the apartment in the local OXID table. After the new network address is recorded, the address is returned to the unmarshal-side OR, where it is cached to avoid additional network requests for commonly used apartments. Although it may seem odd not just to write out the object’s fully qualified network addresses in the marshaled object reference, the level of indirection afforded by protocol-independent apartment IDs (OXIDs) allows a COM-based process to postpone using network protocols until they are needed. This is especially important because COM can operate over a variety of different protocols (not just TCP), and requiring every process to listen for requests using all supported protocols would be extremely inefficient. In fact, if a COM-based process never exports pointers to off-host clients, it will never consume any network resources whatsoever.
4 If the importing apartment is the apartment that the object belongs to, no proxy will be used and the imported pointer will point directly to the object.
5 UDP is preferred over TCP due to the excessive overhead of TCP connection establishment. COM, like DCE RPC, piggybacks its security and protocol synchronization information in the packet headers used to transmit the first RPC request. Because COM-based systems tend to set up and tear down many transient connections, UDP is the superior choice. When using datagram transports such as UDP, the RPC runtime library reimplements the error correction and flow/congestion control algorithms of TCP.
6 A Microsoft Program Manager who shall remain anonymous claims that MEOW stands for Microsoft Extended Object Wire representation. The author, while somewhat gullible, is skeptical of this story but is willing to give the aforementioned source the benefit of the doubt.