Master Multithreading—Carefully

by Ted Pattison

Reprinted with permission from Visual Basic Programmer's Journal, 2/98, Volume 8, Issue 2, Copyright 1998, Fawcette Technical Publications, Palo Alto, CA, USA. To subscribe, call 1-800-848-5523, 650-833-7100, visit www.vbpj.com, or visit The Development Exchange

Multithreading supports background processing and does wonders for your app's responsiveness.

Are you ready for multithreading? If you do it wrong—and it's easy to do wrong—your code will make Yugos look reliable. But do it right and your users will love the way your software responds. With multithreading, you can run lengthy tasks, such as print spooling jobs, in the background, create responsive multiclient servers, and more (see Table 1). Not to mention the bragging rights you'll get at developer conferences.

You can use this technique in VB5 to create multithreaded ActiveX EXEs for multithreaded servers and multithreaded clients. This adds a level of complexity to your code that's only safe if you master the conceptual issues involved. You're diving off the high board with this one.

Start by pondering the Win32 programming paradigm's foundation, built on two high-level abstractions: processes and threads. A process is a running instance of an application that owns an address space of virtual memory and some other resources. A thread is a schedulable entity owned by a single process. The system recognizes every thread in every application, allocating processing cycles to each thread. Each process has at least one thread, called the primary thread.

For example, with the Win32 function CreateThread(), a C programmer—or a VB5 programmer using the AddressOf operator—can create additional threads to carry out background tasks. Multiple threads let two discrete tasks run concurrently without regard to each other's progress. You can put that print spooling job in the background on a low priority thread, using the primary thread to respond to user actions in the app's user interface (UI).

Feel free to try out what I just said if you know the syntax for calling CreateThread() with AddressOf. But your program will crash in a flash—unless you've also got a handle on the Component Object Model (COM). COM layers an abstraction called apartments on top of the operating system's (OS) underlying threading model. And apartments are the key to reliable multithreading in VB.

Apartments provide interoperability between various code libraries. Suppose one programmer writes a thread-savvy library and another one writes a thread-stupid library. If you use these two libraries in a single application, the multithreading library might call a certain function exported from the thread-stupid library using two different threads at the same time. This is a phenomenon called concurrency, and it makes operations that access and change data vulnerable to data corruption. Thread-aware components are designed to deal with concurrency, which provokes unpredictable and often terminal behavior in thread-stupid components. The whole idea behind COM apartments is to enable code libraries to coexist in a process regardless of their level of thread awareness.

Table 1: Beware of Thread Poisoning. Here are a few common situations where extra threads can help along with the price you'll pay. Weigh both sides of the equation before you add threads to an app.

Every COM object (and therefore every VB object) is created in the context of a COM apartment. A thread must find and enter an apartment before creating and interacting with COM objects. Under strict rules defined by COM, a thread in one apartment is not allowed to touch an object that lives in another apartment.

If a method call is made from one apartment to another, COM uses a pair of objects called the proxy and stub (see Figure 1). These objects marshal the method arguments between apartments and re-create the call stack in the object's apartment. Similarly, the proxy and stub must marshal the return value back to the caller.

Fortunately, the system creates the proxy and stub objects transparently behind the scenes, using information in the object's type library. The proxy/stub pair is created whenever an interface reference passes from one apartment to another.

So far, so good. But when you use the Win32 CreateThread() function to create a thread, it doesn't execute in the context of a COM apartment. So any method you invoke against any object will most likely crash the system.

You could make a thread COM-aware by calling a function such as CoInitializeEx(), forcing the thread to enter an apartment. Then you need to marshal interface references (or interface pointers) from one apartment to another. If you pass these interface references through a COM interface, the proxy and stub are created automatically. Pass them yourself and you're looking at a lot of code to write. You need to make complex system calls to create a proxy in the caller's apartment and a stub in the object's apartment. Daniel Appleman wrote an excellent online article detailing how to write this type of code (www.desaware.com/desaware/thartrq.htm). But don't try this unless you know a lot about COM and the OLE32 DLL. However, if you can code at this level, you'd use C++, which lends itself to accessing such a low-level interface. And if you saw someone trying to use VB for this, you might say, "Ain't it just like a VB programmer to bring a knife to a gun fight."

Instead, I'll concentrate on what you can do using reasonable VB techniques, with some straightforward examples and a few more complex ones. These will demonstrate ways to implement multithreading you can use in production with confidence. Now you can dig in by looking at the different apartment types defined by COM.

Currently COM defines two types of apartments. A multithreaded apartment (MTA, or sometimes called free threading) lets many objects and threads exist within a single execution context. MTAs provide what is often called a free threading model, for maximum concurrency. Or you can use a single-threaded apartment (STA, or sometimes called an apartment). STAs also prevent concurrency and provide better integration with user interface code. STAs eliminate the need for programmer-assisted synchronization, but as you'll see, they also provide a suboptimal invocation architecture.

Today only C++ and Java code can create free-threaded components to run in MTAs. A COM app is limited to one MTA (see Figure 2). COM is layered on top of an interprocess mechanism called remote procedure call (RPC). The RPC layer maintains a thread pool and allocates one of these threads to each incoming method call. Each method call is serviced immediately when it arrives, so you must anticipate problems stemming from concurrency.

Figure 1: Marshalling Between Apartments. Every VB object lives in a COM apartment limited to a single thread. Objects that live in separate apartments can't communicate directly with each other. COM uses a pair of objects, the proxy and stub, to invoke method calls across apartment boundaries.

C++ lets you manage concurrency with Win32 locking primitives such as mutexes, semaphores, and critical sections. Although this style of programming yields potentially more responsive components, it also becomes increasingly complex.

VB keeps things simple by only supporting STAs. However, it does support multiple STAs (which implies multiple threads) in one ActiveX EXE. To keep you from having to deal with multithreaded access to global variables, VB creates a separate copy of each global variable for each apartment. This implies that VB objects in different apartments can't share data by using public variables in BAS modules. Are you getting the feeling that we're not in Kansas anymore?

Even though you're limited to STAs in your VB components, you can still boost the performance and responsiveness of an out-of-process server—if you understand when VB creates additional apartments and how multiple STAs work together in a single process (see Figure 3). Remember, STAs prevent concurrent access by limiting an apartment to one thread. Because no external thread can touch an object in an STA, you're guaranteed that an object will never run two method calls concurrently.

You do pay a price for this guarantee. STAs employ a standard Windows message queue to serialize all method requests, which implies that every STA needs an invisible window behind the scenes. The RPC layer uses a standard Win32 PostMessage() call to add incoming calls to the end of the queue. Message calls are serviced on a first-come, first-serve basis. Whenever a method call is being serviced, all other pending calls are blocked and must wait their turn.

This invocation architecture limits you in two ways. First, it's a roundabout way to invoke a method call. Posting and reading messages on the queue takes up valuable processing cycles. Second, you'd be better off if locking happened at the object level, not the apartment level. The queue, as a single point of entry to all objects in an STA, imposes concurrency challenges. An STA prevents multiple calls from executing on a single object concurrently, but it also prevents two objects in the same STA from running method calls concurrently. To get around this, you must create them each in separate STAs.

A future version of COM will add rental-threaded apartments (RTAs). Like STAs, RTAs will eliminate your need to assist synchronization. Unlike STAs, they won't need a Windows message queue in the invocation architecture. Instead, they'll provide a higher performance method invocation scheme that doesn't make you statically allocate a thread for each apartment. RTAs will still use apartment-level locking, but you'll have more apartments per process available, allowing higher levels of concurrency without sacrificing STA's protection from concurrent access. And RTAs will behave like STAs, simplifying your coding task. VB components will simply run faster and more responsively. Let me warn you that I don't know when RTAs will show up, what they'll show up in (perhaps NT5 or COM+), or what they'll be named.

Figure 2: Life Inside an MTA. Life inside a multithreaded apartment is both dangerous and exciting, like life in New York City. Many things happen at once, and an object that isn't prepared for this environment could end up with a knife in its back.

Figure 3: Life Inside an STA. Life in a single-threaded apartment is calm and safe, like life in a rustic country town. Objects are accostomed to the slow pace, and they walk the streets secure in the knowledge they are safe.

As I hope I've demonstrated, you shouldn't create a thread on your own. But you can create a situation where the VB run time and COM work together to create a new STA and a new thread for you. Objects can't change apartments, so you can only play these tricks when a client requests a new object from an ActiveX EXE server. This server also needs the appropriate project setting for its threading model property. Standard off-the-shelf EXEs and ActiveX DLLs will never create new STAs.

Note: an ActiveX DLL or ActiveX control project can also have its threading model set to apartment. This makes your DLLs and OCXs work more efficiently in a multithreaded client such as IE4 (see the sidebar Set Your Model to 'Apartment').

If you want to make your DLLs and OCXs work more efficiently in a multithreaded client such as IE4, I recommend setting your threading model to apartment. Here's why.

A DLL is passive. Client apps load it, and it usually doesn't create new threads. However, each potential class within a DLL or OCX has an associated CLSID marked in the Windows registry with a ThreadingModel attribute: "free," "apartment," "both," or absent. "Free" means newly created objects can only be loaded into an app's multi-threaded apartment (MTA). "Apartment" means a new object will be loaded into the caller's single-threaded apartment (STA). Either way, if the caller isn't the correct apartment type, COM automatically creates (or finds) a compatible apartment type, creates the object in it, and marshals the new object reference back to the caller. "Both" means a new object can be loaded into the caller's apartment, whether it's an MTA or an STA.

Components lacking a ThreadingModel attribute are always loaded into the first STA created within a process. This STA is the main apartment, and classes configured to run there are often called single-threaded or main-threaded classes. When a creation request is made for a main-threaded class from an STA other than the main STA, COM introduces a proxy and stub connection between caller and object. You should use the apartment attribute for the best performance in apps with multiple STAs, such as IE4. Remember that multiple STAs cannot share global data in BAS modules, so the extra performance you get with apartment-threaded DLLs and OCXs does involve a trade-off.—T.P.

Choose one of two settings for an ActiveX EXE project's threading model property, using VB5 Service Pack 2 (mandatory for safe threading). Pick either thread-per-object or thread pool, and indicate the size you like. If you choose thread-per-object, VB will create a new STA (and a new thread) whenever client apps create new objects. The first STA created upon application initialization is called the main apartment. Use a freshly-created STA dedicated to that object to load new objects.

Client apps create new objects—each in a new STA—with either the New operator or VB's CreateObject() function. That new STA helps. When clients invoke methods, the calls of other clients won't block their calls. My sample code for this article provides two examples of multithreaded server projects that demonstrate this technique. The two projects both allow for concurrency, enabling multiple client apps to execute long-running methods simultaneously. Both projects also demonstrate the use of asynchronous methods. The first one uses events to notify client apps of task status and completion, and the second uses a callback interface. To download these example apps, go to the free, Registered Level of The Development Exchange (see the Code Online box at the end of the article for more details).

If you create an ActiveX EXE with the thread pool option selected, VB will create a new STA for each new object request until it reaches the thread pool capacity. If you have a thread pool of size 4, it creates the first three objects in apartments 2, 3 and 4 respectively. It then creates additional objects in the context of existing apartments in this sequence: 3, 2, main; 4, 3, 2, main; and so on. VB uses a reasonable, though somewhat arbitrary, algorithm to match up apartments and new objects. I recommend against taking advantage of your knowledge of this sequence; it might change in VB6 or VB7.

To enhance scalability, a thread pool limits the number of threads, because juggling too many threads per process imposes serious switching time overhead. Every time a thread is switched in or out of the processor, it wastes processing cycles that should be used for doing actual work. VB applications that use thread pools limit the switching overhead; however, they are also more vulnerable to concurrency problems than thread-per-object ActiveX EXEs. For instance, if two clients create objects that happen to share an apartment, one client method invocation will block the other. And if method calls take more than a second or two to complete, you'll be faced with an unacceptable concurrency constraint. Rather than use VB's thread pooling when I need threading scalability, I prefer to deploy my objects in Microsoft Transaction Server (MTS), which gives you more control over thread pooling. For more information, see Jonathan Zuck's article, "In the Trenches with N-Tier Development" [VBPJ November 1997].

If a client invokes a method call on an object in the same apartment, the call can be completed on a single thread. The caller and the object can share the same call stack. However, if a client and an object live in separate apartments, you need a proxy/stub pair to marshal the call between apartments and re-create the call stack in the object's STA. VB and COM always marshal and unmarshal interface references for you between apartments. This transparently forces the creation of the proxy and stub. But watch out: inter-apartment calls typically take from 500 to 1000 times as long as intra-apartment calls, due to the proxy/stub architecture. In fact, inter-apartment calls take almost as long as inter-process calls on the same machine. Avoid designs requiring lots of calls between apartments.

Unfortunately, multithreaded ActiveX servers also provide no easy way to share data between objects in separate apartments. VB's single-threaded apartments use a separate instance of global variables for each apartment, employing something called thread local storage (TLS). As with COM objects, VB was built to obviate programmer-assisted synchronization. If multiple threads could access the same global data, VB programmers would be responsible for synchronizing access to them.

Forget synchronization. VB handles it behind the scenes. But as a consequence, you must either design multithreaded apps that don't require shared data, or you must devise a fairly contrived scheme, where multiple objects in separate apartments each hold a reference to a singleton object. For example, you could use a single-threaded ActiveX EXE that acts as an object broker for your multithreaded ActiveX EXE. An object broker This type of design can get tricky quickly. In R. Mark Tucker's accompanying article, "Set Your Apps Free with Threads," you'll learn about a technique to share data between multiple apartments in an out-of-process multithreaded server.

You can also create multiple threads in a typical GUI app. Remember the scenario where VB creates a new STA when a client requests a new object on an ActiveX EXE with threading model settings? If you build this app as an ActiveX EXE (not a Standard EXE), you can multithread with it. Simply create new objects using the CreateObject() function, not the New operator.

When you use the New operator, VB looks for that class inside the current project. If it doesn't exist, VB calls the service control manager to pass the CLSID to request object creation from an external component. A call to CreateObject() always produces a call to the service control manager. When you pass the ProgID of an internal class to CreateObject(), your app calls the service control manager, which responds by calling back into your running app to request the creation of a new object with the typical COM activation sequence. An ActiveX EXE assumes this creation request is coming from an external client, so it creates a new STA. This way, your new object is created in a new STA. What a hack.

It works like this: when you call CreateObject(), a new STA is created in your process. The new object is created inside it. CreateObject() returns an interface reference to the object; this act causes the proxy and stub to be created to allow for interapartment method calls. Now you have an object in a separate apartment running on a different thread. If you're with me so far, I'll complicate things further by going to the gotchas.

To begin with, you can't test mulithreaded apps in the VB IDE. You can only see how creating those additional STAs works in a compiled EXE. To debug your app, use your old friend, the MsgBox() statement.

When you create an ActiveX EXE instead of a Standard EXE, you must use Sub Main() instead of a startup form. You must create and show your main form in this routine, even though Sub Main()executes each time a new apartment is created. Each new apartment also gets its own instance of the App object. (I wish they named it the Apt object). You must use a programming technique to load and show the main form only during your app's first execution of Sub Main(). The sample code for this article shows how to do this using a hidden form and the Win32 FindWindow() function.

You can't create additional apartments until the main apartment has been fully initialized. You have to wait until Sub Main() completes, and it won't complete until the form you're loading completes its Load() method. You can't create objects on separate threads during either of these methods. I find this a pain because I'm used to creating objects in Sub Main() or in a Load event of my main form.

The need to achieve rocket science ignites when you want two separate threads to run without blocking each other. Currently, COM only lets you run method calls synchronously. When an object in one apartment invokes a method on an object in another, its thread blocks until the method completes. To implement an asynchronous call, use Win32's SetTimer() function. My sample code provides several examples of this function. Fortunately, NT5 will support declarative asynchronous methods, eliminating the need for this technique.

Finally, after you execute an asynchronous call in a secondary apartment, you must use either events or callback interfaces to let the object notify your main form of task status, completion, and failure. You can either use events or callback interfaces to accomplish this. My sample code also provides several implementations of common techniques. R. Mark Tucker's article demonstrates a trickier implementation, with many server-side threads providing callbacks to a single client app.

At this point, are you ready to tackle multithreading? It certainly helps sometimes—but never with overall performance. In fact, it slows things up. You need lots of extra cycles to switch threads in and out of the processor. And multiple threads often require marshalling calls within a single process, versus more efficient intra-apartment calls. Only consider threads when you want to improve application responsiveness. But you always want to do that, don't you? ?

Download the code for this article here