May 1999

Using Visual Basic to Integrate MSMQ into Your Distributed Applications

Ted Pattison

MSMQ and message passing can add dimension to your application that can't be achieved through COM and RPC alone. MSMQ allows you to run transactions from across the network more reliably than COM method calls. This can be critical in an online transcation processing system.

This article assumes you're familiar with COM, MTS, Visual Basic, MSMQ, ADO

Code for this article: MSMQ_samples.exe (68KB) Ted Pattison is an instructor and researcher at DevelopMentor (http://www.develop.com/dm/default.asp), where he manages the Visual Basic curriculum. Ted is the author of Programming Distributed Applications with COM and Visual Basic 6.0 (Microsoft Press, 1998).

You're designing a distributed application based on Windows NT® or Windows® 2000. You're going to use Visual Basic® to create middle-tier business objects that will run inside Microsoft® Transaction Server (MTS) or the COM+ runtime. Now you're considering integrating Microsoft Message Queue Server (MSMQ)—part of Windows NT Server—into the architecture of your application. You think about the limitations of COM-based method calls; in COM, remote method calls are based on RPC, which is a synchronous, connection-oriented communications mechanism. In quite a few situations, MSMQ and message passing can add dimension to your application that can't be achieved through COM and RPC alone.     MSMQ makes it possible to run client applications from across a WAN or a dial-up connection in a much more reliable fashion than a system that relies on RPC. MSMQ also makes it possible to handle peak workloads in a more efficient and controlled manner. MSMQ makes it much easier to move data back and forth from a client application running on a laptop, which is extremely handy if you have disconnected users such as a remote sales force. Finally, MSMQ allows you to run transactions from across the network more reliably than COM method calls. This can be critical in an online transaction processing (OLTP) system.     Let's move beyond the basics of MSMQ programming to discuss some important design and implementation techniques. If you need to get up to speed on MSMQ basics, there are two excellent articles in the July 1998 issue of MSJ. "Microsoft Message Queue is a Fast, Efficient Choice for Your Distributed Application," by David Chappell, explains the motivation behind MSMQ and compares message passing to communications based on RPC. This article also covers the MSMQ programming model, discussing the most important properties of messages and queues. "Use MSMQ and MTS to Simplify the Building of Transactional Applications," by Mark Bukovec and Dick Dievendorff, talks about integrating MSMQ into an MTS application. It shows how to go about creating an MSMQ client and a server-side listener application. I'm going to build on the information presented in these two articles and concentrate on advanced MSMQ programming techniques using Visual Basic.     I'll start with a quick refresher on sending and receiving messages. Next, I'll describe the various ways you can move a complex set of data around in the body of an MSMQ message. You'll see techniques using property bags, persistent classes, and ADO recordsets. I'll finish off by examining some critical aspects of transactional programming with MSMQ. As you'll see, you must pass messages either in an internal or external transactional context to guarantee exactly-one delivery. This is often a requirement when using MSMQ messages to run MTS transactions. MSMQ can help eliminate many of the strange failure modes that are inherent in a COM-based connection. MSMQ 101     Let's start out by examining the code required to send a message that represents a sales order. MSMQ makes this task remarkably easy. You start by using an MSMQQueueInfo object to open an MSMQQueue object for send access. Next, create a new MSMQMessage object and prepare it by setting a few properties. You can then invoke the MSMQMessage object's Send method and MSMQ will route your message to its destination queue. Here's a simple example:

Dim QI As MSMQQueueInfo, RequestQueue As MSMQQueue Set QI = New MSMQQueueInfo QI.PathName = "MyServerComputer\MyQueue" Set RequestQueue = QI.Open(MQ_SEND_ACCESS, _ MQ_DENY_NONE) Dim msg As MSMQMessage Set msg = New MSMQMessage msg.Label = "Sales order #1001" msg.Body = "Customer=Bob;Product=PetRock;Quantity=120" msg.Send RequestQueue q.Close

This code creates a sender application. The sender is typically a client application that wants to run a transaction or retrieve a requested set of data. To complement a client-side sender application, you typically create a server-side listener application. A listener application monitors a queue for incoming messages. It's the responsibility of the listener application to receive and process messages when they arrive at the queue.     To receive a message in a listener application, you first open an MSMQQueue object with receive access, and then you invoke the Receive method to read and remove the first message in the queue:

Dim QI As MSMQQueueInfo, RequestQueue As MSMQQueue Set QI = New MSMQQueueInfo QI.PathName = "MyServerComputer\MyQueue" Set RequestQueue = QI.Open(MQ_RECEIVE_ACCESS, _ MQ_DENY_NONE) Dim msg As MSMQMessage ' Attempt to receive first message in queue. Set msg = RequestQueue.Receive(ReceiveTimeout:=1000) If Not (msg Is Nothing) Then ' You have removed the first message from the queue. ' now process the message Else ' you timed out on an empty queue End If q.close

By default, a call to Receive has no timeout value and will block indefinitely if no message is in the queue. When a message arrives, the call to Receive returns with an MSMQMessage object. If a message never arrives, the call never returns.     If you don't want the thread that calls Receive to block indefinitely, you can specify a timeout interval. You can use the ReceiveTimeout parameter to specify the number of milliseconds that you want to wait on an empty queue. If you call Receive on an empty queue and the timeout interval expires before a message arrives, the call to Receive returns with a null reference instead of an MSMQMessage object. The code in the last example shows how to set a timeout value of 1000 milliseconds. It also shows how to determine whether a message arrived before the timeout expired. If you don't want to wait at all, you can use a ReceiveTimeout value of 0. A ReceiveTimeout value of -1 indicates that you want to wait indefinitely. (-1 is also the default if you don't pass a timeout value.)     You can call Receive repeatedly inside a Do loop to synchronously remove every message from a queue. Here's how to receive all the messages from a queue and fill a listbox with message captions:

Dim QI As MSMQQueueInfo, RequestQueue As MSMQQueue Set QI = New MSMQQueueInfo QI.PathName = "MyServerComputer\MyQueue" Set RequestQueue = QI.Open(MQ_RECEIVE_ACCESS, _ MQ_DENY_RECEIVE_SHARE) Dim msg As MSMQMessage Set msg = RequestQueue.Receive(ReceiveTimeout:=0) Do Until msg Is Nothing lstReceive.AddItem msg.Label Set msg = RequestQueue.Receive(ReceiveTimeout:=0) Loop q.Close

As you can see, you can set the share mode for the opened queue to MQ_DENY_RECEIVE_SHARE so that your listener application won't have to contend with other listener apps when removing messages from the queue. The example also uses a timeout value of 0 to reach the end of the queue and move on to other business as quickly as possible. Receiving Messages with MSMQ Events     Earlier, I demonstrated a technique for synchronously receiving messages from a queue. This is an easy way to read and remove all the messages that are currently in a queue. It also lets you process future messages as they arrive at the destination queue. While this style of coding allows you to process messages as they arrive, it also holds the calling thread hostage. If you have a single-threaded application, the application can't do anything else.     You can use MSMQ events as an alternative to this synchronous style of message processing. MSMQ events let your application respond to asynchronous notifications that are raised by MSMQ as messages arrive at a queue. You can therefore respond to a new message without having to dedicate a thread to block on a call to Receive.     Let's look at how MSMQ events work. The MSMQ event mechanism is based on the MSMQEvent component. To use events, you must first create an MSMQEvent object and set up an event sink. Next, you must associate the MSMQEvent object with an MSMQQueue object that has been opened with receive access. You create the association between the two objects by invoking the EnableNotification method on the MSMQQueue object and passing a reference to the MSMQEvent object. After you call EnableNotification, MSMQ notifies your application when a message has arrived by raising an Arrived event.     To create an event sink, you must use the Visual Basic WithEvents keyword and declare the source object's reference variable in the declaration section of a form or class module. The following code shows how to set up an event sink for a new MSMQEvent object in a form module of a standard EXE project:

Private RequestQueue As MSMQQueue Private WithEvents RequestEvent As MSMQEvent Private Sub Form_Load() Dim QI As MSMQQueueInfo Set QI = New MSMQQueueInfo QI.PathName = "MyServerComputer\MyQueue" Set RequestQueue = QI.Open(MQ_RECEIVE_ACCESS, _ MQ_DENY_NONE) Set RequestEvent = New MSMQEvent RequestQueue.EnableNotification RequestEvent End Sub

Once you set up the MSMQEvent object's event sink and call EnableNotification, you will be notified with an Arrived event as soon as MSMQ finds a message in the queue. Here, an implementation of the Arrived event processes messages as they arrive in the queue:

Sub RequestEvent_Arrived(ByVal Queue As Object, _ ByVal Cursor As Long) Dim RequestQueue As MSMQQueue ' cast to MSMQQueue to avoid late binding Set RequestQueue = Queue Dim msg As MSMQMessage Set msg = RequestQueue.Receive(ReceiveTimeOut:=0) If Not (msg Is Nothing) Then ' process message when it arrives End If RequestQueue.EnableNotification RequestEvent End Sub

This example calls EnableNotification every time an Arrived event is raised. This is required because a call to EnableNotification only sets up a notification for the next message. If you want to receive ongoing notifications, you must keep calling EnableNotification in the Arrived event. Note that the previous code receives every message that was stored in the queue when the MSMQEvent object was set up. In other words, MSMQ raises events for existing messages as well as future ones.     I am very fond of Visual Basic, but I must point out two limitations when using it to create a server-side listener application. First, in a large system it's best to deploy the listener application as a Windows NT service. This will result in higher availability. Second, if your listener application has to handle a high volume of incoming request messages, the best way to increase system throughput is to increase the number of worker threads that are receiving messages from the queue and processing requests.     While some hardcore programmers have found a way to turn an application written in Visual Basic into a Windows NT service and to spawn additional COM-aware threads, these are certainly off-road techniques. Neither of these techniques are supported by Visual Basic and, therefore, result in code that can be pretty difficult to maintain. This means that you should consider using C++ or the Java language when you want your listener application to exhibit maximum availability and throughput. If you can get by with a server-side standard EXE project running on a single thread, then Visual Basic provides a much cheaper approach to creating a middle-tier listener application. Working with a Response Queue     When a client application sends a message to a request queue, it is often desirable to send a response message from the server-side listener application back to the sender. A response message can contain a notification to inform the sender that the transaction either ran successfully or failed. A response message can also return a requested set of data to the sender.     When you prepare a message in the sender application, you can add information into the message header that tells the listener application how to locate a response queue on the sender's computer. Each MSMQMessage object has a ResponseQueueInfo property. You must set up this property using an MSMQQueueInfo object. The receiving application can use ResponseQueueInfo to open the sender's response queue and send a receipt message. Figure 1 shows how the listener application can receive a request message, open the sender's response queue, and send a response message.     When the listener application processes a request message, it should assign the ID of the incoming request message to the CorrelationId of the outgoing response message. This technique is shown in Figure 1. When the client application receives a response message, it can compare the CorrelationId of the response message with the ID from each request message. This allows the sender to correlate messages.     You should also consider using private queues instead of public queues when setting up your response queues. This is especially true in a distributed application that has hundreds or thousands of clients. Unlike public queues, private queues are not published in the Message Queue Information Service (MQIS) database. Using public queues unnecessarily wastes processing cycles, network bandwidth, disk space, and administrative overhead. Packing the Message Body     In many cases, a message represents a request to run a transaction or a request for data. The sender application must pack up a set of request-specific parameters and write them into the message body. During the design phase, you should consider your options for packing parameterized data into the body. On the receiving side, you must also be able to unpack these parameters before you start processing the request. Up to this point, I've only shown you how to pass a simple Visual Basic for Applications string in a message body. Now let's look at some techniques for passing more complex data structures.     A message body is a variant that is stored and transmitted as a byte array. You can read and write the usual Visual Basic for Applications data types to the body, such as a Boolean, Byte, Integer, Long, Single, Double, Currency, Date, and String. MSMQ tracks the type you use in the message header. This makes it quite easy to store a single value in a message body, but it doesn't solve the problem of packing several pieces of data at once. To pack several pieces of data into a message, you must understand how to use the byte array behind the message body.     Using an array behind the message body is tricky because it must be an array of bytes. If you assign another type of array to the message body, MSMQ automatically converts it to a byte array. Unfortunately, once your data has been converted into a byte array, there's no easy way to convert it back to the original array type on the receiving side. This means that a simple technique such as sending your parameters in a Visual Basic for Applications string array will not work as you might hope.     A byte array is flexible because it can hold just about any binary or text-based data. If you don't mind working with a byte array directly, you can pack the message body using code like this:

Dim msg As MSMQMessage Set msg = New MSMQMessage Dim data(3) As Byte ' Fill the array with parameterized data. data(0) = Asc("A") data(1) = Asc("B") data(2) = Asc("C") data(3) = Asc("D") msg.Body = data msg.Send q

How do you unpack this byte array from the message body in a receiver application? It's pretty easy. All you have to do is create a dynamic array reference and assign the message body to it like this:

Dim msg As MSMQMessage Set msg = q.Receive() Dim d() As Byte d = msg.Body ' Now the byte array is populated. ' For example, to inspect value in position 2. Dim Val As Byte Val = d(2)

While it's important to understand that the message body is always stored as a byte array, the technique I have just shown is not always the best way to pack and unpack your parameterized information. Writing and reading byte arrays gives you as much flexibility as MSMQ can offer, but it doesn't offer high levels of productivity. It can also be tricky and time-consuming to write the code for packing and unpacking several pieces of parameterized information to and from a byte array. Several other techniques are easier and faster to program. You should work directly with byte arrays only when the data being packed is fairly straightforward or no other technique can give you the results you need.     Let's put your knowledge of byte arrays to work and pack several parameters into a single message body. Suppose you want to send a request message to submit a sales order. The body of the request message must include a customer name, a product name, and a requested quantity. How do you pack these three pieces of information into a message body? I'll look at four different techniques: using a string parsing technique, using a PropertyBag object, using a persistent Visual Basic object, and using an ADO recordset object. Using a String Parsing Technique     You've already seen that it's easy to write and read a Visual Basic string to and from a message body. A Visual Basic string is stored internally using a COM data type known as a basic string (BSTR). A BSTR maintains the actual string data with an array of Unicode characters. Because a BSTR is based on Unicode, it requires two bytes per character; ANSI strings require only one byte per character.     Packing a BSTR into a message body is easy because MSMQ does the byte array conversions for you behind the scenes. When you assign a string to a message body, MSMQ simply converts the Unicode character array into a byte array. On the receiving side, when you assign the message body to a string variable, MSMQ creates a new BSTR and populates it with the Unicode characters from inside the byte array. The conversion going on behind the scenes is somewhat complicated, but the code that you must write couldn't be easier.     You can use a simple string parsing technique to write the three parameters to a message body. Note that this code will work with Visual Basic 5.0 or 6.0. You can simply create a long string by concatenating your parameters and using a character such as a semicolon to delimit each one. This string can be easily written to and read from the message body. The only tricky part is writing the code to pack and unpack the string.     Parsing strings offers much higher productivity than using a byte array directly. While the code might be tedious to write, it isn't usually very complicated. It's also much easier than working with byte arrays. However, Visual Basic 6.0 presents a few new options for packing your parameters into a message body. Let's look at two Visual Basic 6.0 techniques that offer higher levels of productivity than a simple parsing technique. Using PropertyBag Objects     PropertyBag objects are not new to Visual Basic 6.0. You might have used them if you programmed ActiveX® controls with version 5.0. But Visual Basic 6.0 is the first version that allows you to create PropertyBag objects with the New operator. This means you can create a standalone PropertyBag object to pack and unpack your parameters.     A PropertyBag object is useful because it can automate most of the tedious work of packing and unpacking your parameterized information in a message body. A PropertyBag object allows you to work in terms of name/value pairs. You read and write named properties to the PropertyBag object and it takes care of serializing and unserializing your data from an internal byte array.     Each PropertyBag object has a Contents property, which represents its internal byte array. You can write named values into this byte array using the WriteProperty method. Once you write all your parameters into a PropertyBag object, you can use the Contents property to serialize the byte array into the message body:

Dim msg As MSMQMessage Set msg = New MSMQMessage Dim PropBag As PropertyBag Set PropBag = New PropertyBag PropBag.WriteProperty "Customer", "Bob" PropBag.WriteProperty "Product", "MoeHairSuit" PropBag.WriteProperty "Quantity", 4 msg.Body = PropBag.Contents msg.Send q

The PropertyBag object writes your named values into a stream of bytes using its own proprietary algorithm. Once you pack up a byte array in the sender application, you need a second PropertyBag object on the receiving side to unpack it. You can unpack the message by loading the byte array into a new PropertyBag object and calling the ReadProperty method.

Set msg = q.Receive() Dim PropBag As PropertyBag Set PropBag = New PropertyBag PropBag.Contents = msg.Body Dim Customer As String, Product As String, Dim Quantity As Long Customer = PropBag.ReadProperty("Customer") Product = PropBag.ReadProperty("Product") Quantity = PropBag.ReadProperty("Quantity")

As you can see, the PropertyBag object makes your life much easier. This technique carries some overhead compared to the string parsing technique, however. The PropertyBag object writes proprietary header information into the byte array in addition to the name of each property. To give you an idea of how much overhead is involved, let's compare the two previous code examples. The code for the string parsing technique created a message body 22 bytes long, and the PropertyBag technique created a message body 116 bytes long.     The overhead of the PropertyBag technique depends on the size of the parameters being passed. The overhead becomes less noticeable as your parameter values become larger. Also, keep in mind that the header information for each MSMQ message is quite large itself. An MSMQ message header typically contains 136 bytes or more, no matter how big the body is. You must weigh the trade-offs between productivity and efficiency. Using Persistable Objects With Visual Basic     Another technique for passing parameterized information is to read and write entire Visual Basic objects to and from the message body. But MSMQ can only store objects inside a message body when they implement one of two standard COM interfaces. In particular, MSMQ can serialize the data associated with the object into and out of a message body if the object implements either IPersistStream or IPersistStorage.     The interface definitions for IPersistStream or IPersistStorage contain parameters that are incompatible with Visual Basic. You can't implement these interfaces in a straightforward manner using the Implements keyword. Fortunately, Visual Basic 6.0 has added persistable classes. When you create a persistable class, Visual Basic automatically implements IPersistStream behind the scenes. When you're working with persistable classes, you can read and write objects directly to or from the message body.     Every public class in an ActiveX DLL or EXE has a Persistable property. You must set this property to Persistable at design time to make a persistent class. When you make a class persistent, the Visual Basic IDE lets you add ReadProperties and WriteProperties methods to the class module. You can add the skeletons for these two methods using the wizard bar. (The wizard bar consists of two comboboxes at the top of the class module window in the Visual Basic IDE.) You can also add the InitProperties method, although it isn't required when you use MSMQ.     You can use the ReadProperties and WriteProperties methods to read your properties to an internal PropertyBag object. Visual Basic creates this PropertyBag object for you behind the scenes and uses it to implement IPersistStream. Remember, your object must implement IPersistStream in order for MSMQ to write it to a message body. When MSMQ calls the methods in the IPersistStream interface, Visual Basic simply forwards these calls to your implementations of ReadProperties and WriteProperties.     Using persistable classes with MSMQ is easier than it sounds. You can create a new persistable class and add the properties you want to pack into the message body. Next, you provide an implementation of ReadProperties and WriteProperties. Here's an example of a persistable Visual Basic class module that models a sales order request:

' COrderRequest: a persistable class. Public Customer As String Public Product As String Public Quantity As Long Private Sub Class_ReadProperties(PB As PropertyBag) Customer = PB.ReadProperty("Customer", "") Product = PB.ReadProperty("Product", "") Quantity = PB.ReadProperty("Quantity", "") End Sub Private Sub Class_WriteProperties(PB As PropertyBag) PB.WriteProperty "Customer", Customer PB.WriteProperty "Product", Product PB.WriteProperty "Quantity", Quantity End Sub

As you can see, there isn't too much involved in creating a persistable class. Once you have a persistable class like the one shown above, you can use it to pack a message body like this:

Dim msg As MSMQMessage Set msg = New MSMQMessage ' Create and prepare order request object Dim Order As COrderRequest Set Order = New COrderRequest Order.Customer = txtPCS1 Order.Product = txtPCS2 Order.Quantity = txtPCS3 ' Assign the object to the message body. msg.Body = Order ' WriteProperties is called msg.Send q

When you assign an object to the message body, MSMQ performs a QueryInterface on the object to see if it supports either IPersistStream or IPersistStorage. Since your object supports IPersistStream, MSMQ knows that it can call a method on this interface called Save. When MSMQ calls Save, Visual Basic forwards the call to your implementation of WriteProperties. This gives you an opportunity to write your named property values into the PropertyBag, and they're automatically copied into the message body as an array of bytes.     In the receiver application, things are just as easy. You can rehydrate a persistent object from a message body by creating a new reference and assigning the message body to it like this:

Set msg = q.Receive(ReceiveTimeOut:=0) Dim Order As COrderRequest Set Order = msg.Body Dim Customer As String, Product As String, Dim Quantity As Long Customer = Order.Customer Product = Order.Product Quantity = Order.Quantity

When you assign a message body to a reference using the Set keyword, MSMQ creates a new instance of the object and calls the Load method of IPersistStream. Visual Basic forwards this call to your implementation of ReadProperties. Once again, you use the PropertyBag object to extract your data.     There are a few things to keep in mind when you use persistable classes with MSMQ. First, this parameter-packing technique uses a bit more overhead than the one that uses a standalone PropertyBag object, and it uses considerably more overhead than the string-parsing technique. Second, you should create your persistable classes in ActiveX DLLs so that every application that sends and receives messages can use the same code.     One last thing: if you're using Windows NT, you must install Service Pack 4. Earlier versions of MSMQ are not compatible with the Visual Basic implementation of IPersistStream. Your code will fail when you try to assign an object created from a persistable class to an MSMQ message body. Passing ADO Recordsets     The techniques shown so far have demonstrated a few ways to move the data associated with a single entity (say, a sales order request) in an MSMQ message body. At times, you'll want to move a more complex set of data, such as a list of customers in a single message. The inability to move arrays other than byte arrays makes it pretty difficult to move sets of data around by hand. Fortunately, ADO recordsets are a flexible and convenient way to move more complex sets of data around in MSMQ messages.     Like a persistable class, the ADO recordset component implements IPersistStream. This means you can simply assign an ADO recordset to a message body, and MSMQ and ADO will work together to pack all the data associated with the recordset into the message body. One important thing to note is that this technique only works if you're using an ADO recordset with a client-side static cursor. While this technique is very powerful, it's also extremely simple (see Figure 2). On the receiving side you can harvest the recordset and use it in your client application just as easily.

Set q = qi.Open(MQ_RECEIVE_ACCESS, MQ_DENY_NONE) Set msg = q.Receive() Dim rs As Recordset Set rs = msg.Body ' bind recordset to data-aware grid Set grdDisplay.DataSource = rs

Think of the design possibilities that open up when you use this approach. Suppose you're designing an application for a remote sales force. When a user is connected to the network, you can download the most recent customer list to a local queue on her laptop. Later, when this user is disconnected, she can start adding new sales orders and updating customer information. Behind the scenes, you can track all her inserts and updates in ADO recordsets. When she reconnects to the network, these recordsets can be transparently forwarded to a listener application that adds the new orders and posts customer updates to the database.     You might find that it's much easier to pass around recordsets that are read-only. While ADO makes it possible to update data in a disconnected recordset through the use of optimistic locking, it's not always the best way to go. During the design stage, you must carefully weigh whether updateable recordsets have a place in your application. If you're not careful, a design that includes updateable recordsets can result in lots of unwieldy conflict-management code. It can also severely reduce the overall scalability of your application. If you decide to work only with read-only recordsets, many of these issues disappear.     You might find it makes sense to send data in an ADO recordset even when your application doesn't involve a database. You can create an ADO recordset out of thin air and populate it with your data. The important thing to note here is that ADO is happy to serialize all your data in and out of the message body for you. ADO also generates a schema for the recordset that makes it easier to work with the data on both sides.     Figure 3 starts by defining the schema for a recordset. After opening the recordset, it populates it with a few rows and sends it on its way. The receiving application can deal with this recordset just like any other ADO recordset. The recordset can even be bound to a data-aware grid. How about that for productivity? Sending Messages Inside a Transaction     During your design phase, you should decide how you want the MSMQ transport to move your messages across the wire. You can choose a technique that's faster and less reliable, or you can pick a method that's slower but provides more guarantees. Performance and reliability are always at odds with each other: increasing one will usually compromise the other.     Each MSMQ message has a Delivery property with two possible settings. The default setting is MQMSG_DELIVERY_EXPRESS, which means that the message is sent in a fast but less reliable fashion. Express messages are retained in memory only while they are being routed across various computers toward their destination queue. If a computer crashes while holding express messages, the messages may be lost.     To ensure that a message isn't lost while being routed to its destination queue, you can set the Delivery property to MQMSG_DELIVERY_RECOVERABLE. The message will be flushed to disk as it is passed from one computer to another. The disk I/O required with recoverable messages results in significant performance degradation, but the message won't be lost in the case of a system failure. When you send nontransactional messages, you must explicitly set the Delivery property if you want recoverable delivery. When you send transactional messages, the Delivery property is automatically set to MQMSG_DELIVERY_ RECOVERABLE.     Even when you set the Delivery property to recoverable, you can still experience problems. It's possible that a single message could be delivered to the destination queue more than once. In such a case, the listener application would process the same request multiple times, resulting in undesirable behavior. You can send messages in an even more reliable fashion by using transactional queues.     When you send messages inside a transaction, you get a few guarantees that you don't get with nontransactional messages. First, the MSMQ transport takes extra precautions to ensure that messages are not lost or duplicated. MSMQ also performs extra work to make sure that messages inside a transaction are delivered in the order in which they were sent. Moreover, the sender can always determine when a message has failed on route to a destination queue.     Let's look at an example of in-order delivery. First, you send message A, then you send message B to the same queue. Message A will arrive before message B as long as they are part of the same transaction. This means that message A will be placed closer to the head of the queue. But other messages from other transactions may be interleaved with yours. MSMQ only guarantees the ordering of messages within a single transaction.     In MSMQ, transactional messages must be sent to transactional queues. You can't change the transactional attribute after a queue has been created. Therefore, it is important to indicate whether you want a queue to be transactional when you create it. You can use the first parameter of the MSMQQueueInfo component's Create method to indicate whether you want a transactional queue.     You can send and receive transactional messages with MSMQ in two ways: by using the MSMQ internal transactioning mechanism, or by using external transactions coordinated by the Distributed Transaction Coordinator (DTC) with the two-phase commit protocol. Each offers distinct advantages.     MSMQ provides its own internal transactioning mechanism, which gives the best performance. But when you use internal transactions, MSMQ can't coordinate the transaction with any other type of resource manager, such as a SQL Server™ database. Internal transactions do not use the DTC. Instead, MSMQ uses a more efficient protocol tuned for transactional messaging. Consequently, internal transactions are faster than externally coordinated transactions.     MSMQ supports the OLE Transactions protocol, so you can use it along with other resource managers when you run DTC-based transactions. Your connection to a transactional queue can be enlisted with the DTC. External transactions are also known as coordinated transactions because they are managed by the DTC.     While coordinated transactions are slower than internal transactions, they let you define transactions that incorporate message passing along with operations to other types of resource managers. For example, you can write a transaction that receives a request message, modifies a SQL Server database, and sends a response message. Because the three operations are part of a single transaction, the DTC enforces the ACID (atomic, consistency, isolation, and durable) rules across all this writing activity. MSMQ Internal Transactions     MSMQ provides a shortcut when you want to send only a single transacted message. To send a single message in its own transaction, you can pass the constant MQ_SINGLE_ MESSAGE when you call Send. Here's what a call to Send looks like:

msg.Send q, MQ_SINGLE_MESSAGE

You'll often want to send single messages inside a transaction to guarantee exactly-one delivery. Remember, if you don't send the message inside a transaction, it might be lost or duplicated.     If you want to send multiple messages inside an internal transaction, you must create a new MSMQTransactionDispenser object. You can then invoke its BeginTransaction method to start a new transaction. A call to BeginTransaction returns an MSMQTransaction object. You can pass a reference to this MSMQTransaction object when you call a message-writing operation such as Send, Receive, or ReceiveCurrent.     Here's how you might perform a receive operation on a transactional request queue while sending two messages to a transactional response queue:

Dim td As MSMQTransactionDispenser Set td = New MSMQTransactionDispenser Dim tx As MSMQTransaction Set tx = td.BeginTransaction() Set msg1 = RequestQueue.Receive(Transaction:=tx) msg2.Send ResponseQueue, tx msg3.Send ResponseQueue, tx ' Commit the transaction. tx.Commit

The messages are not sent to the response queue until the transaction has been committed. In this example, you can assume that msg2 will be placed in the queue ahead of msg3. When you receive a message inside a transaction, it is removed from the queue immediately. Another application looking at this queue will not see the message even though the transaction has not been committed. If the transaction is aborted, the message is returned to the queue. This behavior is the same for transactional queues whether you are using internal or external transactions.     There are a few rules you must follow when programming against transactional queues. If you violate one of the rules, MSMQ will raise a runtime error. You can't send a message to a nontransactional queue from inside a transaction. You can't send a message to a transactional queue if the message isn't part of a transaction. You can't receive messages from and send messages to the same queue from inside the same transaction. You can receive transactional messages only when the receiving application is running on the same computer as the transactional queue.     Another thing to keep in mind is that MSMQ transactions run at a lower isolation level than those in SQL Server. SQL Server will always conduct the operations inside an MTS transaction with an isolation level of serializable. MSMQ does not support the serializable isolation level because of the way it manages locks during a transaction.     MSMQ provides an isolation level of read committed between a receiving transaction and a sending transaction. This means that a transaction in a listener application can see the messages sent from a sender's committed transaction even if the sender's transaction started after the listener's transaction. If MSMQ were to run transactions with an isolation level of serializable, it would have to put a lock on the queue and block send operations until the receiving transaction was complete. A locking scheme such as this would pose an unacceptable concurrency restraint in a queuing system.     With respect to two transactions run by two different listener applications, however, the isolation level can be read uncommitted. Let's look at an example. Assume there is a single message in a transactional queue. If transaction A receives the message from the queue, transaction B will see the queue as empty. If transaction A later aborts, the message will be written back to the queue. The queue was really never empty. Transaction B saw the queue as empty because it read the queue in an uncommitted state.     You shouldn't see the inability of MSMQ to run serializable transactions as a flaw. It's really just an inherent problem with transactional queuing. Isolation is sacrificed to maintain higher levels of concurrency. This means you cannot make as many assumptions as you can when working with a DBMS that runs its operations with an isolation level of serializable. External Transactions with MTS     The MSMQ type library contains a component for creating DTC-based transactions called the MSMQCoordinatedTransactionDispenser. This component's interface is identical to the MSMQTransactionDispenser component that you saw earlier. Both components expose a single method, BeginTransaction, which returns an MSMQTransaction object. In fact, in most of your code you can use these two components interchangeably.     The difference between the two components is that the coordinated dispenser works with the DTC, while the internal dispenser doesn't. When you commit an external transaction, the DTC executes the two-phase protocol against MSMQ, which acts as a standard resource manager. The coordinated dispenser also lets you enlist a resource manager other than MSMQ into a transaction. However, there is currently no straightforward way in Visual Basic to enlist another resource manager such as SQL Server using the MSMQTransaction object. You must explicitly enlist a database connection with a low-level call to ODBC or OLE DB. This is something you should avoid in most production code written with Visual Basic. This means that most programmers working with Visual Basic shouldn't use the coordinated dispenser directly.     Fortunately, you can get the benefits of external transactions by sending and receiving messages from within an MTS transaction. The MTS runtime automatically enlists MSMQ—as well as other resource managers such as SQL Server and Oracle—into a declarative transaction. Here's an example of sending a message inside a transactional MTS component:

' assume this component requires a transaction Dim ObjCtx As ObjectContext Set ObjCtx = GetObjectContext() ' (1) write to a SQL server database ' (2) send a message Dim QI As MSMQQueueInfo Set QI = New MSMQQueueInfo QI.PathName = "MyServerComputer\MyQueue" Dim q As MSMQQueue Set q = qi.Open(MQ_SEND_ACCESS, MQ_DENY_NONE) Dim msg As MSMQMessage Set msg = New MSMQMessage msg.Body = "The database has been updated" msg.Send q, MQ_MTS_TRANSACTION q.Close ' (3) commit the transaction ObjCtx.SetComplete ' or call SetAbort

You can pass the constant MQ_MTS_TRANSACTION in a call to Send, Receive, or ReceiveCurrent to indicate that you want your writing operation to be part of the current MTS transaction. If you don't pass a transaction parameter, this value is also the default. Note that passing MQ_MTS_TRANSACTION results in a transacted operation if you make the call from within an MTS transaction.     Passing MQ_MTS_TRANSACTION will result in a Wrong Transaction Usage error if you attempt to send a message inside MTS when running outside the scope of a transaction. If you call Receive or ReceiveCurrent inside MTS when running outside the scope of a transaction, passing MQ_MTS_TRANSACTION will result in a nontransacted receive.     As in the case of an internal transaction, messages inside an MTS transaction are not actually sent until the transaction has been committed. When you receive messages inside a transaction, they are removed from the queue right away. If you call SetAbort to roll back a transaction, your messages are written back to the queue. Remember, there are no guarantees that the message will be placed back in the same position. The previous discussion of MSMQ isolation levels applies to external transactions as well as internal transactions. Transactions with Exactly-one Semantics     Now that you've seen how to pass transacted messages, let's examine the problem associated with running an MTS transaction without MSMQ. COM and RPC are vulnerable to communication failures that can lead to inconsistent results. This is often unacceptable in an OLTP application. In such cases, MSMQ can provide a solution.     Let's look at what can go wrong when a client invokes a COM-based method call to run an MTS transaction. There are three possible cases for failure. First, the method call's request message might fail between the client and the server. Second, the MTS application might crash while the method call is executing a transaction. In both of these scenarios, the intended changes of the transaction are not committed. The third failure scenario is that the method call's response message to the client might fail or be lost after running successfully and committing the transaction.     So here's the problem: what if a client submits a transaction by invoking a COM-based method call on a remote MTS object, but doesn't get a response? The client application has no way of knowing whether the transaction has been committed. If the client submits the same request a second time, there is a chance that the transaction might be committed a second time. As you can see, this creates a problem that COM and RPC can't solve.     Transactional message queues provide a way to submit a transaction request with exactly-one delivery semantics. You can run a transaction with exactly-one semantics by breaking it down into three phases in which messages are sent and received from transactional queues. Figure 4 depicts these three phases.

Figure 4 Exactly-one Delivery

    First, the client submits a message to a request queue. In the second phase, the listener application accomplishes three steps within a single high-level transaction. It receives the client's message from the request queue, it runs the transaction requested by the client, and it sends a receipt message back to the sender's response queue.

    In this second phase, the server must successfully accomplish all three steps or the high-level transaction will be rolled back. If the high-level transaction in phase two is rolled back, the client's message is returned to the request queue. This means that if phase two fails, the listener application can start phase two from scratch by receiving the message from the request queue a second time. When phase two completes successfully, it means the requested transaction has been committed. It also means that there must be a corresponding message in the response queue.

    In the third phase, the client application receives a transactional message from the response queue. The response message indicates whether the requested MTS transaction was completed successfully.

    Transactional queues give the sender the ability to determine which of three possible states a request is in. A request can be waiting to be processed, in the state of being processed, or finished with processing. If the sender sees that the original message is still in the request queue, the request has not been processed. If the sender finds a corresponding message in the response queue, it knows the request has been processed. If there isn't a message in either queue, the sender knows the request is currently being processed. RPC isn't as good as transactional message passing because it doesn't give the client the same ability to determine the exact status of a request.

    While designing your system, you should watch out for a common design problem. It usually occurs when an MTS transaction is rolled back due to business rule violations (say, a transaction is aborted because a customer doesn't have the required available credit). If the aborted transaction also includes the operation that received the original request message, it causes a recursive loop. The request message is repeatedly removed from and rewritten to the request queue. This is known as a poison message.

    You can eliminate the problems associated with a poison message by running two separate transactions. The listener application creates an outer transaction to receive the request message, runs an MTS transaction, and sends a response message back to the client. The MTS transaction is run inside a separate inner transaction. That way, if there's a business rule violation that results in the rollback of the inner transaction, the listener application can determine that and successfully commit the outer transaction by sending a failure notification message to the client.

Summary

    You've seen quite a few techniques for using Visual Basic to integrate MSMQ into your distributed applications. What I hope you take away from this article is the why and how of moving data around the network in MSMQ messages and passing messages inside transactions. MSMQ can give your application a level of flexibility and reliability that you cannot obtain with COM and RPC.

    If you want to do some further reading, I can direct you to two good resources. Surprise! My book, Programming Distributed Applications with COM and Microsoft Visual Basic 6.0 (Microsoft Press, 1998), includes a 40-page introduction to MSMQ programming. For a more complete look at MSMQ, you can pick up a copy of Designing Applications With MSMQ (Addison-Wesley, 1998), by Alan Dickman, which includes many more details about programming MSMQ with both Visual Basic and Visual C++®.

For related information see: Together At Last: Microsoft Transaction Server and Visual Basic 6.0 at http://msdn.microsoft.com/library/techart/msdn_mtsvb.htm. Also check http://msdn.microsoft.com/developer/default.htm for daily updates on developer programs, resources and events.

From the May 1999 issue of Microsoft Systems Journal