Looking to put your client/server app into warp drive before the 21st century? The Microsoft Messaging Queue (codenamed Falcon) provides an effective, reliable, resilient method of communication between systems.

Microsoft® Message Queue (MSMQ), originally codenamed Falcon, was developed to respond to the need for an effective and reliable method of asynchronous communication between systems. MSMQ offers ease of development, greater reliability, a high degree of compatibility with various systems, and other features that are essential to intersystem communication.       Most companies have invested huge amounts of time and money into client/server systems over the past several years and subsequently their operations depend heavily upon these systems. Many of these systems were initially developed as standalone solutions. To make these standalone systems communicate with each other, developers have to write either batch-oriented solutions (systematic file transfers between the systems) or custom interfaces, which are usually specific to the situation and difficult to reuse with other systems. Basically you end up reinventing the wheel each time you want to interface with an existing system. Messaging       Simply put, message queuing facilitates communication between applications by sending and receiving messages. The goal of this article is to concentrate at a high level on what MSMQ is, what it brings to developers, and the basic programming techniques used to develop MSMQ solutions. For the purpose of this article, I will assume that you have a basic understanding of message queuing and message-oriented middleware.       There are several common scenarios when it makes sense to use messaging. The store-and-forward scenario occurs when sending and receiving applications will be running or connected to the network at different times. A defensive scenario occurs in environments where connectivity is normally available, but the cost of message loss or interruption is high. You may need to have your sending application moving on to other tasks while it waits for responses from the receiving application. Situations where all requests must be logged, audited, and possibly reprocessed to recover from failures constitute another candidate for messaging-based solutions.       Message queuing is not a remedy for all intersystem communication problems. In fact, there will be some scenarios where message queuing poses unnecessary overhead. This will become most evident when the client app requires an answer from the other system before it can continue processing.       Message queuing isn't just another fancy name for MAPI and email. In fact, it provides the infrastructure technology needed to write an email system like Microsoft Exchange as well as any other type of system that needs to communicate with other systems. Later on I'll show you a poor man's email system written with MSMQ. MSMQ Features       To meet the requirements of an effective communication vehicle, MSMQ provides a suite of features including: Asynchronous communications High scalability ActiveX® support Guaranteed message delivery Transaction support One time, in-order delivery Message routing services Dynamic configuration Integration with Windows NT® Security Notification services Support for MAPI, Microsoft Exchange, and other Microsoft products       To illustrate each of these features, I will implement sample purchasing and inventory systems. These two systems need to communicate so that the inventory levels are accurately known. An easy solution is to have a batch update job that runs at predetermined times. This job will process all purchases since the last batch job.       This scenario might not require any MSMQ components if the batch job can access the purchasing database. However, the purchasing and inventory systems might not be able to connect to each other directly. You could create a very simple interface between the systems, such as a file produced by the purchasing system. This file would then be transferred to a common location and processed by the inventory system (see Figure 1).

Figure 1: Batch Updating

      This type of solution presents numerous possible points of failure that MSMQ can help avoid. With MSMQ, a message could be sent to the inventory system through the purchasing system every time an item is purchased. Another application could be created within the inventory system that monitors these messages and then processes them accordingly by making adjustments to the inventory database. MSMQ even allows the messages to be encrypted with the Microsoft CryptoAPI.
      MSMQ allows you to perform this entire operation within a transactional unit of work, ensuring the success or failure of the message along with any other related functions, such as database updates. In a successful transaction, the message would be read and erased from the queue and the inventory database would be updated. However, if a failure occurred when updating the database, the rest of the process would be rolled back as shown in Figure 2.

Figure 2: MSMQ Rollback on Failure

      You could also handle much of the communication between two applications through standard OLE Automation interfaces. As long as both systems can access COM interfaces, a DecrementInventory interface in the inventory system could be called from the purchasing system. These systems could easily utilize DCOM and Microsoft Transaction Server in the distributed environment to provide system intercommunication (see Figure 3).

Figure 3: DecrementInventory Interface

      There are several drawbacks to this type of solution. First, all processing is halted on the client application until the server returns control. While for some small systems this might not be a consideration, in many large systems—such as an Internet-based purchasing system—this can be costly (see Figure 4).

Figure 4: Sending a Message to Message Queue

      MSMQ solves this problem by allowing messages to be sent to recipients asynchronously, freeing up the client application to handle other tasks without waiting for a response from the receiving application. In some instances, however, the sending application may want to be notified whether the message was received successfully. MSMQ allows the sender to review responses, let another application handle them, or just ignore them completely. MSMQ can also automatically journal messages to provide audit trails.
      Threads and other advanced programming techniques could also be used to solve this problem. However, these advanced techniques have their own headaches and are not fully supported by all programming languages (like Visual Basic®).
      Note that an MSMQ solution does not preclude a COM/DCOM solution. The inventory example could easily use MSMQ and a COM interface. The server application that processes the messages from the purchasing system could call an ActiveX server that is responsible for updating the inventory system.
      A straight COM/DCOM solution requires both systems to be up and running at the same time and accessible to each other through the network. This assumption is not always a reality: networks go down, limitations exist with regard to accessibility, and network resources are not always plentiful. Several key features of MSMQ (guaranteed message delivery, connectionless messaging, and store-and-forward) help with this issue.
      By successfully sending the message, the client application is guaranteed that the message will be delivered. This delivery could occur immediately if the receiving application is up and monitoring for messages, or could occur at a later time or date. This feature allows system administrators to easily adapt the system as business requirements change. When the system is experiencing small purchase volumes, the inventory server could still be run as a batch job at predetermined times. As business requirements change to dictate a more real-time view of inventory or if volumes increase, the server could run at all times to process purchases. If volumes increase so much that this solution becomes a bottleneck, system administrators have the luxury of starting several instances of the inventory server during peak periods to help reduce the load on any one server.
      The fact that the two applications do not even need to be connected to the network at the same time opens up a whole world of opportunity in store-and-forward situations. This scenario is common in industries like insurance sales where much of the work force is out in the field, disconnected from the corporate network for much of the day, or where batch jobs are run at night while other applications are offline. Historically, the laptop user has to either dial in to the corporate network and work online or transactions are written to disk and later uploaded to the parent system.
      MSMQ frees you from the necessity of writing these types of solutions. You simply make calls to send messages to the receiving application. As soon as the laptop user connects to the corporate network, outgoing messages are routed to the appropriate location by MSMQ, and any incoming messages are sent to the laptop user. MSMQ guarantees that the messages will be delivered as soon as the network connections are established and the receiving applications issue requests for sent messages.
      One of the most important features of an MSMQ solution is the ease with which it can be developed. All of MSMQ's features can be accessed through an API. If API calls are not your cup of tea, Microsoft has also developed an ActiveX interface that allows you to access most of the MSMQ functionality. This feature opens up MSMQ to traditional programming languages such as Visual C++®, Visual Basic, and Visual FoxPro®. In addition, it allows Active Server Pages (ASP), Microsoft Office applications, and Transact-SQL code to access MSMQ. This degree of flexibility allows developers to use the tools they are already familiar with.
      Another important feature offered by MSMQ is the one-time, in-order delivery it offers. MSMQ guarantees that each message is delivered one time only, and that messages are received in the order that they are sent. One-time delivery can be important in a wide range of industries and applications. In a stock brokerage, multiple deliveries of a single buy request could have serious repercussions. Likewise, banking applications could debit or credit customer's accounts multiple times, or an inventory system could order twice as much inventory as desired. Similarly, in-order delivery can be invaluable; if one system finishes its operations, sends a message with the subsequent data to a corollary system, and then sends a message to initiate operations, the misqueuing of these messages could result in initiation of operations prior to the receipt of required information.
      For businesses in which delivery of information possesses a high degree of liability, MSMQ offers guaranteed message delivery. For instance, stock brokerage firms rely heavily on the fact that monetary transactions occur when and where they were reported. Accuracy in these transactions constitutes the lifeblood of such a company. MSMQ ensures that messages will be delivered, regardless of whether receiving applications are running, the connecting network is functional, or any portion of the connection fails.

Integration With Other Products
      MSMQ is tightly integrated with the Windows® platform and many other Windows-based products. MSMQ takes advantage of many of the security features available through Windows NT, and controls queue access through the Windows NT Access Control List mechanism.
      MSMQ can also communicate with MAPI-based applications and Microsoft Exchange. Included in MSMQ is a MAPI Transport Provider and an MSMQ Exchange Connector. These utilities provide all the necessary functionality for MSMQ to interoperate with these messaging facilities.
      Currently MSMQ is available for Windows 95 and Windows NT. Microsoft has also introduced a suite of products called FalconMQ in conjunction with Level8 Systems that extends MSMQ to platforms such as VMS, AS/400, Unix, MVS, OS/2, and UNISYS. FalconMQ also provides interoperability with IBM's MQSeries product on all platforms that support Level 2 MQSeries functionality, as well as IBM's CICS Transient Data Queue on MVS.

MSMQ Architecture and Administration
      MSMQ applications communicate through a series of messages. A message is simply a unit of information or data that is sent between computers. The message contains both source and destination information, as well as textual or binary content. The meaning of this content has to be agreed upon by both sending and receiving applications.
      MSMQ allows messages to be delivered in one of two ways: express or recoverable. Express messages are stored in memory as opposed to the disk storage methods utilized by recoverable delivery. Express messages usually require fewer resources and are faster than recoverable messages. Recoverable messages, on the other hand, will not be lost in the event of computer failure, unlike express messages.
      MSMQ is responsible for storing the messages in holding areas called queues. You can think of these queues as mailboxes for messages that are awaiting delivery. MSMQ utilizes these queues to protect against message loss, as well as to provide a place for receiving applications to monitor for new messages. MSMQ routes and delivers messages based upon a combination of queue priority and message priority. Messages are routed and delivered first by queue priority, then by message priority.
      All computers operate within one MSMQ enterprise. The enterprise is divided into sites, where communication between any two computers is fast and inexpensive. In many cases, sites correspond to the physical grouping of computers, such as all the computers within the same building. A good example of this would be a company with offices worldwide. The company as a whole would belong to one enterprise. This enterprise would be divided up into multiple sites representing each of the company's offices (New York, London, Paris, Munich, and so on). MSMQ handles all communications between different computers within the same site. In fact, the computers within a site do not all have to run the same network protocols, or even be able to communicate directly with each other.
      Sites are connected to other sites within the enterprise through communication links called site links. MSMQ handles message routing across multiple sites on these links. In large enterprises where there are multiple sites, quite a few routing possibilities exist. In these situations, administrators assign site link costs (the cost of sending messages between one site and another). MSMQ will choose the currently available routing path with the lowest overall site link cost. If network connections fail, the message will be rerouted along the route with the next lowest cost. This feature helps lower the number of points of failure and helps increase the overall effectiveness of the product.
      All computers that run MSMQ belong to one enterprise and access information from the same MSMQ Information Store (MQIS). The MQIS is a distributed database that holds enterprise topology, enterprise settings, computer information, and queue information. MSMQ-based applications can query the MQIS to find queues and obtain their properties.
      MSMQ queues can be broken into two main groups: public and private. Public queues are those published in the MQIS. All public queues are replicated throughout the enterprise and can therefore be located by any computer within the enterprise. Private queues are not published in the MQIS, and therefore don't add to the MQIS replication load. Private queues can be accessed only by applications that have access to the full path name or format name of the queue.
      All machines that participate in MSMQ can be classified into three categories: servers, dependent clients, and independent clients. MSMQ servers can be broken out into four subcategories: Primary Enterprise Controller (PEC), Primary Site Controller (PSC), Backup Site Controller (BSC), and MSMQ Routing Server. All of these servers must run under Windows NT. To set up an MSMQ enterprise, a PEC must be set up first. It will also serve as a PSC for one site and as an MSMQ routing server. The PEC must be installed before any other PSCs can be installed. A PEC server contains information about the enterprise configuration and the certification keys that are used in authenticating messages in a database.
      After the PEC is set up and an enterprise is created, a PSC must be installed for each additional site in the enterprise. The PSC contains information about the computers and queues in the site and also functions as an MSMQ routing server. MSMQ routing servers support dynamic routing and intermediate store-and-forward message queuing. They allow computers using different protocols to communicate.
      A site does not require a BSC, but one or more BSCs should be installed at each site to provide load balancing and failure recovery should the PSC or PEC fail. The BSC holds a read-only replica of the PSC or PEC database and also functions as an MSMQ Routing Server (see Figure 5).

Figure 5: MSMQ Enterprise

      MSMQ independent clients can be installed on computers running Windows 95, Windows NT Workstation, or Windows NT Server. They can create and modify queues locally and send and receive messages. Independent clients can also create queues and store messages on the local computer without synchronous access to an MSMQ server. Additionally, independent clients can send messages to public queues while disconnected from the network without any additional application design or configuration. The only caveat is that the independent client must be installed with a connection to the PSC since it must have access to the MQIS on the PSC. The main difference between independent clients and servers is that the independent clients do not have the intermediate store-and-forward capability of MSMQ servers, nor do they store information from the distributed MSMQ database.
      MSMQ dependent clients function very much like independent clients, except that they cannot function without synchronous access to a supporting server. Dependent clients rely on their supporting server to perform all standard MSMQ functions on their behalf (creating queues or sending and receiving messages). Dependent clients can be installed on computers running Windows 95, Windows NT Workstation, or Windows NT Server. MSMQ servers can support up to 15 dependent clients.
Figure 6: MSMQ Explorer

      The entire MSMQ enterprise can be managed through the MSMQ Explorer (see Figure 6). The MSMQ Explorer provides an interface for managing all machines in an MSMQ environment from a single point of control. This reduces the cost of administering an MSMQ environment because administrators in a central location can add, delete, move, and manage queues dynamically.

Poor Man's Email
      Now let's dive into some code and see just how simple it is to write MSMQ applications. To illustrate the ease of MSMQ development, I've created a sample Poor Man's Email application that allows multiple users to send and receive messages. This sample illustrates how you create message queues and send and receive messages. These samples were developed with Visual Basic and the MSMQ ActiveX interface. (However, these samples were written with a beta version of MSMQ. Some of the names within the MSMQ object model may have changed in the release version of the product.)

Identifying Queues and Retrieving Messages
      To keep things simple, the messages for each user of the Poor Man's Email system are stored in a queue named after the user. The user is required to enter a login ID to enter the system. This ID is used to obtain the queue full of messages that belongs to the user.
      Several objects are needed to access an existing queue or create a new one: MSMQQuery, MSMQQueueInfo, MSMQQueueInfos, and MSMQQueue. A variable of each of these types is created by the system when the user logs in.

 Private que as MSMQQueue    'form level variable
 Dim qry As New MSMQQuery
 Dim qInfo As MSMQQueueInfo
 Dim qInfos As MSMQQueueInfos

 Set qInfos = qry.LookupQueue (Label:=sLogin, ServiceTypeGuid:=MailUUID)
     qInfos.Reset
     Set qInfo = qInfos.Next

 If qInfo Is Nothing Then
         Set qInfo = New MSMQQueueInfo
         sComputerName = Environ("COMPUTERNAME")
         If sComputerName = "" Then
            sComputerName = "."
         End If
         qInfo.PathName = sComputerName + "\" + sLogin
         qInfo.Label = sLogin
         qInfo.ServiceTypeGuid = MailUUID
         qInfo.Create 
 End If

 Set que = qInfo.Open(MQ_RECEIVE_ACCESS, 0)
 Set MailEvent = New MSMQEvent
 que.EnableNotification MailEvent

 Private Sub MailEvent_Arrived(ByVal Queue As Object, ByVal Cursor As Long)
 
    Dim msgArrived As MSMQMessage
    Set msgArrived = Queue.PeekCurrent(True, True, 1000)
    Do While Not msgArrived Is Nothing
       PopulateMailMessageInListView msgArrived
       Set msgArrived = Queue.PeekNext(True, True, 1000)
       Loop
 •••
 End Sub

 Private Sub Timer1_Timer()
      lvwMail.ListItems.Clear           'clear out list view
      que.Reset                         'reset the queue
      que.EnableNotification MailEvent  'search for mail messages
 End Sub

Explicitly Retrieving Mail
      Messages can also be received by looping through all the messages contained in the MSMQQueue object. This is implemented in the sample application through the RetrieveMail function. RetrieveMail retrieves the messages in the queue and populates the listview on the main form with their pertinent characteristics. The LookUpQueue method is called on an MSMQQuery object (qry), instantiating an MSMQQueueInfos object. The Open method is then called on the resulting MSMQQueueInfo object, opening the queue. Using the PeekNext method of the queue, the first message is retrieved from the queue. Information describing this message is then posted to the listview (see Figure 7).

Sending Messages to Queues
      Saving the best until last, let's take a look at how easy it is to actually send a new message. Once again, I will be using several of the same objects that I used for retrieving a message: MSMQQuery, MSMQQueueInfo, MSMQQueueInfos, MSMQQueue, and MSMQMessage. First, I obtain a reference to the queue to which I want to send the message. After obtaining this reference, I open the queue, fill in the Label, Priority, and body properties of the message object, and then send the message with the send method of MSMQMessage. This code is shown in Figure 8 .
      This shows just how easy it is to program MSMQ solutions through the ActiveX interface. This example illustrates only some of the more basic programming techniques available to developers through the object model. In a future article, I will take a more in-depth look into developing MSMQ applications with Visual Basic, Visual C++, ASP, and Microsoft Office applications.

Conclusion
      MSMQ provides easily implemented and effective intersystem communications to today's Windows-based development. This technology offers considerable advantages in situations where you need asynchronous communication, guaranteed message delivery, or support for irregular connectivity.
      The features incorporated into MSMQ make it an ideal messaging facility for numerous situations. These features place emphasis on reliability, ease of use and development, and compatibility with existing messaging facilities. MSMQ also offers administrators the ability to manage MSMQ systems from the desktop with point-and-click simplicity.

From the March 1998 issue of Microsoft Interactive Developer.