The Microsoft Message Queue

MSMQ Preface

The new age ushered in by high-speed networks (even the not so fast Internet) has brought about the requirement of new technology to cater to a lot of high scale distributed applications. Everything today has to be fast and reliable even if the applications are distributed with one located in Mars and the other on Neptune, and support the basic principle of completeness at the transactional level. Microsoft has addressed this in a wonderful way with its Windows DNA, 3 Tier Concepts. Ever got irritated when people told they want the latest hot browser interface for your application and not your Classical GUI which you really sweated out to get it running? All this changes with the 3 Tier Concept introduced by Windows DNA.

Windows 2000 (NT 4.0 with its service packs) provides you with MTS, IIS, MSMQ. Microsoft Transaction Server (MTS) provides you with the ability to write services (or packages as they are called) which support transactions by taking part in transactions supported by the Transaction Manager in MTS. MTS allows you to provide object pooling and load balancing. Therefore applications avail great benefit in the way they can be scaled as it would no longer mean that when your applications have to handle a higher load than anticipated it does not necessarily mean scratching your application from start to get that extra push. Rather it would mean putting a few extra processors, a little bit more RAM and may be a few extra processors to get that load balancing and a little bit more business for the Hardware folks.

Your application packages now support the basic two layers: Database Access Layer to interact with your data store, and the business layer to provide the essential business services. A change at the Data base layer does not necessarily mean a change in the business layer - at least not in Business Services Exposed API. Your third layer is your Presentation layer and may be your Classical GUI, or the browser Interface, or even some new interface that some one may ask in the next few years.

Now what happens in the distributed approach is that it is quite possible that all the servers that participate in your transaction are not available at the same time! An example may be the user filled in a request form about his email account details. These details that you have stored in a remote SQL server which may be down. Why not just accept the request and keep it in some kind of message store and then later forward to your SQL server application for it to process the request. The user need not get an "SQL server is down" message, why should he even know? Here enters Microsoft Message Queue, which systemizes this requirement of store and forward message requests to destinations when available.

Prerequisites

MSMQ is supported on Windows 2000, Windows NT with Option Pack 4+, Windows 95 and Windows 98. (Note MSMQ can communicate with IBM Queues defined by the MQSeries.)

To program MSMQ you will need the MSMQ API, for which you need any COM aware Compiler like Microsoft Visual C++, Microsoft Visual Basic or Delphi. Programmer awareness of COM and DCOM at least to a concept level is ideal. Microsoft provides MSQM in two flavors COM API and C API. This tutorial however uses COM API and Visual C++ as the COM API is much simpler to use.

Introduction

MSMQ was originally code named as Falcon. MSMQ enables applications to reliably communicate with each other even in unreliable distributed environments in an efficient yet reliable way where intermediate servers need not be essentially available at all times.

Purpose

Message Queuing (MSMQ) technology enables applications running at different times to communicate across heterogeneous networks and systems that may be temporarily offline. Applications send messages to queues and read messages from queues. The following illustration shows how a queue can hold the messages used by both sending and receiving applications.

MSMQ also provides Journaling for requests that occur, that can enable greatly in creating in data recovery and system audits.

MSMQ deploys the Store and Forward Mechanism in which messages are not exchanged directly between applications but rather through a message store known as a message queue. This allows applications at each end to be not necessarily available at the same time. MSMQ is not a technology that is brand new, but it is a sincere attempt to standardize and make available the same for Microsoft Platform.

How MSMQ?

To program MSMQ you will need the MSMQ API, for which you need any COM aware Compiler like Microsoft Visual C++, Microsoft Visual Basic or Delphi. Programmer awareness of COM and DCOM at least to a concept level is ideal. Microsoft provides MSQM in two flavors COM API.

How MSMQ can participate in Transactions

MSMQ supports the following type of Queues

• Public Queues

  These Queues can be viewed and located by everyone.

• Private Queues

  These Queues can be viewed and located by only the applications, which are aware of the complete path of the Queue.

• Transactional Queues

  These Queues can participate in Transactions and allow in Transactional Support. They have a heavy I/O overhead; they are guaranteed and are recoverable. MSMQ API when used under MTS (Microsoft's Transaction Server for logical transactions) can become part of the current Transaction. In this context messages are transmitted together or not at all, and if sent the messages will be provided in the same sequence as they are transmitted. This discussion is outside the purview of the current discussion and will be discussed in further tutorials.

Native COM Support in Visual C++

Quite often we (VC Developers) tend to get irritated seeing the VB Developers calling COM functions just by creating objects and calling their member functions. Whereas VC developers struggle using the CreateDispatch and slogging out the other details. However it is not all that bad because of VC's Native COM Support whereby you don't have to do the CreateDispatch calls with a GUID all by yourself. VC would internally do it all for you. It would declare the Interfaces and instantiate all the objects using CreateDispatch just when you want to create the objects.

This Native COM Support is possible by the use of the #import directive. This directive allows you to import information from a COM module (like a COM DLL or a type library).

The #import syntax is described as

#import <filename> [attributes] 

The #import directive causes the compiler to generate two files: a .TLH and a .TLI file (in your debug directory). These files contain the Interface declarations and member function implementations. It is quite similar to the pre-compiled header (PCH) that standard VC applications create.

In our sample I have used the no_namespace attribute so that the compile does not generate the namespace for the type library that we intend to use, but uses the namespace of the original interface definition. Further it is quite useful to check out the contents of the generated files by the compiler for additional information.

Programming MSMQ

The class CMSMQApiWrapper is an MSMQ API Wrapper which should suffice for most common use. This wrapper however doesn’t wrap the Event notification for message arrivals in queues, but this is demonstrated in the sample Application using the CMSMQEventNotifier class.

The MSMQ API Wrapper imports the mqoa.dll which contains the message queuing interfaces using the #import "mqoa.dll" no_namespace directive. The constructor of the CMSMQApiWrapper initializes the OLE environment by a call to OleInitialize.

Creating a Queue

//Creating A Queue [Public/Private] 

int CreateQueue(LPCTSTR pszPathQueue,LPCTSTR pszQueueLabel,BOOL bPublic); 
int CreatePublicQueue(LPCTSTR pszPathQueue,LPCTSTR pszQueueLabel); 
int CreatePrivateQueue(LPCTSTR pszPathQueue,LPCTSTR pszQueueLabel);

The above member functions facilitate in the creation of Queues. A private Queue is defined by preceding the Queue with the Private$ path.

A Queue using the above API is created by instantiating an IMSMQQueueInfoPtr object and setting its PathName and Label property and calling the create method.

Locating a Queue

//Locating A Public Queue by Label 

int LocatePublicQueue(LPCTSTR pszPathQueue,IMSMQQueueInfoPtr &qinfo); 

The above member functions facilitate in the Location of Public Queues.

A Queue can be located using the above API by creating a MSMQQuery object and using its LookUp function given the Label. This on success will return a MSMQQueuesInfo Object which can be looped to get the queues matching the given Query.

Deleting a Queue

//Deleting A Queue 

int DeletePublicQueue(LPCTSTR pszQueueLabel); 

The above member functions facilitate in the deletion of Public Queues.

A Queue using the above API is deleted by first Locating the Queue by Label, which accepts an IMSMQQueueInfo pointer. On Success the Delete method is called on the IMSMQQueueInfo object.

Synchronously Purging A Queue

//Synchronously Purging A Queue 

int SynchronousPurgePublicQueue(LPCTSTR pszQueueLabel); 

This method allows you to purge a Public Queue, by first Locating the Public Queue by Label, it then Opens the Queue with Receive Access, and then finally receives all the messages in the Queue, thereby Purging all the messages in the Queue.

Sending a Message to a Queue

// Sending A Message [String other data types and persistent objects for files) 

// Sending A String Message is currentlu supported 

// Other Data types, files and even persistent objects can be sent 

// using MSMQ 

int SendStringMessage(LPCTSTR pszQueueLabel,CString szLabel,CString szMessage); 

This method allows you to send a message of type string. Any kind of data can be passed to an MSMQ Queue as it accepts data in the form of Variants. Therefore, data that can be sent other than the simple data types are files and Persistent Objects (The Objects that implement the IPersistStream Interface). This member function sends a string message using a Variant of type VT_BSTR and a given Label to the specified MSMQ Queue. This is done by first locating the Queue and opening it with Send Access. Then an object of the type IMSMQMessage is instantiated and its Body, Label properties are set and the message is sent using the Send function for the message.

Retrieving a message from a Queue Synchronously

// Reading A Message[String other data types and persistent objects for files) 

// Reading A String Message is currentlu supported 

// Other Data types , files and even persistent objects can be read 

// using MSMQ 

// Requesting Events Notification For A Queue is implemented in the 

// CMSMQEventNotifier class 

int ReadStringMessage(LPCTSTR pszQueueLabel, CString &szLabel, CString &szMessage); 

This method allows you to retrieve a message of type string. Any kind of data can be passed to an MSMQ Queue as it accepts data in the form of Variants. Therefore, data that can be sent other than the simple data types are files and Persistent Objects (The Objects that implement the IPersistStream Interface). This member function retrieves a string message using a Variant of type VT_BSTR and the Label that was sent to the specified MSMQ Queue. This is done by first locating the Queue and opening it with Receive Access. Then an object of the type IMSMQMessage is instantiated and its Body, Label properties are retrieved from the message queue pointer.

Retrieving a message from a Queue Asynchronously

Messages can be retrieved asynchronously from an application using the MSMQEvent object. The MSMQEvent object enables the creation of a connection point to a sink interface in the application. This is demonstrated in the sample application where events are picked up at the message queue server for further processing.

Setting a Queues Properties

// Retrieving and Setting Queue Properties 

int SetAuthenticationLevelOfQueue (LPCTSTR pszQueueLabel,int iAuthenticationLevel); 

This function sets the Authentication Level of the Queue.

int SetPriorityLevelOfQueue(LPCTSTR pszQueueLabel,int iPriorityLevel);  

This function sets the Priority Level of the Queue.

int SetJournalingLevelOfQueue(LPCTSTR pszQueueLabel,int iJournalingLevel);  

This function sets the Journaling Level of the Queue.

int SetMaximumSizeOfQueueJournal(LPCTSTR pszQueueLabel,long lMaximumSize);  

This function sets the maximum size of the Queue Journal.

int SetLabelOfQueue(LPCTSTR pszQueueLabel,CString szLabel);  

This function sets the Label of the Queue.

int SetPrivacyLevelOfQueue(LPCTSTR pszQueueLabel,int iPrivacyLevel);  

This function sets the Privacy Level of the Queue.

int SetMaximumSizeOfQueue(LPCTSTR pszQueueLabel,long lMaximumSize);  

This function sets the maximum size of the Queue.

The above functions Wrappers set the respective properties for the Queue and call its update function after locating the Queue.

Retrieving a Queues Properties

int RetrieveAuthenticationLevelOfQueue(LPCTSTR pszQueueLabel,int &iAuthenticationLevel);

This function retrieves the Authentication Level of the Queue.

int RetrievePriorityLevelOfQueue(LPCTSTR pszQueueLabel,int &iPriorityLevel); 

This function retrieves the Priority Level of the Queue.

int RetrieveFormatNameOfQueue(LPCTSTR pszQueueLabel,CString &szFormatName); 

This function retrieves the Format Name of the Queue.

int RetrieveTransactionLevelOfQueue(LPCTSTR pszQueueLabel,int &iTransactionLevel); 

This function retrieves the Transactional Level of the Queue.

int RetrieveReadLevelOfQueue(LPCTSTR pszQueueLabel,int &iReadLevel); 

This function retrieves the Read Level of the Queue.

int RetrieveJournalingLevelOfQueue(LPCTSTR pszQueueLabel,int &iJournalingLevel); 

This function retrieves the Journaling Level of the Queue.

int RetrieveMaximumSizeOfQueueJournal(LPCTSTR pszQueueLabel,long &lMaximumSize); 

This function retrieves the Maximum Size of the Queue Journal.

int RetrieveLabelOfQueue(LPCTSTR pszQueuePath,CString &szLabel); 

This function retrieves the label of the Queue.

int RetrievePrivacyLevelOfQueue(LPCTSTR pszQueueLabel,int &iPrivacyLevel); 

This function retrieves the Privacy Level of the Queue.

int RetrieveMaximumSizeOfQueue(LPCTSTR pszQueueLabel,long &lMaximumSize); 

This function retrieves the Maximum Size of the Queue.

int RetrievePathNameOfQueue(LPCTSTR pszQueueLabel,CString &szPathName); 

This function retrieves the Path Name of the Queue.

The above functions Wrappers retrieve the respective properties for the Queue and call its Refresh function after locating the Queue.

Sample MSMQ Application

The purpose of this application is to function as a Graphical User Interface Based Registry system as well as to provide a similar WEB interface using Active Server Pages. This system will accept the following details from the user while registering:

• UserName
• Password
• Age
• Sex
• Address
• EMailId
• BirthDate

On registering a message it is placed on the InboundRequests Queue on the MSMQ Machine. A Target Application implementing MSMQEvents provides the Message Processing by adding this information to a database and sending a response email. The user at a later point of time on providing his Email Address can request for all details registered by him to be sent to him by email. It uses an email SMTP component developed by P J Naughter available here.

Shown above is the screen from the MFC GUI Sample Application to register the user. This puts a message into the InboundRequests Queue after creating the Queue if it does not exist.

Shown above is a dialog from the MFC GUI Sample Application for the user to request his/her details. This puts a message into the InboundRequests Queue after creating the Queue if it does not exist.

This test application would require MSMQ to be installed on all the clients, the sample however writes on the local machine Queue and so you would have to do the routing or you could modify the test application to write on the remote Queue. If the remote machine is down MSMQ will ensure your message reaches the destination.

Shown above is the screen from the ASP based Web Page to register the user. This puts a message into the InboundRequests Queue after creating the Queue if it does not exist.

Shown above is the screen from the ASP based Web Page for the user to request his/her details. This puts a message into the InboundRequests Queue after creating the Queue if it does not exist.

The Active Server Pages does not require MSMQ Software to be setup on each client page. The Register and Request user details page call a server side script embedded in srvmsgsend.asp file. The script creates a queue on the server if it does not exist and then places a message on the Queue.

The server side script is as follows:

<% 
on Error resume next 
set queueinfo = CreateObject ("MSMQ.MSMQQueueInfo") 
queueinfo.PathName = ".\InboundRequests" 
queueinfo.Create 
set queue = queueinfo.Open ( 2, 0 ) 
If queue.IsOpen Then 
Set message = CreateObject("MSMQ.MSMQMessage") 
message.Body = CStr (Request.QueryString ( "Body" )) 
message.Label = CStr (Request.QueryString ( "Label" )) 
message.Send queue 
Response.Write "Message has been sent, you will be notified ASAP." 
End If 
queue.Close 
%>

The MSMQReceiver is the test application, which manages events for the MSMQ Queue. It enables notification for the given Queue and on arrival of messages in to the Queue it processes the information by registering or retrieving the user information from an Access MDB and then mailing the registration users to the user.

It implements a CMSMQEventCmdTarget class derived from CCmdTarget MFC's magic class for OLE Automation. This implements the Arrived method as the entry point for message arrivals, which could be customized, for your applications.

The EnableNotification method of the class opens the Queue and calls its EnableNotification method. This method obtains a disp event to the MSMQ.MSMQEvent COM class which is used to Enable Notification on the specified Queue. It then obtains the ConectionPointContainer interface to the sink. It then gets the Connection Point defined by IID_IMSMQSinkEvent. After the successful call the Connection Point is advised about the Dispatch interface to the CMSMQEventCmdTarget class.

The DisableNotification method of the class closes the Queue and removes the connection to the sink. This method obtains a disp event to the MSMQ.MSMQEvent COM class which is used to Enable Notification on the specified Queue. It then obtains the ConectionPointContainer interface to the sink. It then gets the Connection Point defined by IID_IMSMQSinkEvent. After the successful call the Connection Point is unadvised about the Dispatch interface to the CMSMQEventCmdTarget class.

The code to implement the same is relatively simpler than the above description.

Summary

Having understood the functioning and the basic design of the MSMQ, you may now better understand what queuing could offer in your next new generation applications. The web should now be showing more assured and guaranteed message based queuing applications which could offer more security and reliability without a compromise on the performance. You can now go ahead and explore the entire MSMQ and MTS based DNA three tiers Applications. It would be ideal as time goes for application to start communicating with each other using these robust features and standard document formats like XML.

References and some Good Links