How to Use IMessageFilter: the complete edition

• Download source code - 41.5 Kb

Introduction

In my previous article on IMessageFilter , I described how to use this interface in your application. I implemented an IMessageFilter interface (class IMFImpl), and my COM object was inherited from this IMFImpl. Later on I realized that this solution was not good enough, and that I needed to use IMessageFilter as an independent object. The reason for that (which is not relevant here) was the destruction of my object. Generally, after my object was killed, COM held the pointer to my message filter, and tried to kill it, causing a crash.

So, how to use IMessageFilter as a self-dependent object? Well, I couldn't find any example on the net, and so I decided to create my own solution. I have tried to explain this here so that someone might find it helpful.

Understanding the code: server side

IDL

In your COM object's IDL file, add the new interface:

IMyMessageFilter : IMessageFilter
{
    HRESULT block();
    HRESULT unblock();
    HRESULT registerMessageFilter();
    HRESULT unregisterMessageFilter();
     
};

This self-dependent message filter knows how to register and unregister itself. The registerMessageFilter and unregisterMessageFilter methods are good for registering and un-registering the message filter. The block and unblock are good for cases of reentrancy: if we encounter this case, we can decide whether to block the call, or to let it in. (The object has an internal flag that is set in these methods). Then, declare the coclass that implements this interface (MyMessageFilter):

coclass MyMessageFilter
{
   [default] interface IMyMessageFilter;
}

H / CPP files

Create H and CPP files, like in the example. Technically, we create a new STA object that knows how to register and unregister itself. In addition, it supports two methods, block and unblock. By calling these methods, the client tells the message filter what to do with the problematic calls (those that get into the object while it is handling a previous call - re-entrancy!).

Using the new message filter COM object

Your COM object, that needs the solution of this message filter, can use this message filter COM object in several ways. I chose to hold it as a member. So, my main COM object holds a smart pointer to the message filter COM object as a member:

class ATL_NO_VTABLE CServer : 
    public CComObjectRootEx<CCOMSINGLETHREADMODEL>, 
          public CComCoClass<CSERVER &CLSID_Server,>, 
          public IServer 
{
        .....
private:
    MessageFilterServerLib::IMyMessageFilterPtr m_MessageFilter;
}

Then, in the ctor, initialize this pointer:

CServer::CServer() 
{
    HRESULT hr = m_MessageFilter.CreateInstance(
                 __uuidof(TEAMLAYERLIBLib::TLMessageFilter));
    if(FAILED(hr))
        //do something

    hr = m_MessageFilter->raw_registerMessageFilter();
    if(FAILED(hr)) 
        //do something

    m_MessageFilter->raw_block(); 
    // ***

    //if no need to block anymore - unblock here. 

    //otherwise, keep blocking:

    m_MessageFilter->raw_unblock(); 
}

Now unregister, in the dtor:

CServer::~CServer() 
{
    m_MessageFilter->raw_unregisterMessageFilter();
}

Understanding the code: client side

The main() function creates the server. This way it creates the COM server, which is an STA object, into the main STA. A message filter kicks in only during the cross-apartment COM calls. If you make a COM call to the server within the same apartment - the call will not go through the message filter. Hence, in the tester, you can see that the main thread creates the server. In addition, it creates two STA worker-threads, and these workers make the calls. This way, I generate cross-apartment COM calls, and the message filter is on the air.

The main thread that created the server object has to marshal the interface pointer to the workers. That is done using the CoMarshalInterThreadInterfaceInStream and CoGetInterfaceAndReleaseStream (short names...).

If you look at the worker-threads, you will see that each one of them registers its own message filter as well. The reason for that is each STA thread must register its own filter. In my tester, the worker-threads have to register their message filter, and only then we will see both the client and server side of the filtering (see explanation below). This message filter can be the same as that of the server's, or they might have their own. In my example, this message filter is different from that of the server's. The idea is that a client can have many servers, and each one of them might have its own message filter. Which one should the client use? And another reason: How can the server guess what the client wants to do in case of a rejected call? That's why the client has its own message filter. This message filter is of type class CClientMessageFilter.

"Both client and server sides of filtering": As you can read in the literature, when a message filter's IMessageFilter::HandleInComingCall() returns SERVERCALL_RETRYLATER or SERVERCALL_REJECTED, COM calls the client's IMessageFIlter::RetryRejectedCall(). If you want your client to have a special behavior in these cases, it has to register a message filter itself.

I use a message pump on the server thread (the one that created the component in the first place). Message pump is a must here in this case, and you cannot use an event, for example, WaitForSingleObject(), because the main thread is an STA thread, and an STA thread must not block. When it blocks, it cannot accept the incoming COM calls. That is why you must use a message pump and not a "wait" function.

Important things to know

1. The server doesn't have to implement an IMessageFilter, it only needs to register an object that does so. The former has to be running in the same STA apartment where the latter is registered. There can only be one message filter registered per STA thread, but a single thread can house any number of COM objects.
2. We suppose to have here three COM objects:
   1. The server.
   2. The one that implements IMessageFilter and is registered in the server thread.
   3. The one that implements IMessageFilter and is registered in the calling (client) thread.

   The latter two may be two instances of the same COM object, or may be two distinct independent implementations, as you see fit. The last one, the client's message filter, need not be creatable, and does not need any registration. See in the example code what is sufficient (class CClientMessageFilter).

3. The easiest way to reproduce reentrancy would probably be to create two worker threads and have each of them perform one call (and have the called object put up the message box). One of these calls will nest inside the other. This is what my console-tester does.
4. The implementation of Foo() in the server (the message box), causes a reentrancy of type CALLTYPE_TOPLEVEL. This means that actually everything is OK and the reentrant call is ready to be handled. I wanted to reproduce the problematic case, in which the message filter returns SERVERCALL_RETRYLATER or SERVERCALL_REJECTED. This happens in call type of cases CALLTYPE_TOPLEVEL_CALLPENDING or CALLTYPE_NESTED. In general, CALLTYPE_TOPLEVEL_CALLPENDING only happens when you receive an incoming cross-apartment COM call while an outgoing cross-apartment COM call is in progress. That is, when the server accepting the calls is also making other calls. It does not arrive in all cases where reentrancy is possible. In your application, do not return SERVERCALL_RETRYLATER in case of CALLTYPE_TOPLEVEL, but in this tester, I wanted to demonstrate how it works in case of SERVERCALL_RETRYLATER, so for every X calls I returned this value.
5. If you forget to write in your RGS file the ThreadingModel=Apartment clause, the component is treated as a legacy single-threaded component, which means that all its instances must be created on a single thread - so called main STA thread. When the worker thread tries to create it, the interface needs to be marshaled. But IMessageFilter is unmarshallable, because IMessageFilter hooks into marshalling process. Imagine: you are making a marshaled cross-apartment call. For one reason or the other a message filter needs to be called. But it lives in a different thread, so the call to the message filter itself needs to be marshaled. But this marshalling may result in the necessity to call a message filter, which lives in a different apartment. IMessageFilter is unmarshallable because its methods don't return HRESULT.

Conclusion

I owe my knowledge about message filters to Igor Tandetnik. Without his help, this article would not have been written.