Introducing Comet

Introduction

Comet is an Open Source template library written by Sofus Mortensen with help from Paul Hollingsworth, Mikael Lindgren and msyelf, that provides an alternative and more complete set of STL-like COM resource wrappers as well as an interface wrapper generator for implementation and use of COM libraries.

Current State of Play without Comet

COM Memory Resource Management

My work involves dealing with a large amount of COM, ActiveX and OLE controls. Some of these are written in MFC some in ATL, most use MIDL generated headers, some use #import and some use Comet. In addition to getting the semantics of the controls correct, one of the biggest problems I encounter is handing COM resources; making sure that strings, safearrays and variants are allocated and freed, and making sure that reference-counting is handled properly.

If you know all the rules, references and memory management in COM isn't that difficult as long as you are paying attention. It seems however, that many COM programmers either aren't paying attention or are caught up in working out what the program should actually do (the naive fools)! As a result, it would appear to be quite difficult to code and maintain COM code without bodging memory management.

Microsoft COM Wrappers

The two (yes, two) sets of wrapper classes Microsoft provide to facilitate memory handling, both have major flaws in them (some of them are the same flaws), that provide as many pitfalls as they fix up. I'm pretty sure I know them all, well at least I thought I did until I found another one the other day which involved a mismatch between [out] and [in,out] parameters and the Microsoft CComVariant class.

There are two great examples of how badly behaved these classes are. The first is the overloaded operator &, which is the work of the devil, and should be expunged. It is used to return a pointer to the internal raw COM type, and makes some function-calls look really neat.

    CComPtr<IDoSomething> pDoSomething;
    HRESULT hr = pDoSomething.CoCreateInstance( CLSID_CoDoSomething );
    if( FAILED(hr) ) return hr;
    CComBSTR mystring;
    hr = pDoSomething->DoIt(&mystring);
    if( FAILED(hr) ) return hr;

Unfortunately it means that:

1. you can't (easily) pass a pointer to the object into a function call
2. you have to use the nasty CAdapt class in order to contain it inside a templated collection
3. the ampersand operator can only really cover one of two possible uses for the returned pointer.

Not being able to pass a pointer is just a pain, you can use references, but they aren't always appropriate.

Having to use the CAdapt class seems simply annoying, however it is another thing to know, and which the novice user doesn't think to look up in the inadequate documentation. This often results in a temptation to use raw COM pointers in the container (which in my view is a cardinal sin of C++).

The ampersand operator on all of the Microsoft COM wrapper classes is half designed to provide [out] pointer support. An assert in debug modes makes sure that you do not pass in a non-null pointer, which cuts out passing in by reference (the default in VB), but if you compile without debug, you can easily miss the fact that you can leak memory by not releasing the contained resource and passing it into an [out] only parameter!

Given the raw nature of the COM wrappers, a few type-safe methods for common method/functions such as QueryInterface and CreateInstance are essential, and should always be used over their raw counterparts. Unfortunately, they require the use of the Microsoft compiler macro __uuidof() which allows access to the __declspec(uuid()) definition output by the MIDL compiler, however their benefits far outweigh any qualms I would otherwise have in using them.

Unfortunately with the _com_ptr implementation, the Microsoft team went a step further and decided to use the VB model of pointers, which is that assignments between two different _com_ptr types always and without warning causes a QueryInterface. This happens even when the pointers are assignment compatible! Not always what one wants given the tendency for programmers to miss out base classes in their ATL interface maps.

Directions in C++ Programming

Well-behaved Classes & STL

STL (Standard Template Library) provides some excellent models for C++ class usage. Careful definitions of containers, algorithms within the framework of STL Concepts (the template equivalent of an interface) make for efficient and versatile code generation. While STL may not, at the moment, define a complete set of algorithms and container types, it does provide a methodology for implementing efficient templated code that is reflected in the great work done in code libraries like Boost that extends containers to include large-strings (ropes), introduces the concept of graphs and adds significantly to many other facets of C++ programming.

Exception safety, consistent interfaces and an ease of use that makes it hard for the user to mis-manage resources are all part of what make a well-behaved class/class library.

Maintainability

Using wrapper classes to manage resources is an essential part of modern C++ programming. Some of the more obtrusive programming disciplines can be alleviated by well-designed resource wrappers. This should enhance both readability and, more importantly, the maintainability of code, remembering that the next person along may not be as careful about resources!

A good modern compiler should optimize out most of a well-designed resource-wrapper, and the overhead of exception handling is, in my opinion, well worth its weight in reduced maintenance costs.

Exceptions for Error Management

There is a definite choice to be made when it comes to how to handle errors. The choice can be made to handle errors via enums and strings appropriate to each function that can return an error, or to have an error class as the return value, or to use exceptions.

Having ad-hoc enums to handle errors will soon become very boring, with the programming overhead required to translate between different errors. Error classes work well, if the choice not to use exceptions has been made, but it means that any other return values must be passed back as parameters, and it means that you have to assign and check errors in code all the time, and if you forget, it isn't obvious, leading to "was that error ignored on purpose by accident?" questions down the road.

The disadvantage of exceptions is that you need to enable exception handling in the compiler with the penalty of the overhead in creating bunches of extra stack frames, and all code where exceptions can be thrown must be written in an exception safe manner. The advantages, however, are:

1. you can catch various types of errors (you aren't restricted to a single error class),
2. different levels of code can catch different exceptions,
3. you don't have to worry about checking errors unless you want to ignore or handle a particular case,
4. code is generally more readable (the 'return' value is reclaimed and the need for checking error states is reduced),
5. exception-safe coding is more maintainable (return-anywhere safe as well) by more people, and
6. much of the work that may be required in using API functions in an exception-safe manner can be hidden by good programmers in good class wrappers to be used by all.

I had already made a decision a couple of years BC (Before Comet) that STL had the right end of the stick, and that exceptions were the way to go despite the overhead, even having worked with a project that had a very well designed error return class for a while. So that when I saw Comet even in it's alpha days and compared it against #import, I was won over fairly quickly. Since that time, I have seen (and helped) Comet do more and more amazing things.

The Comet Solution

Comet Wrappers

There are two parts to comet, the first being the Comet template library which I will begin with. These wrappers do use exceptions as a way of handling errors (exceptional circumstances), the merits of which I will argue later. It is easy to forget the fact that the second set of Microsoft wrappers also use exceptions extensively wherever a method does not return a HRESULT. Comet has its own error class that inherits nicely of std::exception to provide STL compatible exceptions.

The basic rule of Comet is that no action should be implicit (like QueryInterface and casting), but that explicit operations are as unobtrusive and as understandable as possible. For example:

    com_ptr<IViewObject> viewobj = com_cast( obj ); // <-- non-throwing 

                                                   //query-interface

    if (viewobj.is_null())
    {
     com_ptr<IViewObject2> viewobj2 = try_cast( obj ); // <-- throwing 

                                                      // query-interface

     viewobj = viewobj2; // <-- Assignment between 

                        // assignment compatible types.

    }

This shows three different ways of assigning. The first uses com_cast that causes the assignment to do a QueryInterface, but does not throw an error. The second uses try_cast that again causes the assignment to do a QueryInterface, but causes errors to be thrown. The third assignment is for assignment compatible objects only, and will cause a compiler-error unless (in this case) IViewObject2 inherits off IViewObject.

Comet also applies the simple but explicit rule to getting at raw COM pointers, which it is otherwise very wary about handing out. There is no overloaded operator&, but in their stead, a number of methods that are consistent across classes (effectively providing a concept of OLE COM type wrappers). These are:

• in(): Returns a pointer suitable for [in] parameters,
• out(): Returns a pointer suitable for [out] parameters, first freeing memory allocated,
• inout(): Returns a pointer suitable for [in,out] parameters and
• get(): The same as in(), but using the STL style of pointer accessor.

The best part about these is that in a full Comet project, even these are only required when interfacing with standard OLE interfaces.

Making use of the type-library

The other side to Comet is the ability to generate both interface and implementation wrappers from a type library. The wrappers completely wrap the interfaces and coclasses defined by the TypeLibrary, and provide a proper C++ like interface to them, with wrappers for all of the standard OLE types, as well as some support for structures and other more advanced concepts.

The interface wrappers provide a client wrapper for the interface, using the comet type wrappers, with [out,retval] properties returned, and with failed HRESULTs thrown as exceptions (with IErrorInfo support also).

The implementation wrappers also provide an efficient translation layer so that libraries, coclasses and interfaces can be implemented in a more C++ style, without the programmer having to deal with any nasty raw COM types (unless they really want to), with exceptions being turned into HRESULTs and associated ErrorInfo. (ISupportErrorInfo is supported by default on the standard coclass implementation).

Comet makes use of as much information from the type-library as it can, doing away with the need for nasty macro maps that are required by ATL and MFC in order to specify which coclasses are implemented by the library (these are mentioned in the type-library), and which interfaces might be expected on a coclass (again, mentioned in the type-library). Of course, the defaults can be overridden, but again, no pre-processor macros are required.

The effect is that a full library implementation includes an IDL file, an RC and project file and then an implementation file looking something like this:

#include "MyTypelibLib.h"


using namespace comet;
typedef com_server<MyTypelibLib::type_library > SERVER;
// Declare the server wrapper

COMET_DECLARE_DLL_FUNCTIONS(SERVER)
using namespace MyTypelibLib;

template<>
class coclass_implementation<MyCoclass> : public coclass<MyCoclass>
{
public:
    // IMyInterface implementation

    bstr_t GetName()
    {
        return m_name;
    }
    void PutName( const bstr_t &newName)
    {
        m_name = newName;
    }
};

This is not an exaggeration, this is the complete file. The best part is that using the coclass and interface is just as simple as a process:

#include "MyTypelibLib.h"

using namespace comet;
using namespace MyTypelibLib;
// ...


std::wstring someFunction()
{
    com_ptr<IMyInterface> obj = MyCoclass::create();
    obj->PutName(L"Test");

    return obj->GetName();
}

I would have added a comparison ATL implementation, but after adding all the necessary maps and extra bits required just to implement a coclass, it would have been just too big. (That wouldn't be including the RGS file either).

Default and optional parameters

Methods that have optional parameters with default values are supported for most types, and are not restricted to VARIANT (as with #import). The comet variant_t type explicitly supports missing parameters as well.

Dispatch interface Support

Calling and implementation of IDispatch-only interfaces (dispinterface) and source interfaces are indistinguishable from proper v-table interfaces.

In place of the pre-processor, Comet makes full use of the power of templates as a type-safe way of driving the compiler to produce efficient code. While ATL does use templates (hence the name), it only uses a couple of techniques and uses the bulk of work to user-maintained preprocessor maps.

Connection Points

Connection points are handled particularly poorly by the standard Microsoft ATL wizards, with no support for IDispatch arguments by reference and an uninitialized variable in the v-table implementation.

The Comet generator provides implementations for any connection points mentioned in the type-library, with support for handling errors in a user defined manner. A check for NULL v-table entries is also useful when v-table connection-points are being sunk by VB applications. (VB leaves a NULL in the v-table in place of unimplemented �events�, thank you very much Microsoft).

Firing connection-points is quite simple. If only a single connection-point is provided, the code looks something like this:

connection_point.Fire_RaiseEvent(L"Argument");

and if there are multiple connection points, a scope qualifier provides easily readable access to the event like this:

connection_point_for<IMyEvent>::connection_point.Fire_RaiseEvent(L"Argument");

Struct Wrapper Support

Comet even includes smart wrappers for structs! Some neat tricks are used to allow the structs to appear to the implementor and the user of the structs with all the proper comet type wrappers.

Safearrays

One more gaping hole that was left by Microsoft, especially in the light of an exception based library (the second set of COM wrappers), is the lack of any SAFEARRAY wrappers. Comet has mostly filled the hole by providing proper support for SAFEARRAY vectors (not multidimensional arrays) that comply with the STL random-access container concept.

Comet safearray_t<> supports most types including enumerated types and custom interfaces, doing run-time checking when attaching the SAFEARRAY to make sure they are the correct type. You can even specify safearrays in an interface using the SAFEARRAY( type) in the IDL, and comet will wrap them for you.

Here is an example of how to create and initialize a safearray vector:

    safearray_t<long> longarray(10,0); // 10 items, 0-based vector of longs

    long initvar=100;
    for (safearray_t<long>::iterator it= longarray.begin(); 
                   it != longarray.end(); ++it, ++initvar)
        (*it) = initvar;

Summary

Under this barrage of features, the Microsoft-specific #import seems woefully inadequate. In fact, its inability to handle IPictureDisp interfaces correctly makes me question how much use the Microsoft team have put it to! I have also known it to produce code that doesn't compile, with enumerated types defined after they are used in the interfaces (even though they appear before in the type-library and IDL files).

Ok, so that was a bit harsh. I'm sure Comet will have some similar weirdness, but these will get fixed rather than being added to the feature list.

ATL Compatibility

ATL has much of the support required to be able to build ActiveX/OLE controls and containers, and Comet is not ready to attempt replicating this work, however Comet is quite happy to work along-side ATL.

Selected interfaces (including dispatch interfaces) can be implemented inheriting from the Comet generated interface wrappers and implementing the methods in the usual Comet style.

The suspect ATL implementations of connection points can also be easily replaced by the more robust Comet versions.

There is also support for explicitly adding ATL-style coclass definitions to a Comet project, so you can work it either way!

Fit the modern C++ paradigm

One of the main aims of Comet is to give the ability to the programmer to create COM objects without losing the robustness of modern C++ programming styles.

The programmer is then left to concentrate on design, rather than on the tedious task of COM resource management with either one of two sets of COM resource wrappers that require some knowledge and/or tedious reference to barely sufficient documentation to get correct.

Comet gives back C++ programmers using COM, an exception-safe programming environment using STL and STL-style paradigms.

Type-safe Meta-programming: making the compiler work for you

Well implemented templated libraries should take away much of the difficult work of creating efficient compiled code.

By utilizing the template engine as an evaluation engine operating on the humble 'type', the compiler can be made to make compile-time decisions on implementation, based on information gleaned from the template argument types.

While this can lead to some rather dense template code, it can free the programmer by transferring some of the more esoteric domain knowledge decisions to the library. It also allows decisions on efficiency to be made transparently, without the programmer having to consciously choose which templated class to use.

Exception-safe programming?

Exception handling has a price, and the real question is: what is it that comes in the Exception-handling package?

The whole idea of exception handling is that we are provided with a mechanism for coping with exceptional circumstances. It frees the programmer from having to write in checks for critical errors for every function call, it gives the programmer back C++ operator overloading in areas where exceptional circumstances may arise, and it also gives us back being able to read the code we write!

Writing good, maintainable code unquestionably takes discipline, and these are often encoded into coding standards. Different programming languages require different standards, including C & C++.

It is also worth extending this to different environments within C++. Raw COM programming requires a set of coding standards or disciplines, as does coding in a non-exception handling environment. MFC COM also has its own set of disciplines, as does STL. Coding with exceptions naturally has its own set of disciplines.

The discipline of coding with exceptions is usually called �Exception-Safe Programming�, and I believe it to be far less of a burden than the C-style discipline that should be applied to coding when all errors are passed back as classes or error codes.

The real cost of exceptions is, of course, the extra stack-frames required, and the exception state checking required. I believe, it more than pays for itself in reduced implementation and maintenance costs.

Two cautionary notes on Microsoft implementations of exceptions though:

• Firstly, Microsoft has gone against the standard and allowed system exceptions to be caught by catch(...). This is always going to cause problems, as the compiler would need to assume that every operation could cause an exception! The effect is that the exception may bypass some of the destructors up the exception tree. This could mean a few memory leaks... no big deal... or it could cause resource locks not to be released, causing a multi-threaded environment to seize up.
• Secondly, there is a nasty bug lurking in MSVC6 that is exclusive to catching rethrown exceptions. The effect is that an object that is thrown as an exception is deleted twice! Fortunately, the bug seemed to have been fixed by VC++.NET. The briefest version of the offending code looks something like:

  try { throw std::runtime_error("Error"); }
      catch (...){
          try { throw; }
          catch( std::exception ) {}
      }

The easiest rule to follow to prevent such problems is: if you catch something and you don't know what it is... make sure you throw it back.

Unimplemented Stuff that we know about

There is still a list of things that we would like to enhance/implement in Comet, but are awaiting some of the support and user-base necessary to do so.

• More extensive documentation
• COM Categories
• Extend the windowing and control support.
• Support for merging proxy-stubs
• Tighter threading support for NewEnum

We have some preliminary ideas for Categories, including some pre-processing that will make up for the lack of support for them in the Microsoft IDL files, however these await further development resources, and some specific knowledge in the area.

Wrapping-Up

The biggest argument I get against using Comet in a project is the �unknown� factor. The company can hire people who know ATL or MFC, but that it is much harder to find people who know Comet. My response is always along the lines of, �The problem is that even the people who claim to know these environments still get it wrong, and anyway, it is easy to find people who know STL.�.

It is really surprising difficult to get COM resource management entirely correct. There are many pitfalls for the unwary, or the tired, or the lazy, or those under pressure to deliver. Programmer who claim that they have never fallen into at least two of those categories in the last month are probably lying, or kidding themselves.

ATL, MFC and #import make some things easier, but it is still too easy to do resource management incorrectly. Comet provides resource management for all OLE automation datatypes, structs and more, and not only makes it easy to do resource management correctly, it makes it hard to do it incorrectly. It also goes further than #import in allowing servers to be implemented the same way as they are called, making default handling of exceptions trivial.

The bottom line is that Comet makes COM programming easy, and maintainable. It is definitely worth a go.

The current Comet documentation and source can be found on the Comet website.