Yet another DLL-wrapper for dynamic loading

• Download demo - 41.7 KB

_{The dynlink project is also hosted at https://code.google.com/p/dynlink/}

Introduction  

Especially when controlling hardware through APIs provided by the vendor, and also when trying to avoid errors during application startup due to certain shared libraries (DLLs) not being available on the system, dynamic loading of such DLLs is a common technique.

The different ways of linking to a DLL from C++ have been described in several articles on this site. For those not entirely familiar with the subject, I recommend the article Step by Step: Calling C++ DLLs from VC++ and VB - Part 4 by Hans Dietrich. Unfortunately, dynamic loading involves a lot of repetitive but error prone coding (see below), and extra care is necessary when keeping the code up to date in case the API of the DLL changes. In order to reduce the amount of necessary coding, DLL-wrappers have been proposed, see e.g., the LateLoad DLL Wrapper. Besides avoiding code duplication, such a wrapper allows for more sophisticated error checking.

However, the function declarations still have to be extracted from the header of the DLL, kept up to date, and the code accessing the API needs to be changed. The so-called delay loading on the other hand does not require any code changes or extra code, but it lacks the flexibility of dynamic loading.

This article proposes a new type of wrapper requiring only minimal code changes to switch between dynamic and static loading but still offering all the features of existing DLL-wrappers and more.

Background

Why use dynamic loading?

Several good reasons why dynamic loading can be useful or even necessary are listed in Hans Dietrich's article and most of them are still valid 7 years later. In addition to the reasons listed there, I have encountered cases where I needed to link to different versions of the DLL in response to some user input or depending on the hardware version I accessed and, believe it or not, where I had to load a DLL twice under different names in order to be able to access two pieces of identical hardware.

Delay loading to the rescue?

Delay loading is also described in detail elsewhere. What it does in essence is that it automates the dynamic loading process by having the linker generate the code necessary. It is, if you wish, a linker generated DLL-wrapper. Indeed, it takes care of some of the headaches of the other two methods: No code changes are necessary as delay loading of a DLL is simply enabled by a linker switch and unlike for static loading, your application will still start even if it links to a library which is not present on the current system or if it imports a symbol which is missing from the available version of the DLL. Existing symbols are loaded at runtime when first accessed from your code.

On the downside, when used without extra error checking, your application crashes as soon as you try to access a symbol that is not present in the loaded DLL (but was part of the import library) or if the DLL cannot be loaded when the first symbol is accessed. To make sure this does not happen, you could call __HrLoadAllImportsForDll (see delayimp.h but careful: LoadLibrary is not case sensitive but this API is.) manually and disable the use of the DLL in case not all symbols could be loaded, or you could try to implement more granular error checking by setting hooks that are called when loading a symbol fails. But this involves calling into APIs much more complex than GetProcAddress() and thus writing (unnecessarily) complex code.

What's wrong with the traditional way?

Nothing. It merely is inelegant, requires a lot of cut and paste and search and replace, and it is error prone, especially when APIs change or the vendor ships headers that are out of sync with the DLL (well, or, if you mix them up). Also, it requires you to change your code when switching from static to dynamic linking.

The demo code includes the code necessary to dynamically load psapi.dll in your application and illustrates this. Ideally, you create one header file, say psapi_dynamic.h that contains the necessary typedefs and declaration of the function pointers. It also contains the declaration of a function that loads the DLL and symbols, load_psapi():

#ifndef _PSAPI_DYNLINK_H_
#define _PSAPI_DYNLINK_H_

#include <psapi.h>

typedef BOOL 
(WINAPI * 
EnumProcesses_t)(
    __out_bcount(cb) DWORD * lpidProcess,
    __in DWORD cb,
    __out LPDWORD lpcbNeeded
    );

typedef BOOL 
(WINAPI * 
EnumProcessModules_t)(
    __in  HANDLE hProcess,
    __out_bcount(cb) HMODULE *lphModule,
    __in  DWORD cb,
    __out LPDWORD lpcbNeeded
    );

...

extern EnumProcesses_t pEnumProcesses;
extern EnumProcessModules_t pEnumProcessModules;
...

// Try to load the psapi.dll 
// returns 
//     0 if library was not found, 
//     1 if all symbols were loaded and 
//     2 if some symbols are NULL 
int psapi_load();

If you are good with regular expressions, this header file can be created from psapi.h in a few minutes. It is a good idea to maintain as much of the formatting and the order of declarations as it is in the original header. This allows you to check and update your typedefs with a good file comparison tool and makes this solution quite maintainable.

In total, each symbol shows up four times in your dynamic loading code: once as a typedef, once in the function pointer declaration, and twice in the corresponding .cpp file, where the function pointer is defined and the symbol is loaded in the load_psapi() function:

#include "stdafx.h"
#include "psapi_dynamic.h"

EnumProcesses_t pEnumProcesses= NULL;
EnumProcessModules_t pEnumProcessModules= NULL;
... 

int load_psapi()
{
    HMODULE hDll = LoadLibrary(_T("psapi.dll"));
    if (! hDll)
        return 0;

    bool b_ok = true;
    b_ok &= NULL != (pEnumProcesses = (EnumProcesses_t) 
                             GetProcAddress(hDll, "EnumProcesses"));
    b_ok &= NULL != (pEnumProcessModules = (EnumProcessModules_t) 
                            GetProcAddress(hDll, "EnumProcessModules"));
    ...

    return b_ok ? 1 : 2;
}

Adding a new / changing the name of a function or removing it therefore requires edits at four different places. It is a good idea to edit the list of typedefs first, the compiler will then remind you of the rest of the work to do.

In your code, you simply replace all function calls in the DLL and prepend the 'p' or whatever prefix you have chosen to give your function pointers.

...
    // Try to load the library
    if (0 == psapi_load())
        return;

    // Check whether the symbol was loaded
    if (! pGetProcessImageFileNameW)
        return;

    TCHAR sName1[1024];
    pGetProcessImageFileNameW(GetCurrentProcess(), sName1, 1024);
...

If we continue even if some symbols were not loaded as in the case above, we better not forget to check whether pGetProcessImageFileW is NULL. While these additional checks and the change of function (pointer) names make the transition to dynamic loading permanent, the method offers many advantages over delay or static loading: Instead of checking documentation and reading out DLL versions in order to find out which functions are safe to call, you can check the availability at runtime and enable or disable the functionality based on the result.

DLL-wrappers

However, if you want to generate meaningful error messages or drop in default functions (stubs) for those functions or variables not loaded, you quickly start writing the same kind of code for each library you load dynamically. Any enhancement to this scheme is then manually added for each library (or, more likely, not) and function and variable stubs require a lot of extra coding.

All this calls for macros and wrappers, and this is exactly what DLL-wrappers do. In the aforementioned LateLoad DLL Wrapper, you would type something like:

#include <psapi.h>
#include "LateLoad.h"

LATELOAD_BEGIN_CLASS(CPsapiWrapper,psapi,FALSE,TRUE)

LATELOAD_FUNC_3(FALSE,
BOOL,
WINAPI,
EnumProcesses,
    __out_bcount(cb) DWORD *,
    __in DWORD,
    __out LPDWORD
    )

LATELOAD_FUNC_4(FALSE,
BOOL,
WINAPI,
EnumProcessModules,
    __in  HANDLE,
    __out_bcount(cb) HMODULE,
    __in  DWORD,
    __out LPDWORD
    )

...

LATELOAD_END_CLASS()

in some header psapi_lateload.h. Note again that we kept as much of the original formatting as possible so we can later check and update the file quickly. Calling into the DLL now happens through the wrapper object:

// Declare the wrapper object
CPsapiWrapper psapi;
...
// Test for presence of a symbol
if (! psapi.Is_EnumProcesses())
    return;

// Call into the dll
WCHAR sName1[1024];
psapi.GetProcessImageFileNameW(GetCurrentProcess(), sName1, 1024);
...

This wrapper adds stubs for every function and the Is_EnumProcesses() checks whether the original function or the stub is in place.

While this is much more convenient to use than simple dynamic loading, it comes with some strings attached. First and foremost, creating psapi_lateload.h is much more work than the psapi_dynamic.h before because the argument names have to be removed. The macro used depends on the number of parameters and has to be chosen by hand. Also, each file in which functions from psapi.dll are used maintains its own wrapper object and as the whole function definition resides in (many versions of) a macro, adding functionality to the wrapper involves much code to be written.

More importantly, again, the code has to be changed when switching between implicit and explicit loading. While this seems to be a minor point, it can be quite a show stopper if, e.g., macros exist that rely on the function names to remain unchanged. As an example, the TCHAR version of the functions in psapi.h can no longer be used which results in cluttering #ifdef UNICODE ... #else ... #endif sections wherever you call a function with string arguments.

The dynlink wrapper

Surely, all the problems mentioned above for dynamic loading with or without existing wrappers are no reason to code a new one.

I could have easily wrapped and updated all DLLs I am likely to load dynamically any time soon in the total time that went into the code presented in this article. But it was certainly more fun to write it than to be writing the same dynamic loading code over and over again.

The dynlink wrapper has the following advantages over existing solutions:

• It requires no change to existing code written for static loading of the DLL. It thus encourages you to switch to dynamic loading allowing you to support, e.g., different versions of a certain SDK. 
• Consequently, it allows switching between static and dynamic linking by changing a single #define. This makes unavailable symbols show up at link time and helps keep your code and the DLL in sync.
• It can be used throughout a large project without any overhead.
• A single search and replace on the header file is sufficient for most APIs. You then maintain a changed version of the original header with minimal changes which is easily updated using a simple interactive file comparison tool.
• It provides extensive error information at runtime. When switching to static linking, all error checks are successful, allowing you to add error checking code even if you switch between loading methods.

Using the code

Using the wrapper involves three steps:

Prepare the modified header

Copy the original header, e.g., psapi.h to psapix.h, add a macro defining the library name, define the default library name using the DYNLINK_LIBNAME  macro (usually, this should be the dll's filename, but it can be anything. It is relevant only for the function dynlink_<DYNLINK_LIBNAME>() returning a reference to the library object and for the default filename used when calling the library object's load() method) and wrap all function and variable declarations with the DYNLINK_DECLARE or DYNLINK_DECLARE_EX macro. This requires similar search and replace skills as using the traditional way. The beginning of psapix.h looks like this:

// DYNLINK: Outside the protection through ifndef
#undef  DYNLINK_LIBNAME
#define DYNLINK_LIBNAME  psapi
#include <dynlink.h>

// DYNLINK: Declarations changed by
// DYNLINK: {.*}\nWINAPI\n{[^\(:b]*}:b*{\(([^;]*\n)*.*\)}; -->
//                DYNLINK_DECLARE(\1, WINAPI, \2, \3)
// DYNLINK: in Visual studio regular expressions

#ifndef _PSAPI_H_
#define _PSAPI_H_

...

DYNLINK_DECLARE(
BOOL,
WINAPI,
EnumProcesses,(
    __out_bcount(cb) DWORD * lpidProcess,
    __in DWORD cb,
    __out LPDWORD lpcbNeeded
    ))

DYNLINK_DECLARE(
BOOL,
WINAPI,
EnumProcessModules, (
    __in  HANDLE hProcess,
    __out_bcount(cb) HMODULE *lphModule,
    __in  DWORD cb,
    __out LPDWORD lpcbNeeded
    ))
...

The regular expressions to change the declarations are given in the comment. If you want to load the library several times under different names, you have to put in a little more work: the DYNLINK_DECLARE macros have to be placed outside the sections protected by #ifndef _PSAPI_H_ (preferably without moving declarations around so we can stay close to the original formatting and header file structure). See psapixx.h for an example.

Link to dynlink.lib

Link to dynlink.lib. Alternatively, you can #define DYNLINK_NO_AUTO (or remove the #pragma comment(lib, "dynlink.lib") from dynlink.h) and add dynlink.cpp to your project.

Add loading and error checking code

In the simplest of all cases, you adjust your include statements and pick one file where the symbols are to be defined. Here you add:

#define DYNLINK_DEFINE 

before including the modified header file. 

In addition, there are several macros which modify the behavior of the wrapper. They are described in dynlink.h and allow customizing the way the DLL is wrapped.

• DYNLINK_USE_STUBS
  Whether or not to replace unavailable functions with stubs. When using this the default return values need to be defined (see below).
• DYNLINK_RETVAL_[some type]The default return value for stubs returning [some type]. This can be any R-value, including a function call. See below for a detailed description how to use this for diagnostic purposes. When using stubs, this macro needs to be defined for all return types unless you use DYNLINK_DECLARE_EX which defines the return value on a per-function basis.
• DYNLINK_LINK_IMPLICIT
  When defined, the dynlink.h header falls back to implicit linking. Of course the library can no longer be loaded under a different name and prefixing does not work any longer. Trying to do this will result in compile time errors.
  
• DYNLINK_PREFIX
  Prefix all functions and variables with the value of this macro, e.g., in order to load the same library several times.
• DYNLINK_NOCHECK
  If this is defined, calling an imported function through dynlink should produce virtually no overhead. However this also disables the auto_load and throw feature (see below)

You can link different libraries with different methods and prefixes (and you can link the same library several times with different prefixes). You must only ensure that implicit or explicit linking is consistently defined for each library and prefix and, if applicable, that the prefix is defined before including the modified header in all files you want to access the library from. For each prefix the DYNLINK_DEFINE macro has to be defined in exactly one file. This is a consequence of how the loading and error checking is accomplished by the dynlink library: 

When not using DYNLINK_LINK_IMPLICIT, all symbols are wrapped into an object derived from CDynlinkSymbol named exactly as the function or variable it wraps. The object can be implicitly cast to the correct type of function pointer or variable reference so no code has to be changed as long as you ensure that either stubs are used or calls are only made of symbols that were loaded.

At the same time, you can of course call the member functions defined in CDynlinkBase and CDynlinkSymbol which are pretty self explanatory: 

class CDynlinkBase
{
public:
    //! Get error during library / symbol load. 
    //! @return 
    //!     The last system error that caused a call to this object to fail
    virtual DWORD get_error() const = 0;
    
    //! Get error during library / symbol load. 
    //! @param s_err
    //!     Error string
    //! @return 
    //!     The last system error that caused a call to this object to fail
    DWORD get_error(std::wstring &s_err) const;
    DWORD get_error(std::string &s_err) const;

    //! True if the symbol/library was successfully loaded with no stub
    //! in place.
    virtual bool is_loaded() const = 0;
    //! True if the symbol/library was successfully loaded, possibly with 
    //! stubs in place
    virtual bool is_loaded_or_stub() const = 0;
...
};

//! A symbol in the dll 
class CDynlinkSymbol : public CDynlinkBase
{
    friend class CDynlinkLib;
public:
    // Constructor. Don't use this unless you know exactly what you are doing. 
    // This is used by the DYNLINK_DECLARE macros
    CDynlinkSymbol(CDynlinkLib &lib, LPCSTR s_name);
...   
    //! Get the symbol name
    LPCSTR get_name()
    {
        return m_s_name.c_str();
    }
    //! Get the library from which we are loaded
    CDynlinkLib * get_lib()
    {
        return m_p_lib;
    }
...
}; 

When calling these directly, the code will stop working when switching back to static loading (because, obviously, functions don't have member functions). If you want your code to remain valid for both methods, use the CDynlinkGetError class to access error and status information:

// Just for convenience
#define _E(symbol) CDynlinkGetError(symbol)
...
// This assumes that psapix.h has been included and that DYNLINK_LIBNAME
// was set to psapi in this file.
dynlink_psapi().load();
...
DWORD dwErr;
std::string s_error;

if (!_E(EnumProcesses).is_loaded())
    _E(EnumProcesses).get_error(s_error);
...

When loading statically, all functions will suggest successful loading. get_name() and get_lib() are not available for static loading.

The code for loading and maintaining the state of the library is contained in an object of type CDynlinkLib. It is accessed through a function dynlink_<DYNLINK_LIBNAME>() where <DYNLINK_LIBNAME> is the library name defined in the modified header (see above).
In the simplest case you call its load() method and that is it but it also allows you to fine tune of the behavior of the librarie's symbols when explicit linking was chosen.

//! Interface class for explicit and implicit linking and loading of dlls
//! and contained symbols.
class CDynlinkLib : public CDynlinkBase
{
    friend class CDynlinkSymbol;
public:
    //! Mode of loading
    enum mode
    {
        //! Explicit loading of function addresses
        ex,
        //! Explicit loading, does not fail when a function symbol is
        //! not found and creates a stub instead that will be called and 
        //! executes the code defined in DYNLINK_RETVAL_<rettype> instead
        exstub,
        //! Loading is assumed to be implicit. Function returns always true
        imp
    };

    //! Create a library interface. Use this only if you know EXACTLY what
    //! you are doing. Usually including the modified header will take care 
    //! of construction and defines a function dynlink_<name>() to access
    //! it as a singleton where <name> is the library name defined in the 
    //! header.
    //! @param sName
    //!     The name of the library. (e.g. psapi for psapi.dll)
    //! @param d_mode
    //!     The mode (see above). If this is 'imp' or 'delay' it is assumed
    //!     that the appropriate linker flags were set.
    CDynlinkLib(LPCWSTR sName, mode d_mode = ex);
    CDynlinkLib(LPCSTR  sName, mode d_mode = ex);
    
    //! Get error during library load
    //! @param s_err
    //!     The string to write a formatted error message to
    //! @return 
    //!     The error as returned by GetLastError() during the load_library call.
    virtual DWORD get_error() const;
    
#ifndef DYNLINK_NO_CHECK        
    //! If set to true accessing a variable or function not loaded 
    //! will cause a c++ exception to be thrown. In fact, the symbol throws a 
    //! pointer to itself so you could wrap your code accessing the lib into
    //! try 
    //! {
    //!     code;
    //! }
    //! catch (CDynlinkSymbol * p)
    //! {
    //!     retrive and display error information;
    //! }
    //! @param d_throw
    //!     When to throw the exception
    //!     0: never throw exceptions (default behaviour)
    //!     1: throw exceptions when the function is not loaded (loadable)
    //!        and no stub is available
    //!     2: throw exception always, even if a stub would be available
    //! 
    //! REMARK: this will also occur if you just check the symbol against 
    //! NULL. If this is enabled use the is_loaded() function instead to 
    //! avoid throwing an exception.
    void set_throw(int d_throw)
    {
        m_d_throw = d_throw;
    }

    //! Change auto load behavior. If enabled, the library will load on first 
    //! access to one of its symbols.
    void set_auto_load(bool b_auto_load)
    {
        m_b_auto_load = b_auto_load;
    }
#endif
    //! Load the library and all symbols. For details see above.
    //! @param b_retry
    //!     Retry the load independently of previous results. If not TRUE 
    //!     a loaded library and its symbols will not be touched.
    //!    @param s_name
    //!     Replace the library name given above.
    //! @return 
    //!     State of the loading, see definitions of DYNLINK_ERR ... above
    int load(LPCWSTR s_name = 0, bool b_retry = false);
    int load(LPCSTR  s_name, bool b_retry = false);
    
    //! Fake-load the library and use only stubs. This will not call 
    //! LoadLibrary() and it will not call GetProcAddress() but all error 
    //! handling will indicate that the library was successfully loaded.
    //! This function has no effect if loading is static.
    int load_stubs();

    //! Get loading state.
    //! @return 
    //!     State of the loading, see definitions of DYNLINK_ERR ... above
    int get_state() const
    {
        return m_d_loaded;
    }
    //! Get the loading mode of the library object. See CDynlinkLib::mode for
    //! details.
    //! @return
    //!     the mode with which the library was loaded.
    mode get_mode() const
    {
        return m_mode;
    }
    //! Check whether the library was loaded successfully.
    //! @return
    //!     true if get_state() is DYNLINK_SUCCESS
    virtual bool is_loaded() const
    {
        return (get_state() == DYNLINK_SUCCESS);
    }

    //! Check whether all symbols can be called in the library without NULLPTR
    //! references.
    //! @return
    //!     true if get_state() is either DYNLINK_SUCCESS or DYNLINK_ERR_STUBS
    virtual bool is_loaded_or_stub() const
    {
        return (get_state() == DYNLINK_SUCCESS) || (get_state() == DYNLINK_ERR_STUBS);
    }
    
    //! Unload the library. This will fail for an implicitely linked library
    //! but should succeed for explicite linkage and delayed loading.
    bool free();

    //! Get the module handle for a successfully loaded library. NULL if the 
    //! library was not loaded.
    //! REMARK: when implicitely linking, this may return NULL even if the 
    //! library was successfully loaded but if the module name does not match
    HMODULE get_module() const
    {
        return m_h_dll;
    }

    //! Get the filename of the library loaded. Can fail for implicitely linked
    //! libs if the library name is not correct
    bool get_filename(std::wstring &s_fn) const;
    //! Get the filename of the library loaded.
    bool get_filename(std::string &s_fn) const;
protected:        
...
};

Again, most of it is self-explanatory. Auto-load on first access and the throwing of exceptions when a symbol that has not been loaded is accessed are enabled or disabled at runtime (set_auto_load() and set_throw()) . The latter feature throws an exception of type CDynlinkSymbol* if an unloaded symbol is accessed and allows you to produce diagnostic output in the catch clause.  

The DynlinkTest application which is part of the demo project demonstrates all options currently implemented. Please refer to test_dynlink.cpp. You can uncomment / comment the #define DYNLINK_LINK_IMPLICIT in stdafx.h in order to switch between dynamic and static loading.

The demo project also contains code for the two other methods presented above and might be a good starting point even if you decide the dynlink wrapper is not for you.

Using Stubs 

A small note on the use of stub functions. The header will define the necessary functions automatically, you only have to make sure that for each return type of a function wrapped in DYNLINK_DECLARE (as opposed to DYNLINK_DECLARE_EX which allows definition of a function specific return type but is somewhat incompatible with the paradigm of minimal change to the header) the appropriate DYNLINK_RETVAL_<type> macro is defined where the header is included with DYNLINK_DEFINE defined.

Importantly, the macro can be any R-value, including a function and whereever the macro is used a variable p_symbol of type CDynlinkSymbol * will be defined. This allows you to produce diagnostic output or to change the state of your program when a stub is called for certain symbol names or return types. See the file main.cpp in the test application for an example how to use this feature.

How it works 

The classes are tiny and their implementation more or less straightforward. While the macro definitions seem convoluted at first, they are quite harmless, too:

Basically, DYNLINK_DECLARE gets the function name, return type, calling convention, and argument list separately. If explicit linking is enabled, it defines (or declares if DYNLINK_DEFINE is not defined) an object of type CDynlinkFunc<T> or CDynlinkVar<T> where the template parameter is the type of the function pointer or the variable, respectively. Both these classes have a cast operator too that allows their use in very much the same way as the original symbol defined in the header:

The variable class CDynlinkVar takes an optional initial value for the stub in the constructor:

//! Class to simulate a variable imported from a dll.
template<class T> class CDynlinkVar : public CDynlinkSymbol
{
public:
// Create and set the stub value to be returned
    CDynlinkVar(CDynlinkLib &lib, LPCSTR s_name, const T&val) : 
                   CDynlinkSymbol(CCDynlinkLib &lib, LPCSTR s_name)
    {
        m_stub_value = val;
        m_p_stub = (void *) &(m_stub_value);
        m_p_ptr = NULL;
    }
    // Create without stub
    CDynlinkVar() : CDynlinkSymbol(CCDynlinkLib &lib, LPCSTR s_name)
    {
        m_p_stub = NULL;
        m_p_ptr  = NULL;
    }
    // Use as variable of type T
    operator T&()    
    { 
#ifndef DYNLINK_NO_THROW
        check_throw();
#endif
        return *(T*) m_p_ptr; 
    }
protected:
    // The thing 
    T m_stub_value;
};

and the function class takes an optional function pointer to use as a stub:

// A function. T is the type of the function Pointer
template<class T> class CDynlinkFunc : public CDynlinkSymbol
{
public:
    // Create and set the stub value to be returned
    CDynlinkFunc(CDynlinkLib &lib, LPCSTR s_name, 
                 T stub = NULL) : CDynlinkSymbol(lib, s_name)
    {
        m_p_stub = (void *) stub;
        m_p_ptr = NULL;
    }
    // Use as variable of type T
    operator T()    
    { 
#ifndef DYNLINK_NO_THROW
        check_throw();
#endif
        return (T) m_p_ptr; 
    }
};

The design is more complex than the one member function per symbol design of the LateLoad wrapper. This is mainly due to my wish to allow declarations scattered over one or several header files which forces us to wrap each symbol definition into an object definition which adds it to the CDynlinkLib object during construction. The approach has the advantage that the library object can, e.g., loop over its symbols at runtime and that functionality can be added both to the symbol base class and to the library object without changing anything about the more error prone macro definitions.

History 

• 07/12/2011: Initial version.  
• 03/20/2011: Added auto loading, cleaned up the design. Updated demo application.