Introduction
This article describes a useful technique for using a closed source C++ DLL, loaded at run-time, to access an API for a popular consumer peripheral. It’s assumed the developer does not have access to import libraries or source code.
Background
This is a technical follow-up to my Continuum blog post describing how I automated a simple software task to enable an automatic pan-and-zoom feature of a consumer webcam that didn't provide an API. We needed to do this without disturbing the user experience, so standard program automation tools like AutoIt were off the table.
Throughout the process, I realized some of the pitfalls in making calls on C++ DLLs and identified workarounds. The inspiration for this article was a blog post from Recx Ltd, Working with C++ DLL Exports without Source or Headers. An example project is provided, but our specific case relied on a proprietary DLL that could not be included due to licensing.
API Monitor is an invaluable tool for monitoring the activity of API calls. It has more comprehensive features for Windows API calls,
but it can also be used to view calls to external DLLs. You can spawn processes
from within the tool or attach to a running process. The state of the call
stack can be seen before and after API calls are made, return values are
visible, and it even provides breakpoint functionality.
With API Monitor, I spawned an instance of the web cam
controller application, which allowed me to see which DLLs were being loaded. Monitoring
from process execution (as opposed to attaching to running process) can be
important when trying to see initialization behavior.
The module dependency view revealed a large list of DLLs.
Most of them were Windows system DLLs or related to the Qt framework. One in
particular stood out for what we needed: CameraControls_Core.dll. I set up the API Monitor to log all calls to this DLL and this was the relevant output:
LWS::CameraControls::CameraControls_Core::Init ( ... )
LWS::CameraControls::CameraControls_Core::GetCurrentVideoDevice ( ... )
LWS::CameraControls::CameraControls_Core::SetCurrentVideoDevice ( ... )
LWS::CameraControls::CameraControls_Core::GetFaceTracking ( ... )
Monitoring the API activity live, I checked and unchecked
the facial recognition box. I noticed calls to SetFaceTracking () were being
made. A quick look at the call stack revealed Boolean values being sent as parameters to the method. I used the Microsoft dumpbin utility on
CameraControls_Core.dll to view the exported method list. It was quickly evident that I was dealing with a C++ DLL (due to the use of name decoration).
The method names were like so:
1 0 00001366 ??0CameraControls_Core@CameraControls@LWS@@QAE@XZ
2 1 000015C3 ??1CameraControls_Core@CameraControls@LWS@@UAE@XZ
3 2 00038740 ??_7CameraControls_Core@CameraControls@LWS@@6B@
I ran the output of the dumpbin tool into Microsoft’s undname utility to undecorate the names.
This provided a more coherent output:
1 0 00001366 public: __thiscall LWS::CameraControls::CameraControls_Core::CameraControls_Core(void)
2 1 000015C3 public: virtual __thiscall LWS::CameraControls::CameraControls_Core::~CameraControls_Core(void)
3 2 00038740 const LWS::CameraControls::CameraControls_Core::`vftable'
...
40 27 000012E9 public: bool __thiscall LWS::CameraControls::CameraControls_Core::Init(class QString)
...
86 55 000016D1 public: long __thiscall LWS::CameraControls::CameraControls_Core::SetFaceTracking(unsigned long,long)
It was around this point in time that I stumbled on the above mentioned Recx Ltd article. I realized that our particular DLL would present a few challenges
that made a simple script-kiddie application of this technique impossible. I moved a copy of the DLL into my application’s working directory and attempted
to write some code to load it dynamically.
static HINSTANCE CameraControlDllHandle = NULL;
CameraControlDllHandle = LoadLibrary(L"CameraControls_Core.dll");
Stepping through the code revealed the first dependency library of many: Qt4Gui4. As the errors presented themselves, I copied the dependencies into the working directory. It turns out ten additional DLLs
were required. I ran through the code one last time, and got an error about MSVCR90.DLL being missing.
Placing this DLL in the directory results in an error about the C-runtime being loaded improperly.
MSVCR90.DLL is the C-runtime library for Visual C++ 2008. I tried
re-building my project to run with this runtime, so it would use the same
runtime as the DLL, but it didn't mitigate the problem. As it turns out, Microsoft introduced a new form of system-wide DLL
management in Windows 98 for allowing conflicting DLLs to exist simultaneously
in memory (Wikipedia DLL Hell). DLL’s complying with this standard have a
manifest file that is imported into the DLL to inform a calling process
which dependencies need to be loaded.
To overcome the C run-time dependency, I needed to create a manifest file pointing to the run-time library and embed it into my
CameraControls_Core.dll. This MSDN article outlines how to use Microsoft’s
MT.EXE utility to embed a manifest file into an already built executable or
DLL.
The manifest file I needed to use for MSVCR90.DLL is as follows:
="1.0"="UTF-8"="yes"
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
<dependency>
<dependentAssembly>
<assemblyIdentity type="win32" name="Microsoft.VC90.DebugCRT"
version="9.0.21022.8" processorArchitecture="x86"
publicKeyToken="1fc8b3b9a1e18e3b"></assemblyIdentity>
</dependentAssembly>
</dependency>
<dependency>
<dependentAssembly>
<assemblyIdentity type="win32" name="Microsoft.VC90.CRT"
version="9.0.21022.8" processorArchitecture="x86"
publicKeyToken="1fc8b3b9a1e18e3b"></assemblyIdentity>
</dependentAssembly>
</dependency>
</assembly>
After this step, the library was loading into memory without
exceptions! Now, I just needed to verify method calls would work. Early attempts at calling essentially any of the methods were throwing
exceptions. I realized this was because some critical init function was
probably not being called. The reader should read through Working with C++ DLL Exports without Source or Headers first.
I’ll jump straight to the working solution and explain it:
#include <QString>
typedef long (__thiscall *_SetFaceTracking)(DWORD *pThis, unsigned long, long boolValue);
typedef long (__thiscall *_GetFaceTracking)(DWORD *pThis, unsigned long, long *boolValue);
typedef void (__thiscall *_CameraControls_Core)(DWORD *pThis);
typedef bool (__thiscall *_Init)(DWORD *pThis, class QString param1);
typedef long (__thiscall *_GetCurrentVideoDevice)(DWORD *pThis, unsigned long *param);
typedef long (__thiscall *_SetCurrentVideoDevice)(DWORD *pThis, unsigned long param);
static BOOL freeResult;
static _SetFaceTracking CameraSetFaceTracking;
static _GetFaceTracking CameraGetFaceTracking;
static DWORD dwFakeObject[512];
static HINSTANCE CameraControlDllHandle = NULL;
static unsigned long currentVideoDevice = 0;
PRINCEDLL_API HRESULT APIENTRY InitLibrary()
{
long retValue = 0;
QString inputString = "";
CameraControlDllHandle = LoadLibrary(L"CameraControls_Core.dll");
if (CameraControlDllHandle == NULL)
return E_FAIL;
_CameraControls_Core CameraImportedConstructor = (_CameraControls_Core) GetProcAddress(
CameraControlDllHandle,"??0CameraControls_Core@CameraControls@LWS@@QAE@XZ");
if (CameraImportedConstructor == NULL)
return E_FAIL;
CameraSetFaceTracking = (_SetFaceTracking) GetProcAddress (CameraControlDllHandle,
"?SetFaceTracking@CameraControls_Core@CameraControls@LWS@@QAEJKJ@Z");
if (CameraSetFaceTracking == NULL)
return E_FAIL;
CameraGetFaceTracking = (_GetFaceTracking) GetProcAddress (CameraControlDllHandle,
"?GetFaceTracking@CameraControls_Core@CameraControls@LWS@@QAEJKPAJ@Z");
if (CameraGetFaceTracking == NULL)
return E_FAIL;
_Init CameraInitializer = (_Init) GetProcAddress (CameraControlDllHandle,
"?Init@CameraControls_Core@CameraControls@LWS@@QAE_NVQString@@@Z");
if (CameraInitializer == NULL)
return E_FAIL;
_SetCurrentVideoDevice CameraSetCurrentVideoDevice = (_SetCurrentVideoDevice) GetProcAddress (
CameraControlDllHandle, "?SetCurrentVideoDevice@CameraControls_Core@CameraControls@LWS@@QAEJK@Z");
if (CameraSetCurrentVideoDevice == NULL)
return E_FAIL;
_GetCurrentVideoDevice CameraGetCurrentVidDevice = (_GetCurrentVideoDevice) GetProcAddress (
CameraControlDllHandle, "?GetCurrentVideoDevice@CameraControls_Core@CameraControls@LWS@@QAEJPAK@Z");
if (CameraGetCurrentVidDevice == NULL)
return E_FAIL;
memset (dwFakeObject, 0x00, 512);
CameraImportedConstructor(dwFakeObject);
retValue = CameraInitializer(dwFakeObject, inputString);
retValue = CameraGetCurrentVidDevice (dwFakeObject, ¤tVideoDevice);
retValue = CameraSetCurrentVideoDevice (dwFakeObject, currentVideoDevice);
return S_OK;
} A function or method's calling convention determines how information
is passed and returned from the caller to the callee. The calling convention
our DLL used differed from those presented in the above
article. Their example used __cdecl,
which I just learned is now a standard calling convention for all functions on
Win x64 systems ( this was done to eliminate the problem of so many complicated calling conventions).
In __cdecl,
all the parameters are passed on the call stack. The object instance this pointer is passed last. Our DLL used the
__thiscall convention. __thiscall,
it turns out, informs the callee that the this pointer is being passed via the ECX register (not on the call stack). The __thiscall keyword tells the compiler that the first parameter should be placed in the
ECX register. Technically, the first parameter should be a pointer to the object instance.
Since our DLLs
Init() function requires a QString as a
parameter, I needed to determine the version of Qt being used by the target DLL, build Qt, and statically link it to my DLL. Fortunately, passing an empty
QString to the Init function completed initialization without errors and
allowed the other methods to be called. The dwFakeObject array is required
because C++ instance methods expect to be passed a reference to the object they are designed to work on. We reserve an area in memory and treat this as a
reference to a dummy object that is passed to the methods within the DLL.
Points of Interest
Hopefully this will prove insightful to someone going
through a similar situation with attempting to use third party DLLs. Without
access to source for the definitions of the underlying interface and data
types, the problem can be more or less complex than this example. However, it
serves to illustrate that there isn’t always a general purpose solution for accessing DLL functions and some cases will require an ad-hoc approach.
Topics touched on:
- Embedded DLL Manifest files
- Viewing export functions and making calls from C++ DLLs
- APIMonitor as a tool for spying on API calls
- Calling conventions
History
17 Jan 2013 - Initial
publication.
