Introduction
The following article is my response to "John" comment on my other post about native C++/C# interop using C++/CLI.
The initial request was rather simple: for whatever reason John wanted to use the C++/CLI wrapper from Python and not from native C++.
It seemed technically feasible because Python has a remarkable tool to interact with native code: the ctypes module.
The only issue is that ctypes only supports C interfaces not C++ classes so in this case it can’t directly use the YahooAPIWrapper class.
In fact it’s a minor issue because this kind of situation is well known and a well documented pattern exists to circumvent it: building a C wrapper around the C++ API.
This looks a little crazy because you now have 2 layers between the Python client code and the C# Yahoo API:
Python -> C wrapper -> C++/CLI wrapper -> C# API
So, while I don’t think this layering could have any usefulness in real-life, this was a challenging and interesting question.
Looks simple no? Well, as you know when you start to pile up heterogeneous layers unexpected issues can appear and this is exactly what happened there, and it has revealed one that is worth talking about.
So keep reading!
A naive implementation
I first started with this simple C wrapper:
extern "C"
{
__declspec(dllexport) void* YahooAPIWrapper_New()
{
return new(std::nothrow) YahooAPIWrapper();
}
__declspec(dllexport) void YahooAPIWrapper_Delete(void* wrapper)
{
delete wrapper;
}
__declspec(dllexport) double YahooAPIWrapper_GetBid(void* wrapper, const char* symbol)
{
return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetBid(symbol);
}
__declspec(dllexport) double YahooAPIWrapper_GetAsk(void* wrapper, const char* symbol)
{
return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetAsk(symbol);
}
__declspec(dllexport) const char* YahooAPIWrapper_GetCapitalization(void* wrapper, const char* symbol)
{
return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetCapitalization(symbol);
}
__declspec(dllexport) const char** YahooAPIWrapper_GetValues(void* wrapper, const char* symbol, const char* fields)
{
return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetValues(symbol, fields);
}
}
Nothing special: it just forwards the calls to the C++ class, including one function for creating a new instance, YahooAPIWrapper_New, and another for deleting an existing instance, YahooAPIWrapper_Delete.
The Python script
Here is the Python script, "test.py", that mimics the behavior of the native C++ program from the original article:
import ctypes
YahooAPIWrapper = ctypes.CDLL('YahooAPIWrapper.dll')
YahooAPIWrapper_New = YahooAPIWrapper['YahooAPIWrapper_New']
YahooAPIWrapper_Delete = YahooAPIWrapper['YahooAPIWrapper_Delete']
YahooAPIWrapper_GetBid = YahooAPIWrapper['YahooAPIWrapper_GetBid']
YahooAPIWrapper_GetBid.restype = ctypes.c_double
YahooAPIWrapper_GetAsk = YahooAPIWrapper['YahooAPIWrapper_GetAsk']
YahooAPIWrapper_GetAsk.restype = ctypes.c_double
YahooAPIWrapper_GetCapitalization = YahooAPIWrapper['YahooAPIWrapper_GetCapitalization']
YahooAPIWrapper_GetCapitalization.restype = ctypes.c_char_p
YahooAPIWrapper_GetValues = YahooAPIWrapper['YahooAPIWrapper_GetValues']
YahooAPIWrapper_GetValues.restype = ctypes.POINTER(ctypes.c_char_p)
wrapper = YahooAPIWrapper_New()
stock = ctypes.c_char_p(b"GOOG");
bid = YahooAPIWrapper_GetBid(wrapper, stock)
ask = YahooAPIWrapper_GetAsk(wrapper, stock)
capi = YahooAPIWrapper_GetCapitalization(wrapper, stock)
bidAskCapi = YahooAPIWrapper_GetValues(wrapper, stock, ctypes.c_char_p(b"b3b2j1"));
print("Bid: " + str(bid))
print("Ask: " + str(ask))
print("Capi: " + capi.decode())
print("BidAskCapi[0]: " + bidAskCapi[0].decode())
print("BidAskCapi[1]: " + bidAskCapi[1].decode())
print("BidAskCapi[2]: " + bidAskCapi[2].decode())
YahooAPIWrapper_Delete(wrapper)
If you’re not familiar with ctypes here is what it does:
- load the DLL, and specify it uses the cdecl calling convention (it’s the default for C++)
- get a proxy for each of the exported functions, and specify their return types: this is important because by default ctypes considers the return type as int
- call the functions taking care of marshalling and unmarshalling the strings
There is not a lot of plumbing as ctypes manages all the hard work for us.
The issue: assemblies probing
But running the script for the first time gave:
>python.exe test.py
Traceback (most recent call last):
File "test.py", line 21, in <module>
wrapper = YahooAPIWrapper_New()
OSError: [WinError -532462766] Windows Error 0x%X
</module>
Ouch! Without more information it’s hard to find the source…
But by playing with the code I’ve found that the issue was not related to Python or C++/CLI but to the C# library, more exactly to the way it is probed by .Net.
Indeed, remember "YahooAPI.dll" is a .Net assembly, and .Net does not load all the dependencies at startup but loads them on the fly, only when it encounters a type which is in an assembly not already loaded.
And as in the YahooAPIWrapper constructor we need the YahooAPI type, the first time it encounters it, the .Net runtime, the CLR, searches the assembly it knows contains the YahooAPI type (thanks to the metadata emitted during the compilation): "YahooAPI.dll".
It first looks into the GAC, doesn’t find it, then looks in the directory of the current process executable, and that’s precisely the issue: the current program is the Python interpreter, "python.exe"; so the CLR was searching for the YahoAPI.dll assembly into the Python install path; well you guess it: it doesn’t found anything useful there and expressed its vexation by crashing.
Note that this issue doesn’t exist when the code calling the C++/CLI wrapper is a native C++ application, as in the original article, because the native executable is typically run from the same directory as the assembly.
The fix: AssemblyResolve to the rescue
This is a classic issue with .Net when you use it from another context, e.g. when you write some Excel addins; and the fix is quite simple and standard: you just have to help the CLR.
Indeed when the CLR is not able to find an assembly it can ask you to do the job by raising an event: AssemblyResolve.
You only need to provide an handler and try your best to find the assembly and returns it to the CLR so that it can load its types and continue running.
In other words we have to add one more pipe to complete the plumbing.
I’ve used a direct approach: using the currently executing assembly ("YahooAPIWrapper.dll") path as a hint.
Here is the AssemblyResolve handler:
static Assembly^ AssemblyResolve(Object^ Sender, ResolveEventArgs^ args)
{
AssemblyName^ assemblyName = gcnew AssemblyName(args->Name);
if (assemblyName->Name == "YahooAPI")
{
String^ path = Path::Combine(Path::GetDirectoryName(Assembly::GetExecutingAssembly()->Location), "YahooAPI.dll");
return Assembly::LoadFile(path);
}
return nullptr;
}
So with words here is what it does:
- get the current assembly path (e.g. "C:\temp\wrapper\YahooAPIWrapper.dll")
- extract the directory ("C:\temp\wrapper\")
- build the "YahooAPI.dll" assembly full path ("C:\temp\wrapper\" + "YahooAPI.dll")
- load the assembly
- return the assembly to the caller (the CLR)
Simple and IMHO reliable.
The handler is registered in an additional "Initialize" method:
void Initialize()
{
if (!isInitialized)
{
AppDomain::CurrentDomain->AssemblyResolve += gcnew ResolveEventHandler(AssemblyResolve);
isInitialized = true;
}
}
It is registered only once thanks to the isInitialized safeguard.
This "Initialize" method is itself called at each new instantiation of a YahooAPIWrapper:
__declspec(dllexport) void* YahooAPIWrapper_New()
{
Initialize();
return new(std::nothrow) YahooAPIWrapper();
}
So to sum it up:
- the first time
YahooAPIWrapper_New is called it calls Initialize which registers for the AssemblyResolve event, - later when the CLR sees the
YahooAPI type, so for the first time, it tries to load the YahooAPI.dll assembly but fails, - as a last resort it raises the
AssemblyResolve event, - our
AssemblyResolve event handler is called, gets the requested assembly, and send it back to the CLR, - the CLR is happy, loads the assembly and continues running, executing the
YahooAPIWrapper constructor - the next times we call
YahooAPIWrapper_New, Initialize returns immediately because it knows, thanks to the isInitialized flag, that the assembly has already been loaded
Source code
So here are the different parts:
the C++/CLI wrapper header, YahooAPIWrapper.h:
class YahooAPIWrapperPrivate;
class __declspec(dllexport) YahooAPIWrapper
{
private: YahooAPIWrapperPrivate* _private;
public: YahooAPIWrapper();
public: ~YahooAPIWrapper();
public: double GetBid(const char* symbol);
public: double GetAsk(const char* symbol);
public: const char* GetCapitalization(const char* symbol);
public: const char** GetValues(const char* symbol, const char* fields);
};
And the new wrapper implementation in YahooAPIWrapper.cpp:
#using "YahooAPI.dll"
#include <msclr\auto_gcroot.h>
#include <new>
#include "YahooAPIWrapper.h"
using namespace System; using namespace System::IO; using namespace System::Reflection; using namespace System::Runtime::InteropServices;
class YahooAPIWrapperPrivate
{
public: msclr::auto_gcroot<YahooAPI^> yahooAPI;
};
static Assembly^ AssemblyResolve(Object^ Sender, ResolveEventArgs^ args)
{
AssemblyName^ assemblyName = gcnew AssemblyName(args->Name);
if (assemblyName->Name == "YahooAPI")
{
String^ path = Path::Combine(Path::GetDirectoryName(Assembly::GetExecutingAssembly()->Location), "YahooAPI.dll");
return Assembly::LoadFile(path);
}
return nullptr;
}
YahooAPIWrapper::YahooAPIWrapper()
{
_private = new YahooAPIWrapperPrivate();
_private->yahooAPI = gcnew YahooAPI();
}
double YahooAPIWrapper::GetBid(const char* symbol)
{
return _private->yahooAPI->GetBid(gcnew System::String(symbol));
}
double YahooAPIWrapper::GetAsk(const char* symbol)
{
return _private->yahooAPI->GetAsk(gcnew System::String(symbol));
}
const char* YahooAPIWrapper::GetCapitalization(const char* symbol)
{
System::String^ managedCapi = _private->yahooAPI->GetCapitalization(gcnew System::String(symbol));
return (const char*)Marshal::StringToHGlobalAnsi(managedCapi).ToPointer();
}
const char** YahooAPIWrapper::GetValues(const char* symbol, const char* fields)
{
cli::array<System::String^>^ managedValues = _private->yahooAPI->GetValues(gcnew System::String(symbol), gcnew System::String(fields));
const char** unmanagedValues = new const char*[managedValues->Length];
for (int i = 0; i < managedValues->Length; ++i)
{
unmanagedValues[i] = (const char*)Marshal::StringToHGlobalAnsi(managedValues[i]).ToPointer();
}
return unmanagedValues;
}
YahooAPIWrapper::~YahooAPIWrapper()
{
delete _private;
}
extern "C"
{
bool isInitialized = false;
void Initialize()
{
if (!isInitialized)
{
AppDomain::CurrentDomain->AssemblyResolve += gcnew ResolveEventHandler(AssemblyResolve);
isInitialized = true;
}
}
__declspec(dllexport) void* YahooAPIWrapper_New()
{
Initialize();
return new(std::nothrow) YahooAPIWrapper();
}
__declspec(dllexport) void YahooAPIWrapper_Delete(void* wrapper)
{
delete wrapper;
}
__declspec(dllexport) double YahooAPIWrapper_GetBid(void* wrapper, const char* symbol)
{
return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetBid(symbol);
}
__declspec(dllexport) double YahooAPIWrapper_GetAsk(void* wrapper, const char* symbol)
{
return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetAsk(symbol);
}
__declspec(dllexport) const char* YahooAPIWrapper_GetCapitalization(void* wrapper, const char* symbol)
{
return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetCapitalization(symbol);
}
__declspec(dllexport) const char** YahooAPIWrapper_GetValues(void* wrapper, const char* symbol, const char* fields)
{
return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetValues(symbol, fields);
}
}
Compilation and results
Compilation is the same as for the original wrapper:
>cl /clr /LD YahooAPIWrapper.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 16.00.40219.01
for Microsoft (R) .NET Framework version 4.00.30319.18047
Copyright (C) Microsoft Corporation. All rights reserved.
YahooAPIWrapper.cpp
Microsoft (R) Incremental Linker Version 10.00.40219.01
Copyright (C) Microsoft Corporation. All rights reserved.
/out:YahooAPIWrapper.dll
/dll
/implib:YahooAPIWrapper.lib
YahooAPIWrapper.obj
Creating library YahooAPIWrapper.lib and object YahooAPIWrapper.exp
And this time when running the Python script we get the expected results:
>python.exe test.py
Bid: 879.5
Ask: 880.0
Capi: 292.3B
BidAskCapi[0]: 879.50
BidAskCapi[1]: 880.00
BidAskCapi[2]: 292.3B
Conclusion
I don’t know if the whole plumbing could have any practical application, because to interact with C# from Python COM seems a better approach.
Anyway all this plumbing illustrates some interesting points:
- integration with the .Net framework and its assembly loading and probing policy,
- implementation of a simple assemblies resolver,
- creation of a C wrapper for a C++ API,
- interop between Python and C with ctypes
And it confirms a fundamental principle of programming: if your plumbing doesn’t work you’re just not using enough pipes.
There is of course a lot of things to improve to make this production ready, but for learning and playing, this is enough. 
If you catch some typo or mistakes, or have additional questions, or think it could be useful in any context (well really unlikely except legacy 
) feel free to let a comment.
To follow the blog please subscribe to the RSS feed 
