Best gotchas in C++/CLI
For managed developers, C++/CLI allow them to take advantage of
native
C/C++ code
in the .NET world.
For unmanaged developers, C++/CLI allow them to take advantage
of managed .NET
code
in the native world.
ITW is the acronym for It just work. Most of
the time, it is used by clueless developers that want to feel smart -...ok I am
one of those ![Smile | <img src=]()
" src="http://www.codeproject.com/script/Forums/Images/smiley_smile.gif" /> -, and that
only mean they used C++/CLI and not pInvoke to
interact with native code.
Why do I call them clueless ? Because it IJDNWTW (It Just
Does Not Work That Way), and you have to know current C++ practices and
internals to code correctly... Thus the need of this article.
Why would you need to use C++/CLI ? because pInvoke
is also difficult to get it right, and require you to write grunt wrapper code to
interact with the unmanaged API.
pInvoke is the right solution when you have only 2 or 3 unmanaged
call without complex structures in your application...
More than that, C++/CLI will save you time but only if you are
well advised.
In Visual Studio 2010, Microsoft stopped intellisense support on
C++/CLI.
C++ is hard enough with intellisense, so without it, C++/CLI
was useless, and classic
C# pInvoke code was much easier to write.
And then Visual Studio 2012, re-enabled it. I think that the strategy
of Microsoft
is first to bring C++ developers on the metro plateform, then smoothly bring them
to managed world... That's just speculation though. After all, they did that
with VB6 developers with VB.NET, and it
worked...
Last days, I made an article around
HttpSysManager, a tool that made heavy use on the unmanaged HTTP Server
API.
My code worked, and I had a simple goal : Get rid of this awful pInvoke code, and
use C++/CLI instead.
My unit tests was done, so it was just really a matter to move managed classes to
C++/CLI, and call unmanaged API without the pInvoke layer.
This is a story about my journey, the good, the bad and the ugly.
Here is from what I started.
Simple enough, a WPF project, with all my interop code, and a test project.
Here was the classes of HttpSysManager, and what I wanted to do
with it :
As you can see, the only class that will left managed is HttpSysManager,
this is the facade used accross my codebase and unit tests.
Most of the code will be deleted, because most of the code was using pInvoke.
With C++/CLI I will be able to call unmanaged API directly.
So I opened my project in Visual studio 2012, and create a new C++/CLI library project
called HttpSysManager.Native.
I don't understand, most of this stuff... my way with dealing with the unknow
is
to delete it. Because it is only when things crash that we can learn something
new.
Now the compiler yell at me
error C1083: Cannot open precompiled header file:DebugHttpSysManager.Native.pch':
No such file or directory.
I don't know what is a precompiled header, I deleted it, and now it wants it...
I've gone in my project property, and found the settings Precompiled
Header setting, I changed the Use (/Yu) setting to Not
Using Precompiled Headers.
Then, I removed the #include in AssemblyInfo.cpp, and it compiled !
Great now I'm the master of the situation.
Along the way, I notice the parameter General -> Plateform Toolset set to Visual
Studio 11 (2012).
It corresponds to the compiler, linker, and VC++ runtime libraries used by your
project. I set it to
Visual Studio 10 (2010), because I don't want to rely on a RC version.
Now, I add a reference from my managed project to my native project and...
A reference to 'HttpSysManager.Native (Visual Studio 2010)' could not be added.
An
assembly must have a 'dll' or 'exe' extension in order to be referenced.
Nevermind... just a bug on VS2012 RC. The workaround I found on internet
is
to reference the .dll instead of the project.
I did not like it because when I will compile HttpSysManager in Release
mode,
it will pick the debug dll of HttpSysManager.Native, and it will
not compile the dependent project automatically, if that is not alreaedy done !
I little hack in the HttpSysManager.csproj fix that problem, at the
expense of an annoying warning message in the error list.
<ItemGroup>
<ProjectReference Include="..\HttpSysManager.Native\HttpSysManager.Native.vcxproj"></ProjectReference>
<Reference Include="System" />
<Reference Include="System.Core" />
Let's try to create a new managed class in HttpSysManager.Native and
reference it in HttpSysManager.
public ref class Test {
};
Ok I can reference it, I can instantiate Test...
I run it and... *boom* BadImageFormatException.
Maybe I need to address the warning : There was a mismatch between the processor
architecture of the project being built "MSIL" and the processor architecture of
the reference
HttpSysManager.Native "x86".
Ok that mean that my VC++ project is x86, and my managed project
AnyCPU... but wait !
Gotcha number 1 : AnyCpu mode does not exist for C++/CLI assembly
I don't want to release two different binaries x86 and x64, so I just changed
HttpSysManager
plateform to
x86, which is also supported in
x64 OS with virtualization
(I will talk about that soon).
Ok, now it works !
Let's run it on another computer, without visual studio...
*boom* it crashed and burn in flames !
Ok maybe I forgot a dependency ?
procmon.exe
helps me :
Ok, I missed msvcr100D.dll, but, like a popular coffee machine maker
would say :
what else ?
A cool tool to see dependencies of your native library is Dependency Walker, I runned it on
HttpSysManager.Native.dll.
All dll are part of windows, except msvcp100D and msvcr100D.
Why do I need them ? This research brings me to an old concept I forgot on C/C++
linkers and linking
in general...
Do you remember this popular program ?
#include<stdio.h>
void main()
{
printf("hello world !");
}
Most of you remember that stdio.h contains the declaration
of printf.
But where is the actual implementation ? Response : It depends.
With static linking, the implementation is in a .lib
file, and merged
at compile time to your .exe, or .dll.
With static linking, your program does not have any dependency
at
runtime, at the expense of its size.
In the .NET world, we use ILMerge for such thing.
In the .NET world .lib files does not exists, only other managed
.dll can be merged inside another .exe or
.dll.
Another alternative, is load time linking. This means that the
windows loader will fetch dependencies only when the
DLL, or EXE are loading into your program.
These dependencies are specified in the header of the DLL or EXE file. (we call
that the
PE header)
It is what we have seen with Dependency walker.
At compile time, the C/C++ linker always need
.lib because it needs to solve
your code's references by giving them an implementation... even if the real
implementation is resolved at load time ... So the trick is to
give to the linker .lib files with empty implementations, these are
only
stubs that will be replaced at load time.
These .lib file are within the Windows SDK.
All the real implementations are in dll files, loaded at load time.
Thus, the executable size is smaller and your program is much more modular... But
deployment is harder.
The last linking type, but not the least, is runtime linking, that
is the equivalent that what we do in .NET with Assembly.Load...
We don't need any .lib at compile type, but the dll need to be present at
runtime during LoadLibrary.
#include <Windows.h>
typedef int (*PPrintf)(const char * _Format, ...);
int main()
{
PPrintf printf; HMODULE module = LoadLibrary(L"MSVCR100D.DLL"); printf = (PPrintf)GetProcAddress(module, "printf"); printf("hello world");
return 0;
}
How can I know, in the previous code snippet, that MSVCR100D.dll has
the implementation of printf ?
Well... every .dll export their functions, they have an export
table that map a function name
to their address in the dll, you can see this table with dependency walker.
GetProcAddress just use this information, add the DLL base address,
and return the address.(base address = where the dll was loaded in virtual memory
of the process)
In the .NET world, all .NET assemblies are loaded at JIT time, this means that if
you inspect a normal .NET assembly, you will not see your managed dll with dependency
walker.
Maybe now you understand why my application crash on another computer... the problem
is
just that
HttpSysManager.Native link the C Runtime Library
(printf and all basic functions) at load time !
So I tried to set it up to static linking to remove dependencies
(Multi-threaded Debug)... Except that it does not work in C++/CLI !
(Correction : static linking works but not on the C Runtime Library -printf,memset,memcpy and the likes)
Thus gotcha number 2 : With C++/CLI, forget static linking on the C Runtime Library, and embrace VC++ redistribuable
dlls.
(In C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\redist)
I don't want to create an installer, and ask my user to install
Microsoft Visual
C++ 2010 Redistributable Package (x86) so the solution was to copy these
.dll in the output folder at compile time with this piece of msbuild code at the
end of my C++/CLI project (.vcxproj)
<Target Name="AfterBuild">
<ItemGroup>
<VCRedist Include="$(VCInstallDir)redist\$(VCRedistDir)*.dll"></VCRedist>
</ItemGroup>
<Copy SourceFiles="@(VCRedist)" DestinationFolder="$(OutputPath)"></Copy>
</Target>
$(VCInstallDir) is the install directory of your visual studio tool
set. I know it existed, because lots of dialog box in vcxproj' configuration show
all the list:
For that, I have gone in Project properties/Linker/Input/Additional dependencies,
and clicked on the arrow then edit.
$(VCRedistDir) is a folder specific to my environment I needed to specify.
The path for the C Runtime Library in x86 Debug mode is
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<CLRSupport>true</CLRSupport>
<CharacterSet>Unicode</CharacterSet>
<VCRedistDir>Debug_NonRedist\x86\Microsoft.VC100.DebugCRT\</VCRedistDir>
</PropertyGroup>
This PropertyGroup is only used in Debug|Win32 (x86).
So in summary, in x86 debug mode with vs 2010 toolset, $(VCInstallDir)redist\$(VCRedistDir)*.dll,
is equal to this files C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\redist\*.dll.
For details, I already
wrote a lot on MSBuild.
But wait, that is not all : Gotcha number 3 : C++/CLI output dir
is SOLUTION_DIRECTORY/Debug|Release,
as opposed to PROJECT_DIRECTORY/bin/Debug|Release for managed projects.
Using a project reference from my managed project to the native one did not copied
the
C Runtime Library dlls from the native's project
directory.
The solution was to change the output directory of my managed project to ../Debug|Release
as opposed to the default bin/Debug|Release...
Now it works fine on my other computer... Few... and I did not start coding yet !
A common problem with native/managed interaction, is that the managed world use
System.String whereas the native world use char* and whchar*
(ansi and unicode strings).
The common solution is to use
IntPtr Marshall::StringToHGlobalAnsi|Unicode(String
managedStr)to allocate native string from managed one, and then free
the allocation after that... Here is a simplified version.
#include <stdio.h>
using namespace System;
using namespace System::Runtime::InteropServices;
void main()
{
String^ line = Console::ReadLine();
char* cStr = (char*)Marshal::StringToHGlobalAnsi(line).ToPointer();
printf(cStr);
delete cStr; }
Thanks for trying, but that's not my day !
This problem brought me within the memory management of windows !
In this journey I met
VirtualAlloc,
GlobalAlloc,
RtlAllocateHeap, and the
CRT (C Runtime Time) Debug Heap.
At the lowest level, Windows can only request whole memory pages
to the memory manager, each page have the same fixed size (you can see it with
GetSystemInfo).
And you can use
VirtualAlloc, to allocate at the page level.
Your question is now : So how is it possible that in my program, I can allocate
at the
byte level ? The response is : thanks to a data structure
called the heap. And your program create its own heap called the
global heap at the start of your program with
RtlCreateHeap.
In normal case, every malloc, or new or free
or delete are just call to the underlying Heap API on the global
heap. (RtlAllocateHeap,
and
RtlFreeHeap)
Marshal::StringToHGlobalAnsi means : allocate an ANSI native string
from the Global Heap, so why my code crashes like that ?
The answer is... The
CRT (C Runtime Time) Debug Heap !
In debug mode, malloc, delete, free, new does not allocate in the
Global Heap
, but in a special debug heap for debugging purpose.
This heap will directly tell you want you free the same memory block twice ! Very
handy, you can use
DbgInt.h with the
debug heap api to check its state at runtime for debug configurations !
In summary : My code crash because StringToHGlobalAnsi allocate memory
from the Global Heap, but, my native project compiled in debug
mode delete and alloc memory on the CRT Debug Heap !
Here is how to fix that.
#include <stdio.h>
using namespace System;
using namespace System::Runtime::InteropServices;
void main()
{
String^ line = Console::ReadLine();
char* cStr = (char*)Marshal::StringToHGlobalAnsi(line).ToPointer();
printf(cStr);
Marshal::FreeHGlobal(IntPtr(cStr));
}
Note that my previous code would work perfectly fine in Release
mode.
Update: Thanks to CodeProject, I always learn better way to do things " />
Here is another way to do the same things easily:
#include <stdio.h>
#include <msclr/marshal.h>
using namespace System;
using namespace msclr::interop;
using namespace System::Runtime::InteropServices;
void main()
{
String^ line = Console::ReadLine();
marshal_context context;
const char* cStr = context.marshal_as(line);
printf(cStr); }You can even see in the implementation that for wchar_t (unicode char) it allocates with Marshall::StringHGlobalUni and free with Marshal::FreeGlobal. (Thanks to Espen Harlinn for the trick. For more information go to Overview of Marshalling in C++.)
Gotcha number 4 : In Debug mode, you are not using the same native heap
than .NET Marshal class
You can check everything I tell you by yourself with
API montitor.
(By the way this tool is great to reverse engineer other programs and get WINAPI
functions !)
Another gotcha involve template classes take this code :
template<class T>
public ref class Base
{
public: void MethodBase()
{}
};
public ref class Derived : Base<wchar_t>
{
};
public ref class Derived : Base<wchar_t>
{
};
And now, I try to use it in my managed project:
Derived derived = new Derived();
derived.MethodBase();
And here is the pretty error : 'Derived' does not contain a definition for
'MethodBase' and no extension method 'MethodBase' accepting a first argument of
type 'Derived' could be found.
Ok, I ILSpy my native assembly
and see that :
First question : Why there is 2 non generic Base class instead of one
generic Base<T> class ?
Second question : Why I can't see my MethodBase method ?
The problem is that I am using a template and not a generic...
The difference is that templates are classes generated at compile time !
So why not use a generic ?
generic<class T>
public ref class Base
{
public: void MethodBase()
{}
};
And the problem is : gotcha number 5 you can't use generic with generic native type parameter...
This line give me a compilation error because _GUID is a native type...
public ref class Derived2 : Base<_GUID>
{
};
Ok, nevermind, I don't care !
Now question two : Why can't I see my MethodBase method ?
Well here is
why : You cannot access templates defined in a referenced assembly with C++/CLI
language syntax, but you can use reflection. If a template is not instantiated,
it’s not emitted in the metadata. If a template is instantiated, only referenced
member functions will appear in metadata.
Solution 1 : makes MethodBase virtual. (Thanks Espen Harlinn)
generic<class T>
public ref class Base
{
public: virtual void MethodBase()
{}
};Solution 2 : references MethodBase in the native code, and somehow call it from accessible code.
template<class T>
public ref class Base
{
private: void _DummyAndUselessMethod() {
if(false)
{
MethodBase(); }
}
public: Base()
{
_DummyAndUselessMethod(); }
public: void MethodBase()
{
}
};
That explain why we can see in HttpSysManager.Native such code:
template<class TSet>
public ref class ManagedSet
{
private: void ShowToManagedCode()
{
if(false)
{
Add();
Update();
Delete();
ConfigId = HttpServiceConfigCache;
}
}
Thus Gotcha number 6 : You have to reference and somehow call methods from
template class to use them in managed code.
This is about two gotchas that combine their strenght together to do another bigger gotcha...
Let's see a simple and innocent code like this one :
namespace TestCLI {
#pragma managed(push, off)
public class MyNativeClass
{
public:
MyNativeClass(){printf("constructor called");};
~MyNativeClass(){printf("deleted");};
};
void InitGlobal()
{
static MyNativeClass globalObject;
}
#pragma managed(pop)
public ref class MyManagedClass
{
public:
MyManagedClass()
{
InitGlobal();
};
};
}
Thanks to Petro Protsyk, I found that this code was a culprit of vicious crash at the end of the program... Here is my console app calling this code.
static void Main(string[] args)
{
new TestCLI.MyManagedClass();
}
Run it annnnd... it crashes when the program closes.
0xC0020001: The string binding is invalid !
The workaround ? Simple enough... You just have to make the function InitGlobal managed. (I moved the #pragma managed(pop) up)
namespace TestCLI {
#pragma managed(push, off)
public class MyNativeClass
{
public:
MyNativeClass(){printf("constructor called");};
~MyNativeClass(){printf("deleted");};
};
#pragma managed(pop)
void InitGlobal()
{
static MyNativeClass globalObject;
}
public ref class MyManagedClass
{
public:
MyManagedClass()
{
InitGlobal();
};
};
}
Another workaround, is to move the static native object to class scope.
namespace TestCLI {
#pragma managed(push, off)
public class MyNativeClass
{
public:
MyNativeClass(){printf("constructor called");};
~MyNativeClass(){printf("deleted");};
static MyNativeClass globalObject;
};
MyNativeClass MyNativeClass::globalObject;
void InitGlobal()
{
}
#pragma managed(pop)
public ref class MyManagedClass
{
public:
MyManagedClass()
{
InitGlobal();
};
};
}Gotcha number 7 : You can't instantiate a static global native object with a destructor from a native function or method...
That would be good, as long as your code is running in the default AppDomain... Because if you don't as in the following code :
static void callback()
{
new TestCLI.MyManagedClass();
}
static void Main(string[] args)
{
var domain = AppDomain.CreateDomain("te");
domain.DoCallBack(callback);
AppDomain.Unload(domain);
}
It crashes !
Gotcha number 8 : You can't instantiate a static global native object with destructor from a managed method if you are not in the default app domain !!!
Now you can combine Gotcha number 8 and Gotcha number 7 together and you get the ubber gotcha !
Gotcha number 9 : You can't instantiate a static native object with destructor in any way if your code does not execute in the default app domain !
Tarmik found a way to overcome Gotcha 9 and 7 ! The problem seems to be that the process call static object's destructor when the CLR is always freed. It seems that the C++/CLI library is unloaded by the CLR, but its static variables' destructors are called by the C Runtime Library just after.
Anyway, the solution is clever, you just need to call destructors before the C Runtime Lib, but before CLR close. For this, Tarmik took the code crtdll.c and reverse engineered it to produce the following snippet : It calls destructors, and remove them from the destructor's list.
In the following code, CrtDestroyStatics is called when winform close the application. (System::Windows::Forms::Application::ApplicationExit) You can call it elsewhere depending on your situation.
Update : for MSVCRT10.dll, check bendenajones'comment.
Update : for VS2013 you have to follow an other bendenajones'comment.
#ifdef _MANAGED // Works only with managed C++
#pragma once
#pragma managed(push, on)
#include lt;stdlib.h> //free
void CrtDestroyStatics(void);
void OnApplicationExit( System::Object^, System::EventArgs^)
{
CrtDestroyStatics();
}
class CRegisterManagedShutdown
{
public:
CRegisterManagedShutdown()
{
System::Windows::Forms::Application::ApplicationExit += gcnew System::EventHandler( &OnApplicationExit );
}
}gCRegisterManagedShutdown;
typedef void (__cdecl *_PVFV)(void);
extern "C" void * __cdecl _decode_pointer(void *);
extern "C" void * __cdecl _encoded_null();
extern "C" _PVFV *__onexitbegin;
extern "C" _PVFV *__onexitend;
void CrtDestroyStatics(void)
{
_PVFV * onexitbegin = (_PVFV *)_decode_pointer(__onexitbegin);
if (onexitbegin)
{
_PVFV * onexitend = (_PVFV *)_decode_pointer(__onexitend);
_PVFV function_to_call = NULL;
_PVFV * onexitbegin_saved = onexitbegin;
_PVFV * onexitend_saved = onexitend;
while (1)
{
_PVFV * onexitbegin_new = NULL;
_PVFV * onexitend_new = NULL;
while (--onexitend >= onexitbegin && (*onexitend == NULL || *onexitend == _encoded_null()))
{
}
if (onexitend < onexitbegin)
{
break;
}
function_to_call = (_PVFV)_decode_pointer(*onexitend);
*onexitend = (_PVFV)_encoded_null();
(*function_to_call)();
onexitbegin_new = (_PVFV *)_decode_pointer(__onexitbegin);
onexitend_new = (_PVFV *)_decode_pointer(__onexitend);
if ( ( onexitbegin_saved != onexitbegin_new ) || ( onexitend_saved != onexitend_new ) )
{
önexitbegin = onexitbegin_saved = onexitbegin_new;
önexitend = onexitend_saved = onexitend_new;
}
break;
}
free ( onexitbegin ) ;
__onexitbegin = __onexitend = (_PVFV *)_encoded_null();
}
}
#pragma managed(pop)
#endif
- Gotcha number 1 : AnyCpu mode does not exist for C++/CLI assembly.
- Gotcha number 2 : With C++/CLI, forget static linking on C Runtime Library, and embrace VC++ redistribuable
dlls.
(In C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\redist).
- Gotcha number 3 : C++/CLI output dir
is SOLUTION_DIRECTORY/Debug|Release,
as opposed to PROJECT_DIRECTORY/bin/Debug|Release for managed projects.
- Gotcha number 4 : In Debug mode, you are not using the same native heap
than .NET Marshal class.
- Gotcha number 5 : you can't use generic with generic native type parameter.
- Gotcha number 6 : You have to reference and somehow call methods from
template class to use them in managed code.
- Gotcha number 7 : You can't instantiate a static global native object with a destructor from a native function or method...
- Gotcha number 8 : You can't instantiate a static global native object with destructor from a managed method if you are not in the default app domain.
- Gotcha number 9 : You can't instantiate a static global native object with destructor in any way if your code does not execute in the default app domain !
Don't get me wrong, C++/CLI is awesome... My code is now simpler and has no
memory leaks thanks to the RAII principle. However be careful, it is not as easy as it seems, and bring you its load of challenge !
