Introduction
My adventures in moving memory (in the modern .NET era) began when I decided I wanted to program the hardware of my own computer. I found WMI to be intolerably slow, and I refused to believe I was hallucinating about the fact that it was intolerably slow.
I had several things going on, all at the same time. A lot of p/Invoke had to happen and a lot of complicated structure processing needed to occur, and the UnmanagedMemoryAccessor object did not prove to be quite as useful to me as I would have liked, so I decided to sort of re-invent the wheel, with purpose.
I needed (wanted) disk access, network access, file association access, virtual disk access, hardware tree access, I really needed to do a lot of memory manipulation.
But none of that was to be found in the .NET Framework. The closest we had to all that hardware and virtual drive interfacing was COM and WMI, and we all know that these things take a bit of time to work through. I appreciate and understand the usefulness of having a standard like WMI, but I personally found its organizational and executive nature to be a bit counter-intuitive, confusing and inconvenient for common tasks.
So. I took a lot of these different interfaces and re-abstracted them. I wrote them, again, from the ground, up. This is, of course, how it is always done in the .NET Framework (at least, how it should be done): I scoured the base "C" language code of the Windows API, itself, reading volume after volume of MSDN documentation, and wrapped CLI libraries around all that I had learned.
Usually, it's fair to say, if someone wants to copy a block of memory in either C# or VB, they p/Invoke the RtlMoveMemory function in kernel32.dll or the memcpy function in msvcrt.dll.
However, this is not the most efficient way to do this.
In doing my research, I realized that the standard and as-provided functions for interfacing with unmanaged memory were inefficient and confusing (especially when it came to writing newly abstracted APIs, where a lot of memory has to be shuffled between the OS and your application, very quickly).
I remedied this with my little memory project: .NET Memory Tools.
But this is only just where my journey begins.
Background
The three CIL opcodes that Visual Basic has no ability to express are cpblk and ldind/stind. Not even C# has a native ability to emit cpblk.
These things, in my opinion, are the secret behind nearly everything that happens in the .NET Framework... or, at least, how fast it happens.
Because Visual Basic cannot produce certain IL opcodes, naturally, I went on a hunt for how to get this done. After years of thinking, I couldn't do anything but p/Invoke to copy my memory (without messing with the somewhat inconvenient InteropServices.Marshal class), I realized I could code in virtual IL; I realized that there was a substantial difference in performance (especially when you start moving around a lot of memory), and I realized that there were even substantial differences among the different ways I could use to improve my memory access performance.
Suddenly, the chart:
(Testing on 1,000,000,000 byte, byte-at-a-time moves from unmanaged memory to an array element and vice-versa.)
Virtual Via Delegate Set Value: 00:00:05.4812442
VB/Pure IL Property Set: 00:00:02.3712762
C#/Unsafe Property Set: 00:00:02.2942013
Virtual Via Delegate Get Value: 00:00:07.4971873
VB/Pure IL Property Get: 00:00:02.8006884
C#/Unsafe Property Get: 00:00:02.4123148
All of the Above, In One Loop: 00:00:20.5727366
p/Invoke CopyMemory Set: 00:00:24.9369384
p/Invoke CopyMemory Get: 00:00:25.2172065
All tests were conducted on an Intel Core i5 4430 Haswell with 16 GB of 1333 DRAM running in Windows 8.1.
Testing with RyuJIT
RyuJIT is the new Just-In-Time compiler that Microsoft is developing for future deployment in the developer suite. Tests run using RyuJIT were, on average, 15% faster than tests run using the current production compiler.
Working With IL OpCodes in Dynamic Functions
The CIL or Common Intermediate Language, is a form of assembly language to which all .NET languages compile down into. It is the CIL byte codes that are executed by the JIT or AOT compilers when a program is run.
The MSDN Library has a reference for the OpCodes and how they are expressed and used.
Constructing the MemCpy function in VisualBasic is relatively straight-forward:
First, you need to import the appropriate namespace:
Imports System.Reflection.Emit
Next, you need to declare your dynamic delegate and function:
Public Delegate Sub MemCpyFunc(dest As IntPtr, src As IntPtr, byteLen As UInteger)
Public ReadOnly MemCpy As MemCpyFunc
Finally, you need to put this code in a module, or any constructor that will be called before MemCpy is required.
Putting it all together, thus:
Imports System.Reflection.Emit
Module Native
Public Delegate Sub MemCpyFunc(dest As IntPtr, src As IntPtr, byteLen As UInteger)
Public ReadOnly MemCpy As MemCpyFunc
Sub New()
Dim dynMtd As New DynamicMethod _
(
"MemCpy",
GetType(Void),
{GetType(IntPtr), GetType(IntPtr), GetType(UInteger)}, GetType(Native)
)
Dim ilGen As ILGenerator = dynMtd.GetILGenerator()
ilGen.Emit(OpCodes.Ldarg_0)
ilGen.Emit(OpCodes.Ldarg_1)
ilGen.Emit(OpCodes.Ldarg_2)
ilGen.Emit(OpCodes.Cpblk)
ilGen.Emit(OpCodes.Ret)
MemCpy = CType(dynMtd.CreateDelegate(GetType(MemCpyFunc)), MemCpyFunc)
End Sub
End Module
Incorporating Pure IL Into A Function
Sometimes, you may want to implement a function in a Visual Basic or C# library that requires the use of OpCodes that are otherwise unsupported in the language.
As I mentioned, earlier, the three IL opcodes that are most commonly used in day-to-day programming that are missing from Visual Basic are cpblk, ldind and stind. 'cpblk', as we saw, above, copies a segment of memory from one memory location to the other. 'ldind' and 'stind', on the other hand, are used to retrieve blittable variables from a memory pointer. Supported variables include signed and unsigned integers and floating point variables up to 8 bytes in length. These two OpCodes can be found implemented in C#'s 'unsafe code' feature, as pointer dereferences.
In order to include pure IL functions in libraries that are otherwise coded in Visual Basic, I chose a free Visual Studio add-in called IL Support. IL Support adds the ability to compile pure .il files as 'partial' classes, or extensions to classes already present in your Visual Basic or C# code. To achieve this, the add-in makes use of an attribute that can be applied to functions that implement what is called a forward reference, indicating that the implementation of the function is provided elsewhere. We do this to provide a declaration that can be used in the compiler environment that gives Intellisense the ability to validate your code. Implementing the IL function is up to you. Debugging is also a tad bit difficult in that you can't always step into a misbehaving IL function. IL Support also provides a rudimentary editor with syntax highlighting.
First, we need to import a namespace:
Imports System.Runtime.CompilerServices
Next, we declare the function in VB:
Namespace Memory
Public Class MemoryTool
Public Handle As IntPtr
<MethodImpl(MethodImplOptions.ForwardRef)>
Public Function GrabBytes(byteIndex As IntPtr, length As Integer) As Byte()
Return Nothing
End Function
End Class
End Namespace
The function, as defined above, returns 'Nothing.' This line of code is ignored in the final compilation, and the code, below, is inserted in its place, because of the MethodImp() attribute. I put that line of code there to prevent IntelliSense from throwing a warning about a function call with no return value.
The byteIndex parameter is declared as an IntPtr because in the CIL, IntPtrs are converted into a type called native int. Native ints are different size depending on which platform is used to compile the binary: 4 bytes for 32-bit platforms and 8 bytes for 64-bit platforms. This provides a natural computational limit for the possible values for byteIndex.
As you can see, we give it a Namespace, and a class, so that all parts of the feature can be demonstrated.
Next, we will implement the actual function in the accompanying .il file:
.namespace Memory
{
.class public MemoryTool
{
.method public instance uint8[]
GrabBytes(native int byteIndex, int32 length) cil managed
{
.maxstack 3
.locals init
(
uint8[] x
)
ldarg.0
ldfld native int Memory.MemoryTool::Handle
ldarg.1
add
starg 1
ldarg.2
newarr [mscorlib]System.Byte
stloc.0
ldloc.0
ldc.i4.0
ldelema [mscorlib]System.Byte
ldarg.1
ldarg.2
volatile.
cpblk
ldloc.0
ret
}
}
}
IL Support with Visual Studio 2013
As of the writing of this article, the current version of IL Support does not, by default, install correctly for Visual Studio 2013, although it can be made to work with it, in two steps.
First, download the .vsix file, copy it to a temporary file and give it a .zip extension. Open the .zip, and inside there will be a file called 'extension.vsixmanifest'. Open that file, and find this block:
<VisualStudio Version="11.0">
<Edition>Ultimate</Edition>
<Edition>Premium</Edition>
<Edition>Pro</Edition>
<Edition>IntegratedShell</Edition>
</VisualStudio>
Directly below this block (directly after </VisualStudio>, insert the following block of text:
<VisualStudio Version="12.0">
<Edition>Ultimate</Edition>
<Edition>Premium</Edition>
<Edition>Pro</Edition>
<Edition>IntegratedShell</Edition>
</VisualStudio>
Save the file back into the .zip, and give the file back its old '.vsix' extension. You will now be able to install IL Support for Visual Studio 2013.
However, the problem, at this point, is that IL Support cannot find ilasm and ildasm, which are required because IL Support performs a post-compile operation to disassemble the compiled project into IL, insert the user-developed IL code, and recompile the final EXE or DLL.
To get a program to actually compile, you need to copy all the files from:
C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools
to:
C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin
This method works perfectly running on Windows 8.1. I don't know what method you will have to use for Windows 7, but I am assuming that a similar work-around exists.
The .NET Framework, Itself, May Be Flawed
I was given the .NET Framework code, to review, myself, and I was somewhat perplexed by the Buffer, UnmanagedMemoryAccessor and BitConverter classes, which were all written much the same way that any simple .NET programmer might write them. I was sincerely disappointed when I got a hold of the code and realized that they were doing p/Invoke down to MemCpy to move most of their memory.
The common legend that I've seen is that the cpblk CIL opcode was implemented poorly in the 32-bit version of the .NET Framework glue, with byte-at-a-time memory moves. The 64-bit version was more clever in that they used 128 bit registers for their memory copying. The CIL opcodes compile down into what is called the 'fast lane' of the JIT/AOT compiler... there is very little pre-compiled logic that goes into dealing with those opcodes; they are run, in real-time, as if they were pure byte-code assembly. MemCpy, on the other hand, no matter how much they optimize the call, is still p/Invoke.
I was utterly flabbergasted by my discovery. I only hope they improve this situation in the RyuJIT and Roslyn builds they have, forth-coming. But for anyone who may need to know this: they have not changed the Buffer.BlockCopy code since .NET 2.0 (to my knowledge).
After all, things like cpblk and localloc are built in to the very fabric of the .NET Framework, but they are not utilized, whatsoever, to even achieve the simplest of tasks in their code, even unto mscorlib.
The one question I've never gotten an answer to is, "if you built it, why don't you use it, yourself?"
I am somewhat disappointed that not only have they not been able to take advantage of their own technology, they haven't even felt the need to be bothered to correct their original implementation of a fundamental yet flawed operation.
Update: This situation is currently being reviewed by Microsoft.
Summary and Conclusion
The advantages of using pure IL OpCodes to manipulate memory cannot be overstated, especially for developers that program in Visual Basic. The Visual Basic programming language simply has no ability to perform some common memory manipulation tasks without using the array of built-in .NET Framework classes; sometimes external function calls like those can produce unacceptable performance degradation, especially when doing rapid manipulation of memory.
I believe that at least some support for embedding CIL into Visual Basic or C# applications should be considered by the language team, at Microsoft. I also believe that in order for Visual Basic to reach true parity with C#, there needs to be some method for allowing ldind and stind to be emitted by the language. With the birth of Microsoft's open compiler project, Roslyn, we may soon get the chance to explore some of the ways in which this could be done.
I enjoy programming in Visual Basic, very much. I also enjoy developing in the .NET platform and utilizing the many exciting technologies that Microsoft has developed, including WPF and WCF. I would like to see more support for native language memory manipulation included into Visual Basic, but for the time-being the methods suggested, here, should provide some relief for Visual Basic programmers who may be struggling to achieve performance that is equivalent to C# in matters such as these.
Special Thanks
I would like to give special thanks to Lucian Wischik and Anthony D. Green, the co-leads on the Visual Basic Language Team at Microsoft. They gave me invaluable advice as I was exploring these avenues and great support regarding various topics related to my research into this project. I may also say that they have a great sense of humor.
External Links and Code
