Introduction
Here is real code which demonstrates a quick and not so dirty way in which to write and read multidimensional arrays of elements of blittable type in binary form.
Background
Often, it's required to write and read arrays of simple data in binary form. Most of the time, the elements are of simple blittable types. And most of the time, there are BinaryWriter
or BinaryReader
given as IO interface. So, this will be simple for one given array, but if you want to create a library which deals with multidimensional arrays of unknown element type and size, you will get an idea that this would not be such a simple task.
To write/read all elements, it's required to iterate over all elements of array with apriory unknown dimension - how to code this? Then, Read/Write of elements with BinaryWriter
/BinaryReader
of unknown type will require to use keyword dynamic
for writing and switch
-case
for reading.
Combining of the two will require to use either strong generic typing, reflection, type tables or some other tricks - this all is a lot of work and in some cases, has huge performance penalties.
After that, the next idea will be... "Blittable types?" "How are they stored?", "YES!!! Write the underlied memory to stream. Simple!".
But if you take a look at BinaryWriter
/ BinaryReader
, you will see that there are no members that consume IntPtr
or other type of pointer, only byte[]
.
So, a simple try with search engine gets you a full nightmare: "How to create an interop library to use fwrite
?", "PInvoke
to FileWrite
", "How to convert something to byte[]
. Oh no! This is not possible, but you can copy memory..." - the whole program of expert tips and tricks including some sophisticated benchmarks.
But we can get this really very simple - only compile with /unsafe
will be required, but no unmanaged code, no PInvoke, no buffer copy, no tricks - very small and simple code!
The idea besides is to "wrap a memory" to an instance of UnmanagedMemoryStream
. Never heard of?
Using the Code
Here are two independent full working methods just to show the real code.
Method to Write an Array
static void testWriteArray()
{
int[, ,] array3D = new int[,,] { { { 1, 2, 3 }, { 4, 5, 6 } },
{ { 7, 8, 9 }, { 10, 11, 12 } } };
FileStream writeStream = new FileStream("c:\\test\\csharp.net-3darray.dat", FileMode.Create);
BinaryWriter writeBinary = new BinaryWriter(writeStream);
GCHandle arrHandle = GCHandle.Alloc(array3D, GCHandleType.Pinned);
IntPtr arrPtr = arrHandle.AddrOfPinnedObject();
unsafe
{
int arrLen = array3D.Length;
if (arrLen > 0)
{
IEnumerator enumerator = array3D.GetEnumerator();
enumerator.MoveNext();
int arrSize = System.Runtime.InteropServices.Marshal.SizeOf(enumerator.Current) * arrLen;
using (UnmanagedMemoryStream arrStream =
new UnmanagedMemoryStream((byte*)arrPtr.ToPointer(), arrSize))
{
arrStream.CopyTo(writeBinary.BaseStream, (int)arrStream.Length);
}
}
}
arrHandle.Free();
writeStream.Close();
}
Method to Read an Array
static Array testReadArray()
{
int[, ,] array3D = new int[2, 2, 3];
FileStream readStream = new FileStream("c:\\test\\csharp.net-3darray.dat", FileMode.Open);
BinaryWriter readBinary = new BinaryWriter(readStream);
GCHandle arrHandle = GCHandle.Alloc(array3D, GCHandleType.Pinned);
IntPtr arrPtr = arrHandle.AddrOfPinnedObject();
unsafe
{
int arrLen = array3D.Length;
if (arrLen > 0)
{
IEnumerator enumerator = array3D.GetEnumerator();
enumerator.MoveNext();
int arrSize =
System.Runtime.InteropServices.Marshal.SizeOf(enumerator.Current) * arrLen;
using (UnmanagedMemoryStream arrStream = new UnmanagedMemoryStream
((byte*)arrPtr.ToPointer(), arrSize, arrSize, FileAccess.Write))
{
readBinary.BaseStream.CopyTo(arrStream, (int)arrStream.Length);
}
}
}
arrHandle.Free();
readStream.Close();
return array3D;
}
What You Can Expand or How to Use in Your Library?
If you want to use the snippets in your real life library, simply write rank and dimensions to stream and use this information later together with Array.CreateInstance
.
Points of Interest
I am always interested in ways to optimize my work.
History
- 27.01.2015 - First version