This is a pretty complicated setup. I have a class that allocates bytes from NativeMemory.Alloc() and constructs a byte[] on that memory. Currently, the returned byte[] works just fine, so that is not an issue. The problem is when freeing the memory. For some reason, it will work correctly for the first 10 or so allocations and then will throw a heap corruption error. At first, I thought this was the garbage collector treating the object as if it was allocated on managed memory and the manual free was causing a double free error, so I made the GC pin the object to prevent it from deallocating it (the gc never ran during the execution anyway, so I am not sure of this). I have logged the addresses that are (de)allocated so I know it is not because it is the wrong address being freed.
Just to reiterate, I am allocating a
managed byte[] on
native memory with this class, the byte[] does work fine when accessing elements, it only craps out when trying to free it.
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;
namespace InfinityOfficialNetwork.Shared.Memory.Generic
{
public unsafe class NativeAllocator : IAllocator
{
private ConcurrentDictionary<nuint,object> addresses = new ConcurrentDictionary<nuint, object>();
private static class AllocateBytesHelper
{
public static readonly nuint methodTablePointer;
public static readonly nuint arrayHeaderSize;
static AllocateBytesHelper()
{
byte[] bytes = [];
nint** mtablePtr = (nint**)&bytes;
methodTablePointer = (nuint)(**mtablePtr);
arrayHeaderSize = (nuint)sizeof(nuint) * 3;
}
}
public ref byte[] AllocateBytes(nuint size)
{
nuint* memory = (nuint*)NativeMemory.Alloc((AllocateBytesHelper.arrayHeaderSize + size));
Debug.WriteLine($"Allocating {size} bytes of memory at address {((nuint)memory).ToString("X")}");
memory[0] = (nuint)memory + (nuint)sizeof(nuint) * 2;
memory[1] = 0;
memory[2] = (nuint)AllocateBytesHelper.methodTablePointer;
memory[3] = size;
ref byte[] memoryAsArray = ref *(byte[]*)memory;
object handle = GCHandle.Alloc(memoryAsArray, GCHandleType.Pinned);
addresses.AddOrUpdate((nuint)memory, handle, (a, b) => { return b; });
return ref memoryAsArray;
}
public unsafe nuint AllocateBytesPointer(nuint size)
{
return (nuint)NativeMemory.Alloc(size);
}
public void DeAllocateBytesPointer(nuint memory)
{
NativeMemory.Free((void*)memory);
}
public void DeAllocateBytes(ref byte[] memory)
{
byte* bytes = (byte*)(Unsafe.AsPointer(ref memory));
object handle;
if (addresses.Remove((nuint)bytes, out handle))
{
GCHandle gCHandle = (GCHandle) handle;
gCHandle.Free();
Debug.WriteLine($"Freeing memory at address {((nuint)bytes).ToString("X")}");
NativeMemory.Free(bytes);
}
else
{
throw new ArgumentException("Address was not allocated by this allocator instance or is managed memory", nameof(memory));
}
}
}
}
[TestMethod]
public unsafe void TestMethod2()
{
List<nuint> ptrs = new();
for (int n = 1; n < 1024 * 1024 * 128; n *= 2)
{
IAllocator allocator = new NativeAllocator();
ref byte[] bytes = ref allocator.AllocateBytes((nuint)n);
Assert.IsTrue(bytes.GetType() == typeof(byte[]));
for (int i = 0; i < n; i++)
bytes[i] = (byte)i;
for (int i = 0; i < n; i++)
Assert.IsTrue(bytes[i] == (byte)i);
GC.Collect();
allocator.DeAllocateBytes(ref bytes);
}
}
What I have tried:
I have tried to pin the memory in GC so it wouldn't try to move it or deallocate it. I am pretty sure that GC is not the issue because when I manually call GC.Collect() in the loop, it still works for about 10 iterations. I made sure that the addresses are the same so I know it is not caused by some weird offset issue.
Allocating 1 bytes of memory at address 1BA825659A0
Freeing memory at address 1BA825659A0
Allocating 2 bytes of memory at address 1BA82565340
Freeing memory at address 1BA82565340
Allocating 4 bytes of memory at address 1BA82565160
Freeing memory at address 1BA82565160
Allocating 8 bytes of memory at address 1BA825650A0
Freeing memory at address 1BA825650A0
Allocating 16 bytes of memory at address 1BA82565AF0
Freeing memory at address 1BA82565AF0
Critical error detected c0000374
The process hit a breakpoint the Common Language Runtime cannot continue from.
This may be caused by an embedded breakpoint in the native runtime or a breakpoint set in a can't-stop region.
To investigate further, use native-only debugging.
Exception thrown at 0x00007FFBC51BDAB5 (ntdll.dll) in testhost.exe: 0xC0000374: A heap has been corrupted (parameters: 0x00007FFBC52EB0E0).
The Common Language Runtime cannot stop at this exception. Common causes include: incorrect COM interop marshalling and memory corruption. To investigate further use native-only debugging.
Unhandled exception at 0x00007FFBC51BDAB5 (ntdll.dll) in testhost.exe: 0xC0000374: A heap has been corrupted (parameters: 0x00007FFBC52EB0E0).
The Common Language Runtime cannot stop at this exception. Common causes include: incorrect COM interop marshalling and memory corruption. To investigate further use native-only debugging.
Unhandled exception at 0x00007FFBC51BDAB5 (ntdll.dll) in testhost.exe: 0xC0000374: A heap has been corrupted (parameters: 0x00007FFBC52EB0E0).
The Common Language Runtime cannot stop at this exception. Common causes include: incorrect COM interop marshalling and memory corruption. To investigate further use native-only debugging.