Introduction
This code snippet can be used to detect and free memory leaks in C programs.
It isn't suitable for C++ in its present form. It is fast and introduces little
overhead in the way of extra memory, especially if thread contention is rare.
This means it is suitable for use in release builds to auto free memory leaks in
C DLLs and programs that are written by others. This avoids the
problem of possibly introducing new bugs as a result of editing the code to fix
the leak.
In diagnostic mode it reports the line number and file where the leak
occurred. This adds an extra 22 bytes on average to each memory allocation.
Since MSVC can do this already, and probably most compilers, I suppose the main
advantage here is that you can customize the code, - also you can use it with
release builds to test them for memory leaks too, in case where the release
differs significantly from the debug build.
In auto freemode, you can call it to free any memory allocated, that got
allocated. In this case it adds 10 bytes on
average, to each memory allocation. This is probably most useful for a DLL,
- if the allocations are done separately for each call of a routine in the DLL, you could set it to auto
free memory when the routine in the DLL returns.
It is implemented for malloc, realloc,
calloc, and strdup. It would be easy to extend it to
the rarer memory allocation routines such as _expand, and the wide
string versions of strdup.
Note
You don't want to mix this library with ordinary uses of malloc and free because
when it gets given a pointer to free, the first thing it does is to decrement the pointer and access the decremented pointer to find the record number (this technique is what makes it so fast). This will cause an access violation if the pointer was originally allocated by normal use of
malloc. The other way round, if you try to free one of its pointers using normal free then because the pointers are incremented before they are passed to the calling app, you will get errors there too.
Every allocation made using this library needs to be freed using it, and it should only be used to free memory that it allocated itself. However, you can use it for a particular sections of your source code by including its header for those sections only. Alternatively, define code_uses_redefined_malloc and then replace malloc by _malloc_, calloc by _calloc_, free by _free_ and strdup by _strdup_ throughout the section where you want to use it.
Also be sure to note - it is for pure C programs which rely on use of malloc etc. for memory allocation. It doesn't call Dtors, nor is it appropriate for use with new.
Background
Memory leaks are an issue in programs and dlls that run 24/7. This library may be useful to auto free memory in such cases. It could be useful particularly if the dll is a conversion of a program that was originally written with reliance on the operating system to auto free it when the program exited. You just need to include the header and there is no need to edit the source code and potentially introduce new bugs. It can also be used for memory leak diagnostics to locate the places where the leaks occurred.
Using the code
How to use it - general
You just use your memory allocation routines as usual - and then when you
have freed all the memory you are going to free, call
MemLeakCheckAndFree()
This will show a diagnostics report if it found any pointers still
unallocated - and will free them. If you switch off diagnostics then this
routine is the one that does the auto free for you.
In diagnostic mode, the warnings are written to stderr as is
useful for a console app. However if you are using it with a Windows program you
would make your own callback routines to handle the warnings. Example callbacks,
which you can use for this are included within comments in the header:
SetMemLeakWarnProc(MemLeakWarnProc);
and
SetMemLeakLogProc(MemLeakLogProc);
The log proc is used by MemLeakCheckAndFree since a potentially
long list of warnings may need to be shown - typically one might write those to
a file. Here is what its output might look like:
************************************************************
2 PM Saturday, July 12, 2003 GMT Daylight Time
************************************************************
Memory Leak!
File: I:\Lissajous\main.c, line 46, size 100
File: I:\Lissajous\main.c, line 248, size 1000
It also calls MemLeakWarnProc once at the end, to alert the user
that there has been a memory leak.
To use it as a DLL
Link with the relevant DLL, either MemLeakCheckt.dll for diagnostics,
or MemLeakCheckat.dll to auto free only (add the relevant .lib
file to your linker options).
Now include the header after all your standard includes and before your own
source code where the allocation and free occurs.
In the header, define
#define LINK_MEMLEAK_CHECK_AS_DLL
Then, if you are linking with MemLeakCheckt.dll (the one with the
diagnostics), you need to define
#define cl_mem_leak_check
#define cl_mem_leak_diagnostics
or for MemLeackCheckat.dll, just define the
cl_mem_leak_check
Since you may very well use a different version of the C run time library
from the DLL (whether single or multithreaded, release or debug), it might be an
idea to add this code at the end of one of your source files after all the
memory allocations, where the #undefs won't affect them. Then call
it when your app starts, to set the memory allocation routines for the DLL to
the ones for your CRT:
#ifdef cl_mem_leak_diagnostics
#undef malloc
#undef calloc
#undef realloc
#undef memcpy
#undef free
#include "malloc.h"
void init_memleakcheck_alloc(void)
{
set_mlc_malloc(malloc);
set_mlc_calloc(calloc);
set_mlc_realloc(realloc);
set_mlc_free(free);
set_mlc_memcpy(memcpy);
set_mlc_strlen(strlen);
set_mlc_strcpy(strcpy);
}
#endif
I don't know if this step is necessary. It is needed in the case of file
pointers as if you use a file pointer in a library that uses a different version
of the CRT from the calling app, then it is no longer valid and will probably
cause an access violation, and this method can fix the problem in that case. But
the malloc and free pointers seem to work okay in the calling app.
Even if you don't do it. I think it is probably safest though, to do it like
this. Any information about this appreciated :-).
If you want to use it to auto free memory in a DLL each time one of the
routines exits, then I think it is best to include it as source code and to use
thread local variables - see next section.
To use it as source code as part of your project
Add the source code to your project, and add the header after all the
includes and before uses of malloc etc. This header will redefine
malloc, realloc, calloc, and
strdup so that the versions in MemLeakCheck get used
instead of the standard ones. These then in turn call the standard
malloc, etc. to do the actual memory allocation.
As before, define:
#define cl_mem_leak_check
#define cl_mem_leak_diagnostics
and make sure you comment out the two definitions that are needed to build it
as a DLL or link to it as a DLL.
If you want to use it to auto free memory in a DLL each time one of the
routines exits, then it is probably best to use thread local variables - in case
several threads simultaneously want to use the same routine.
When you use thread local variables, each thread has to call the autofree
routine separately. In the case where you free the memory every time a
particular routine returns, then this is easy, as each time it will only free it
for the particular thread it was called by.
To use thread local variables define this in the header:
#define MLC_USE_THREAD_LOCAL_VARIABLES
Points of interest
The idea is to use the first few bytes of each allocation to hold the
information you want for the allocation - in diagnostic mode it has a pointer to
the static string for the file name, and the line number. In both modes it also
has a record number which is an index into an array of pointers.
Then when memory needs to be allocated, you allocate the memory needed plus a
bit extra for this extra information, and return a pointer incremented to point
at the data itself at the end of the record. When the memory needs to be freed,
you decrement the pointer given to you by the calling app in order to find its
record, and the record number.
The special trick used to make it fast is the addition of the record number
here. When memory needs to be freed, one can go straight to the relevant entry
in the array of pointers, with no need to look it up or do any kind of a search.
Then when you free the memory, you set the entry to NULL to
indicate that the pointer is no longer in play. From time to time you need to
compact the array of pointers - just walk through it from the start and move all
the pointers that are not null back, and access them and update their record
numbers as you do so.
To deal with thread contention, since this is expected to be rare, it uses
InterlockedExchange in combination with an efficient busy wait (surrendering the
rest of each time slice using Sleep(0) until access is obtained).
This is very fast, just a single assembly language instruction to execute to get
access to the section, so better than critical sections.
In the case where contention does occur frequently, then this means that one
of your threads is repeatedly calling Sleep(0) while the other
allocates memory. This shouldn't have too much of a performance impact - the
repeatedly sleeping thread can't do anything until it gets the memory anyway. It
shouldn't affect other programs that may be running at the same time either, as
it only uses a few assembly language instructions per time slice. So - I can't
see that this will have much performance impact - please correct me in this if
needed and I'll be interested in information about how it performed in such a
situation if anyone tries it for an app. with high thread contention for memory
allocation. If it was found to have a performance impact, then one solution
would be to use thread local variables, and do the auto free separately for each
of the contending threads.
The memory and time requirements are both low. However from time to time it
compacts the array of pointers by traversing the array. This involves inspecting
each entry in an array of n pointers where n is at least 1000, and up to 1.5
times the number of allocations you have in use and not yet freed. So I think it
isn't much of a time penalty in most situations.
The time needed to compact the array of pointers may be significant if one
has tens or hundreds of thousands of small allocations in play at once, and if
the app. is also highly time critical. In this case one might want to add a
routine to the library in order to temporarily suspend the array compaction
whenever one is doing time critical work, and resume it again at an appropriate
later occasion (an easy mod). Perhaps there is little one can do to speed it up,
since the idea of doing things this way was to make the allocation and frees,
immediate with no need even for as much time penalty as is needed to insert an
entry in a binary tree, which I think is more important than eliminating the
array compaction step. Then to compact the array I hardly see how one can
improve on a simple array traversal from the start, without adding to memory
requirements or slowing the addition or removal of elements to it. Again please
be sure to correct me if there is a way to improve on this - I'll be delighted
to hear of any improvements.
For more information see the Mem Leak
Check home page.
History
- Update
Updated the description to correct earlier version. I thought that memory needed to be freed
whenever you exit from a program, but it seems windows does it automatically - see the discussion.
Added a note about its use for just a particular section, as one might use it in 24/7 type dlls.
- First release
