Introduction
"Prevention is better than cure." --Desiderius Erasmus
One of my fields of expertise is finding and eliminating memory leaks, and although there is plenty of information online about this topic, I thought I should share my own approaches and maybe add a thing or two to the general knowledge. When done on a small-scale app, leaks are fairly easy to diagnose using the regular windbg method (shown down here, among other slightly more nifty stuff), but on a large-scale this method tends to get harder and more complex, since there are a lot more types and more complex code, and discerning just which types to check and and which instance is leaking, in a tree of the same objects isn't as easy any more.
A quick overview
The problems:
- subscribing without unsubscribing
- 3rd party code controls and software
- not disposing of objects (exceptions, or just malcoding)
- cache collections and static collections
The solutions:
- Using windbg to find and solve particular issues
- Creation callstack insertion
- Memory profilers
- Using WeakRefs to plug specific holes
- Using WeakRefs in infrastructure to prevent your programmers from doing harm
- weakrefs in caching dictionaries
- weakrefs in infrastructure events
- weakrefs as barriers when wrapping 3rd party controls
1. Using windbg to find and solve particular issues
the conventional means of cleansing a program from managed leaks would be:
- download and install windbg (a win32 debugger from microsoft)
- run c:\program files\debugging tools for windows\windbg.exe
- attaching it to a fat process which has a memory leak (F6)
- then launching sos by writing:
- .loadby sos mscorwks (.net 2 - 3.5)
- .loadby sos clr (.net 4.0+)
- !dumpheap -stat -type
- find an interesting object in the list
- !dumpheap -mt
- pick one of the instances (save it's address)
- !gcroot
- this gives you all the roots which are holding a reference to the instance you are looking at, and a general sence where to change your code
A good example about this method can be found here. (he has an interesting extra bit at the end concerning event handlers). Well, after doing this for a while, you will get the sense that: "This method lacks an essential bit of info", this bit is something that will tell you which instance you are looking at and where it belongs in your program, this is usually supplied by MemoryProfilers in the form of a creation callstack - meaning the callstack at the point where the object was created (showing you who created it and where it belongs).
So now we know this, we are set to ask for this info in our windebugging session.
2. Creation callstack insertion
Inserting a creation callstack to an object of interest in your application's code is fairly simple, all you have to do is create a new string member like this:
#if DEBUG
string m_strCreationCallstack = Environment.StackTrace;
#endif
Then all you have to do to view this from Windbg is:
- !do <instance address>
- find the m_strCreationCallstack and copy it's address
- !do <found address>
- now you know where it came from...
3. Memory Profilers
There are several .NET memory profilers:
These are all commercial software and are not cheap, But they have some benefits... wrapped up in rich graphical interface, profilers retrieve massive amounts of information about your software, and some of them (like .net memory profiler) have helpful pop-up tip advice about where to look for the problems. On the other hand, these programs are heavy memory consumers, store a huge amount of info which is sometimes hard to navigate in (depends on the GUI) and attach with profiler API, which usually means running with your application from the startup. To sum it up: you will usually use profilers in the software dev env and not in production / preprod / test environments, so if a tester tells you he's got a leak, it's not the tool to use (back to windbg).
4. Using Weakrefs
Now, you may think "Well - this sounds very helpful" and it is, but it is also a bit dangerous, since an object which is held only by a weak reference can be claimed without warning by the GC and disappear (this can happen at any time as the GC is independent from the application and has its own rules). To prevent problems with this mechanism, we have the System.WeakReference object, which is a wrapper around the weakRef concept and lets us check IsAlive, to make sure the object still exists before using it.
Weakrefs can be used in several ways to prevent API users (your application programmers) from creating managed leaks (in case you were wondering this is the prevention stuff i was hinting at with the quote..), they can also be thrown in (as a last resort) when faced with difficult memory problems.
weakrefs & specific, hard to solve problems:
For example, in a project I worked on, we had UIActions (which were an implementation of the command design pattern simular to wfp's Command), these Action objects were initialized at the start of the application, to be used by all the buttons, menus and shortcuts in the GUI, these actions were ment to be thin objects with few logic lines for deciding if the button should be enabled and launching the action. As things go this didn't stay this way and before long programmers coded members into these Actions, this caused memory problems since keeping a member in a static object holds the referenced object tethered to a static (or in this case singleton object). the obvious solution was not to keep any members, but they were still needed in several places - so weak references were introduced, and IsAlive was added to IsNull checks.
Usage is pretty straightforward:
WeakReference m_objDataMember = new WeakReference();
public DataRow MyProperty
{
get
{
if ( m_objDataMember != null && m_objDataMember.Target != null)
{
object hold = m_objDataMember.Target;
if (m_objDataMember.IsAlive )
{
return (DataRow)m_objDataMember.Target;
}
else return null;
}
}
set
{
m_objDataMember.Target = value;
}
}
weakrefs wrappers in infrastructure:
One of the major problems in csharp applications
weakrefs wrappers in infrastructure:
One of the major problems in csharp applications (specifically GUI client side apps) is that subscribing (+=) to an event of an object will hold you in the memory until you either unsubscribe(-=) or the object calls Events.Dispose() or is removed from the memory (these cases are usually very close since Events.Dispose is traditionaly only be called from dispose), programmers will forget to unsubscribe. So we need to protect them.
Weakrefs can be used in two main ways on events:
A. On the registering side (+= WeakRefHandler) - It is possible to construct a proxy object I like to call a DelegateContainer which contains a function wrapping the requested delegate funtion, that forwards the call to a weakly held member object (which will be the original EventHandler you wanted to register)
B. On the registrator's side (the event publisher) - When you have ownership of the event, you can do something kinda neat: by overriding the add and remove functions and constructing your own event, which uses weak References to store the registering objects, you implement a real weak event. Registering to this event will not mean beeing attached to the memory tree in any way.
DelegateContainers
This method isn't prefect since you will need to create a different wrapper for each Event handler you want to wrap (MouseEventHandler != EventHandler) and don't even think of getting all "I'll build a greate new infra generic factory maggigy that will spew out these things by reflection" cause you are going down the rabbit hole, and you don't want to go there... In case you do want to explore that possibility, bare in mind that none of the solutions is perfect (easy API code /good performance/air tight solution) and look these up:
I think another example is in order: (this is the simple weakEvent and I tend to use it ,since there is usually one specific place
I want to plug using this method.)
public class DelegateContainer
{
private EventHandler m_handler = null;
private WeakReference m_weakRef = null;
private MethodInfo m_observerMethod = null;
private DelegateContainer()
{
m_handler = new EventHandler(callObserver);
}
private void callObserver(object obj, EventArgs e)
{
object hold = m_objDataMember.Target;
if (m_weakRef.IsAlive)
{
if (m_observerMethod != null)
{
m_observerMethod.Invoke(m_weakRef.Target, new object[] {obj, e});
}
}
}
private EventHandler initWeakDelegate(EventHandler handler)
{
m_weakRef = new WeakReference(handler.Target);
m_observerMethod = handler.Method;
return m_handler;
}
private void register(EventHandler caster, EventHandler handler)
{
m_weakRef = new WeakReference(handler.Target);
m_observerMethod = handler.Method;
}
public static EventHandler createWeakDelegate(EventHandler e)
{
DelegateContainer cont = new DelegateContainer();
return cont.initWeakDelegate(e);
}
}
Constructing real WeakEvents
The final method I want to show here is applicable when you own the event publisher class' code, this lets you construct a different backend for the event, thus creating a real weakEvent, to which subscribing is done normally (+=), and that will never hold your object in memory:
public override event ListChangedEventHandler ListChanged
{
add
{
object objTarg = value.Target;
int intHashKey = objTarg.GetHashCode();
if (m_colListChanged == null)
m_colListChanged = new Dictionary<int, KeyValuePair<MethodInfo, WeakReference>>();
lock (m_colListChanged)
{
if (!m_colListChanged.ContainsKey(intHashKey))
{
if (!m_colWeakRefs.ContainsKey(intHashKey))
{
WeakReference objWref = new WeakReference(objTarg);
m_colWeakRefs.Add(intHashKey, objWref);
}
m_colListChanged.Add(intHashKey, new KeyValuePair<MethodInfo,
WeakReference>(value.Method, m_colWeakRefs[intHashKey]));
}
}
}
remove
{
lock (m_colListChanged)
{
int intHashKey = value.Target.GetHashCode();
if (m_colListChanged.Keys.Contains(intHashKey))
m_colListChanged.Remove(intHashKey);
}
}
}
That's for collecting the event subscriptions, now for the event invocation code:
<span class="Apple-tab-span" style="white-space: pre;">
</span>
public void InvokeListChanged(object sender, ListChangedEventArgs e)
{
List<int> colCodesToRemove = new List<int>();
if (m_colListChanged != null)
{
List<int> keyList = m_colListChanged.Keys.ToList();
foreach (int code in keyList)
{
KeyValuePair<MethodInfo, WeakReference weakWrapper;
object objInvokeValue = null;
if (m_colListChanged.ContainsKey(code))
objInvokeValue = m_colListChanged[code];
weakWrapper = (KeyValuePair<MethodInfo, WeakReference>)objInvokeValue;
object hold = weakWrapper.Value.Target;
if (weakWrapper.Value.IsAlive)
{
weakWrapper.Key.Invoke(weakWrapper.Value.Target, new object[] { sender, e });
}
else
{
colCodesToRemove.Add(code);
}
}
for (int i = 0; i < colCodesToRemove.Count; ++i)
{
int code = colCodesToRemove[i];
lock (m_colListChanged)
{
if (m_colListChanged.ContainsKey(code))
m_colListChanged.Remove(code);
}
}
}
}
WeakReference m_objDataMember = new WeakReference();
public DataRow MyProperty
{
get
{
if (m_objDataMember != null && m_objDataMember.Target != null)
{
object hold = m_objDataMember.Target;
if (m_objDataMember.IsAlive)
return (DataRow)m_objDataMember.Target;
else return null;
}
}
set
{
m_objDataMember.Target = value;
}
}