Debugging Release Mode Problems

Overview

If you're reading this you've probably just discovered that you've made a classic developer mistake. You've been working under the assumption that your working Debug mode application will work in Release Mode. Now you know that there are no guarantees that debug code works in release mode.

If you're like me you've probably discovered this after months of development without ever building or testing a Release Mode version. This leads to our first rule:

Rule 1: Always regularly test both the Debug and Release mode versions of your applications frequently.

The longer you wait to discover a release mode problem, the more difficult it will be to resolve. Build release mode versions often (at least once a week) and test them. This will save your behind if your schedule is tight.

Don't Remove Needed Code in Release Mode

This sounds pretty obvious, but is easy to do unintentionally. There are several macros that are commonly used that are completely removed when you compile in release mode. For example ASSERT macros and TRACE macros are quietly removed in Release builds. Any code that may be required, which was to be executed in an ASSERT or TRACE command will also be removed.

This leads to our second rule:

Rule 2: Be very cautious that you don’t place code that must be executed in a place where it will be compiled out in release mode.

Rule 3: Don't use Rule 2 as a justification to remove ASSERTs from your code. ASSERTs are an invaluable tool, but like any tool than can be used incorrectly. Use them with an appropriate amount of caution.

Making Debug Mode more like Release Mode

If your Release Mode problem is caused by code being unintentionally removed. It is very likely that you can reproduce the problem in debug mode.

Some problems can be caused by differences during preprocessor symbols used to compile Debug and Release projects. You can make your Debug compile more like a release compile by making the following changes:

• In Project Settings (Alt-F7) under the C++/C tab, set the category to "General" and change the "_DEBUG" preprocessor definition to "NDEBUG".
• Under the C++/C tab, set the category to "Preprocessor" and add the preprocessor definition "_DEBUG" to the "Undefined Symbols" edit field.
• Recompile using "Rebuild All"

If this recreates your problems in your Debug version then make the following changes to your code:

• Change all occurrences of ASSERT() to VERIFY().
• Look for any code you've placed inside of a "#ifdef _DEBUG" wrapper that might be needed in the Release version.
• Look for code in TRACE() must be executed. TRACEs like ASSERTs are compiled out in release mode.

If this fixes your problems in Debug mode, try rebuilding the Release mode version. It will very likely work this time.

Erroneous Assumptions

Are you assuming that your variables, and allocated data is initialized to some value (probably 0)? Are you assuming that all the resources that you've referred to in your code exist in your project? These bad assumptions can lead to problems between Debug and Release mode.

Rule 4: Never assume that your variables are initialized unless you've explicitly initialized them in your code. This includes global variables, automatic variables, malloced data, and newed data.

Rule 5: Make sure all references to resources are removed when you delete them from your resource file. This includes references in resource.h.

Variables and allocated memory are often initialized differently in Debug versions than in Release version of software. If you've assumed that your variables are zeroed when they are declared could easily cause strange behavior in Release mode on Win9x. You can also introduce bugs that are Win9x specific. WinNT, for security reasons, zeros all memory before it is used.

If you have references to resources that have been removed from your resource file, your debug version may work but your release version may crash.

Are You Listening to Your Compiler?

What is your compiler warning level set to? Often the warning level gets set to a ridiculously low level to reduce the noise in a build.

It's possible that by turning up the compiler warning level will expose your problem. I always set compiler warnings to either "Level 3" or "Level 4" depending on how masochistic I'm feeling. Go through and resolve every compiler warning (this is a very good idea to do before you release your code anyway). This can expose initialization problems and many other potential errors that may be causing your problem.

Rule 6: When you start a project turn the warning level to "Level 3" or "Level 4" and make sure your code compiles with no warnings before you check your code in.

Last Resorts

Debugging in Release Mode

I've heard a myth repeated many times by my fellow VC++ developers. It is that it is not possible to debug in Release Mode. Fortunately, this is a myth.

Rule 7: When all else fails try debugging in release mode.

It is possible to debug in Release mode. The first step is to turn on symbols:

• In Project Settings (Alt-F2) under the "C++/C tab" set the category to "General" and change the Debug Info setting to "Program Database".
• Under the "Link tab" check the "Generate Debug Info" tab.
• Recompile using "Rebuild All"

This will allow you to see the symbols in the release version. You might want to also consider the following:

• You may want disable your optimization settings when debugging your release version (though this isn't absolutely necessary).
• If you have trouble placing breakpoints, the command "__asm {int 3}" will cause your program to stop at that line (be sure to remove these lines from your program when you are done debugging).

Debugging in release mode has several limitations.

• The most annoying one is that you can't easily see into MFC because you don't have the symbols included for the MFC DLL.
• You must add symbols to the release versions of all libraries and DLL's you wish to debug in your release application.

Is the Compiler Generating Bad Code?

There is an outside chance that you've discovered a code generation problem in VC++. Frankly, folks often jump to this conclusion far too soon. To test this, simply turn off optimization in release mode.

If this fixes your problem you probably have used some questionable C++ coding practice. Believe it or not, it is possible to define C++ code that has some ambiguity in how it should be evaluated or that is just plain wrong but seems to work in some cases. For example this code will appear to work "correctly" in Debug Mode but not in Release Mode

#include <stdio.h>


int* func1()
{
&#9;int retval = 5;
&#9;return &retval;
}

int main(int argc, char* argv[])
{
&#9;printf("%d\n", *func1());
&#9;return 0;
}

I'm sure if you're creative or have read a few PC-lint ads that you could come up with similar examples.

Rule 8: If turning optimization off in Release Mode make your code work, assume that it is caused by poor programming practices. This means you should inspect your code for bad assumptions about stacks, pointers, and so forth. This also means that the fact your code works at all is probably dumb luck and should be fixed before it bites you later (probably when an angry customer has your superiors' attention over a mysterious program crash).

Rule 9: If you've inspected you code and found no problems, you should try turning on elements of the optimizer to help narrow down the culprit.

BTW- the problem code shown above is noticed by the C++ compiler. If you had followed Rule 6 you would have resolved this before it was a problem.

In my personal experience, I've rarely seen a problem that really was bad code generation. My experience with VC++ 6.0 has been that the compiler occasionally asserts while generating code (especially when using templates), but I've not experienced any problem with my code associated with bad code generation. Installing service release 3 seems to have taken care of all the code generation asserts that I've seen in previous versions.

Final Thoughts

I've found that by adding a bit of rigor to my daily routine, that I've been able to avoid most last-minute Debug -v- Release mode problems. This is what I do.

1. Check out files to be changed.
2. Make changes, resolve all warnings, and then build both a Debug and Release version.
3. Smoke-test both versions (this means I single-step through each new line of code, then I really try to break the code in the area that I've been working in).
4. Fix any problems found.
5. Check in my changes.
6. Do a complete build using all currently check in code (this makes sure my code is integrated properly with everyone else code).
7. Smoke-test my code again.
8. Fix any more problems found (this may require backing out the checked in code for awhile)

Doing this seems to have resolved most of my end-of-project integration surprises.