When a MessageBox in InitInstance Didn't Show

Once I created one utility which had the below piece of code:

BOOL CMyApp::InitInstance()
{
    AfxEnableControlContainer(); 
#ifdef _AFXDLL 
    Enable3dControls(); 
#else 
    Enable3dControlsStatic(); 
#endif 
    HANDLE hMutex = CreateMutex( 0, 0, _T("Some_Mutex")); 
    if( GetLastError() == ERROR_ALREADY_EXISTS ) 
    { 
        AfxMessageBox( _T("Only one instance of the application can be run")); 
        CloseHandle( hMutex ); 
        return FALSE; 
    }
    CMyDlg dlg;
    m_pMainWnd = &dlg; 
    int nResponse = dlg.DoModal(); 
    CloseHandle( hMutex ); 
    return FALSE;
}

My purpose was to run only one instance of the application. If more than one instance is run, the second instance will show a message box and exits. Before the release of the utility, I did some touch ups in the application. Fortunately just before the release I noticed that if two instances of the application are run, the second instance simply exits without showing that message box.

Then I started to roll back the touch ups one by one. After removing one of them, the message box showed again. And that touch up was the manifest file I added for giving XP look and feel. I searched a lot and finally found out that the problem is because of not calling the InitCommonControls() function. It was right there in the documentation Using Windows XP Visual Styles [^] , but I never followed it.

The actual purpose of the InitCommonControls() is described in The Old New Thing [^]. It says if we didn't call the InitCommonControls() function, our application will not be having any reference to the COMCTL32.DLL and therefore will not be loaded while my application starts. So when I tried to create a window, the class will not be registered and so the CreateWindow() function will fail.

However my doubt was, even without the InitCommonControls() function, my application dialog was showing correctly. Only the Messagebox had the problem. InitCommonControls() isn't necessary for the dialog? However when I stepped into the source code of the MFC, things became clear. Inside the function, it is calling the InitCommonControls(). The AfxDeferRegisterClass MACRO was doing this job.

Now I got another doubt, Since MFC calls the InitCommonControls() function, the COMCTL32.DLL will be statically linked to MFC42.dll and my application is statically linked to the MFC42.dll. So COMCTL32.DLL should definitely load in the beginning itself. But for some reasons, even though MFC42.dll was loaded, COMCTL32.DLL didn't get loaded.

The only possibility for such a scenario will be a delay load of COMCTL32.DLL in the MFC42.dll. To confirm, I opened the MFC42.dll in PEView. As expected the COMCTL32.DLL was added in the "DELAY IMPORT Address Table".

Let's look back to the original problem. "Message box got displayed in normal case even if I didn't call the InitCommonControls()". So COMCTL32.DLL is not needed if didn't use manifest?? The answer is no. In Windows, some window classes are registered by User32.dll(Like button, edit, etc.) and some other control (Windows common controls) are registered by COMCTL32.DLL. But if we add manifest files, user32.dll will not be registering any of the window classes, instead COMCTL32.DLL will do all the registrations. If you check the manifest file of the COMCTL32.DLL version 6, you can see the list of window classes versioned in it.

(The file is located in "C:\WINDOWS\WinSxS\Manifests\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.0.0_x-ww_1382d70a.Manifest".)

So to conclude, what I learned because of message box problem is:

1. Whether or not you are using Manifest file, you should call InitCommonControls() or InitCommonControlsEx().
2. MFC42.dll has set COMCTL32.DLL to delay load. ( But when I checked in MFC8.dll, the delayed loading of COMCTL32 is removed )
3. If we use manifest file, COMCTL32.DLL will be doing the registration of all System Classes instead of user32.dll. So it is necessary to ensure that COMCTL32.DLL has loaded before you create a window belonging to system class.