Introduction
This article will step you through adding a custom action DLL to a Windows Installer setup. In my opinion there is a fairly large learning curve on creating MSI files, so this article will go step by step, on how to create a DLL, and how to add it to an MSI file. Be warned, this is my first article submission. I typed up these step by step instructions as I was creating the sample. So if you open the sample project, you will see the finished result. The same is true for the sample MSI file.
Step 1 - Creating the Project in VC++
Create a new project, and give it a name. In my sample, I used "MyCustomAction" as the project name. Select "Win32 Dynamic-Link Library", and click Next. Now select "A simple DLL project". This will create a basic DLL that does nothing. You can compile it without any errors, no big deal.
Step 2 - Adding required #includes and linking the .lib
Open up the stdafx.h file, and add the following:
#include <msi.h>
#include <msiquery.h>
#include <stdio.h>
Open up the Project | Settings (Alt + F7) and add "msi.lib" to the Link tab's Object/Library Modules section. You will want to do the same for the release and debug versions of your project. Do a compile to see if you need to add the directory of the SDK files to Visual Studio. You may need to point Visual Studio to the \Include\ and \Lib\ directories of the SDK.
Step 3 - Exporting functions
Now you will need to add a .def file to the project which tells the compiler which functions the DLL will be exporting (making available to the Windows Installer). Go to File | New, and select a new text file, and give it the name of your project, but use a .def extension. Now you need to add some stuff to that file. Rather than going into the details of it, and how a DLL works, I will just show you what I put in mine:
; MyCustomAction.def
;
; defines the exported functions which will be available to the MSI engine
;
LIBRARY "MyCustomAction"
DESCRIPTION 'Custom Action DLL created for CodeProject.com'
EXPORTS
SampleFunction
SampleFunction2
Step 4 - Creating the functions
Now that we have exported 2 functions, we obviously have to implement them. You will have to open up the .cpp file for your project. To be able to call a function from the MSI, it must be declared/implemented using the following syntax:
UINT __stdcall YourFunctionName ( MSIHANDLE hModule )
I've made the mistake of leaving out the __stdcall and pulled my hair out trying to figure out why the DLL compiles fine, but would never get executed during the install. After creating the functions, you will probably want to add something to them so you can tell that they were actually executed. In my sample, I added a MessageBox to both the functions. In my MessageBox call, I set the parent window to NULL. I would not recommend doing this, but for testing purposes, I can live with it. The problem with this is you the message box can end up hidden behind the MSI dialogs. If you needed to show a standard message box in a custom action, and keep it on top of the installation dialog (child window), you would need to implement it differently. If there is demand for it, I can type up another article that describes how to do that, using INSTALLMESSAGE_USER. At this point my .cpp file looks like:
#include "stdafx.h"
BOOL APIENTRY DllMain( HANDLE hModule, DWORD ul_reason_for_call, LPVOID lpReserved )
{
return TRUE;
}
UINT __stdcall SampleFunction ( MSIHANDLE hModule )
{
MessageBox(NULL, "Hello world", "CodeProject.com", MB_OK);
return ERROR_SUCCESS;
}
UINT __stdcall SampleFunction2 ( MSIHANDLE hModule )
{
MessageBox(NULL, "Hello world", "CodeProject.com", MB_OK);
return ERROR_SUCCESS;
}
At this point you should be able to compile the program without any errors. You may have some un-referenced variable warnings, but that is it.
Step 5 - Editing the MSI
So hopefully you have an MSI already. If not, you can create a simple MSI using a number of tools such as InstallShield, Wise, or the FREE Visual Studio Installer from Microsoft. It doesn't matter which one you use to create the MSI, just make sure the MSI successfully installs and uninstalls. FYI, I created mine with InstallShield, then edited it so I wouldn't have to click through all the dialogs each time I test it. Mine does not install any files either, it just creates a regkey. For a good comparison between the editors available, visit www.installsite.org.
If you have not already installed Orca, you need to do so now. Orca is a MSI database editor that is included in the Platform SDK. I'm not sure where it is, but just search for Orca, and you should find Orca.msi somewhere in the SDK directory. Double click it to install. Now you can right on any MSI file and edit its contents. Also, another little tip, you can right click the MSI to uninstall it. This will save you time running to the add/remove applet after each test.
Binary Table
So, open your simple MSI up in Orca. I'm not going to explain what each table does, because that would take a week or more (thus the learning curve of Windows Installer that I mentioned before). So, on the left, click on the "Binary" table. This is where you can embed files into the MSI file. Add a new row to the binary table, and give it a Name. I used "CustomDLL". In the Data field, you will need to point it to the DLL you just compiled. Enter the path, or use the browse button to locate it.
CustomAction Table
Now navigate to the CustomAction table (click CustomAction on the left pane), and add a new row. It needs a name, which goes in the Action column. I used "CustomAction1". Set the Type = 1. The Source will be a foreign key back to the binary table. Since I called my entry in the binary table "CustomDLL", that is what I would enter for the Source. For the Target field, you will need to enter the name of the function in the DLL. So I entered "SampleFunction" in the Target column.
InstallUISequence Table
Those of you who are familiar with the Windows Installer will know that you can call the custom action practically anywhere. You can associated it with a button click, or as I am going to do, have it executed during the UI sequence. Navigate to the InstallUISequence table. Add a new row to this table, and for the Action field, this will be a foreign key to the CustomAction table, so enter the name of your CustomAction. In my case it was called "CustomAction1". For the Condition field, I am going to leave it blank. If you wanted this to only execute during an install, you can enter "Not Installed" as the condition. Now you need a sequence number. The sequence number determines what order these actions will occur. I suggest you find the Welcome Dialog (it may be called InstallWelcome, or practically anything). In my case its called InstallWelcome and it has a sequence of 650. So I want the Custom Action to take place before this, so I sorted by the Sequence to see what else came before it. I also wanted it to take place before the maintenance dialog, so I gave my new action a sequence number of 601.
Just so you are aware, adding our custom action to the InstallUISequence does not guarantee that it will be executed. If the user does a silent install, or uninstall, it will never get executed (nothing in the InstallUISequence gets executed during silent installs/uninstalls).
Step 7 - Test it!
Now save and close the MSI file. If all went as planned, when you double click the MSI, there should be a standard Windows Installer progress window, maybe a splash screen for your MSI, and finally a message box should pop up. Presto, you are inside the DLL! Programmatically, you can do anything you want inside the DLL. You can check to see if you application is running (via what ever method you want: FindWindow, CreateMutex, etc.). You can copy regkeys from your old product's registry key to the new version's. The possibilities are endless! I leave it up to you what you decide to do. Hopefully you did not get one of those cryptic Windows Installer error messages. Usually they have a 4 digit error code associated with them. If you got one, open and search through the Msi.chm to find a slightly longer explanation of the error.
Debugging
So the next question you will probably have is how to debug the DLL. Well, in my opinion, its kind of a pain. The process is described in the MSI.chm (Platform SDK). However I will provide you with an alternative to debugging.
Logging the MSI
Included in the source code is a .reg file. Double click this, and it will add the special registry key that will log every MSI install you ever run, even through the add/remove applet. The log files will show up in your %temp% folder, but the name of it is different each time, so sort by date and you should find it. This log file shows all the actions that the Windows Installer performs. You should be able to search for your custom action name, "CustomAction1" in my sample, to see where it got executed. It sure would be helpful to have some more logging show up there, rather than the time stamp of when the function call began and ended. That's where the next section comes into play.
Logging from the DLL
In my sample, I included 2 additional files. MSI_Logging.h and MSI_Logging.cpp contain a single function that will write a string of text to the log file. Just add these two files to your project, and add the #include "MSI_Logging.h" to your .cpp file, and you should be all set. In the example, you can see how I grabbed the Product Name from the MSI, and then formatted that into a friendly string to be written to the log file.
Conclusion
Hopefully this helps you get started in the world of MSI setups. As you can see, its just a DLL, so you ca do practically anything you want. As mentioned before, if there is demand for it, I can provide more examples relating to MSI setups, which is sort of my specialty.
History
28 Apr 2002 - Added an MsiMessageBox function for modal message boxes.
