Using a Win32 Mutex to Figure Out if an Application Instance is the First One

• Download source - 22.2 KB
• Download EXE - 1.3 MB

Introduction

This question came up in the Microsoft discussion forums and while the answer is relatively simple (use a mutex), the practical implementation requires some understanding of how Windows does things. I made an implementation myself out of curiosity, and decided to share my results.

My implementation uses the win32 API directly for the mutexes. For the rest, I am using the win32 helper library which I am extending with every article just to make life easier because every article uses some functionality of the previous article. The sources are included with this article and as soon as I can do some final cleanup, I'll be opening it on Github.

Background

When implementing mutexes, the idea is simple. We don't actually use the mutex for signalling. Instead, we use it because it is an object that is created in the kernel namespace. And when it is created, the API tells us if it was newly created or already existing. There are a couple of details that matter.

Console Sessions

When a user logs on to Windows, they get assigned a session ID. At any given time, there can be multiple users active on the system. Whether they log in remotely, or directly on the actual physical console is just a technical detail. If we are looking as the requirement of 'is this the first instance', we have to look beyond just our own session, but into other sessions as well. The API for getting the current session ID is WTSGetActiveConsoleSessionId.

The Kernel Namespace

kernel objects live in the kernel namespace. They have a unique name that cannot be reused for another kernel object in the same namespace. The namespace is divided in 2 parts: a global namespace and a local namespace. The global namespace is shared among all user sessions. The local namespace is unique to each session.

Say we're making a mutex named 'bob'. Then 'Global\bob' would be the name of mutex bob in the global namespace. 'Local\bob' would be the name of mutex bob in the local namespace. Note that these names are case sensitive and need to be spelled as shown. If we just use 'bob', then this is automatically assumed to be a local name and placed in the local namespace.

Creating a Mutex

A mutex is created with the API call:

HANDLE CreateMutexA(
  [in, optional] LPSECURITY_ATTRIBUTES lpMutexAttributes,
  [in]           BOOL                  bInitialOwner,
  [in, optional] LPCSTR                lpName
);

For our purposes, the name is key to identifying our application. We'll discuss the name in detail later. This could be 'Global\bob' for example. bInitialOwner specifies if -when creating the mutex- the thread that creates it is the initial owner of the mutex as well. We don't care about that because we don't use it for signalling.

Suppose we use this call and successfully create a mutex. A subsequent call to GetLastError will tell us if the mutex was newly created (NO_ERROR) or if a handle was opened to an existing mutex (ERROR_ALREADY_EXISTS). That is already half the answer to the question: 'is this the first instance of the application'.

lpMutexAttributes is important mostly for objects in the global namespace. Any mutex that is created without specific security attributes gets the default security descriptor. For objects in the local namespace, that usually doesn't matter because the applications using that mutex are all running with the same user token.

However, for objects in the global namespace, this is not true. If a process running as LocalSystem creates a mutex in the global namespace without specifying security attributes, other processes may not be able to open a handle to that mutex because of security restrictions. For our purposes, this doesn't matter because that failure provides information. If we cannot open a handle to a mutex with that name because of a security restriction, we also know that we were clearly not the first process trying to create it.

Choosing the Mutex Names

With these two namespaces, we can easily determine if the application is the first in the current user session, and if it's the first on the computer as a whole.

Naming a mutex 'bob' isn't a terribly good idea though. It's not inconceivable that another application on the computer would use the same name. Ideally, a mutex has a unique name. Luckily, we have something for that: GUIDs. A GUID is guaranteed to be unique.

However, that's not enough. In some cases, you may want to prohibit the application from running multiple instances concurrently, except if the application is a copy of the application in another location or with another name. For example, if the application is c:\temp\myapp.exe, then in some scenarios, you may want to also allow c:\temp\copy_of_myapp.exe to run next to it. So optionally, we need to consider making the module path part of the mutex name.

The problem is that this way, we get potentially very long mutex names. And '\' is not an allowed character in a mutex name. There is a very easy solution though: we simply push all that information (the GUID and the full module path) through a hash algorithm, and use the resulting hash hex string as a mutex name.

The CAppInstance Class

The implementation is done in a class with the following declaration:

private:
    CHandle m_LocalMutex;
    CHandle m_GlobalMutex;

public:
    CAppInstance(
        LPCWSTR instanceName,
        bool allowCopiesToRun);
    ~CAppInstance();

    bool IsFirstInSession();
    bool IsFirstOnComputer();
};

There isn't a whole lot to say about it, except that upon construction, we initialize two mutexes which can then be examined later to determine if the process was the first instance of that application. CHandle is a class that wraps an actual HANDLE just for the purpose of making sure it is closed eventually and to check if it is a valid handle or not.

Determining the Names

This is where the heavy lifting is done. First, we figure out the names we want to use:

wstring globalMutexName = L"Global\\MUTEX_";
wstring localMutexName = L"Local\\MUTEX_";

CBCryptProvider hashProvider;
CBCryptHashObject hashObject(hashProvider);

//Guarantee global uniquess by hashing the GUID
AddDataToHash(hashObject, wstring(instanceGuid));

//If copies are allowed to run but not the same image on disk,
//then we add the module path to the hash,
//guaranteeing a uniqueness per each copy of the image.
if (allowCopiesToRun) {
    TCHAR modulePath[4096];
    DWORD numChars = GetModuleFileName(NULL, modulePath,
                                       sizeof(modulePath) / sizeof(TCHAR));
    if (0 == numChars) {
        throw ExWin32Error();
    }

    AddDataToHash(hashObject, wstring(modulePath));
}

wstring name = hashObject.GetHashHexString();

globalMutexName += name;
localMutexName += name;

We need to create two names: one in the Global namespace, the other in the Local namespace. Because the final name will be an unreadable hex string, we use a 'MUTEX_' prefix. Should we be troubleshooting later on and viewing named objects in the namespaces, at least we know we're looking at a mutex.

In order to create a hash, we use the CBCryptProvider and CBCryptHashObject classes which were Implemented in a previous article. The exact hash method doesn't matter so it defaults to BCRYPT_SHA256_ALGORITHM, and we don't have to bother with a hash secret either because we only use it as a glorified statistically unique checksum.

First, we add the GUID to the hash. If we only want to permit a single instance to run, regardless of whether it is copied on diks or not, that is where we stop. No matter where the image is started from, if the mutex is made with just that GUID, there can be only 1.

Now if we want to allow only 1 instance of a specific image on disk, but are fine with running, e.g., c:\temp\myapp.exe and c:\temp\copy_of_myapp.exe, then we simply push the module path into the hash. That will guarantee a unique name for that GUID, per module.

Detecting the Instance

Previously, we discussed that if we create a mutex handle, three things can happen:

1. It is created without special error information. In that case, it didn't exist before.
2. It is created and the error status is ERROR_ALREADY_EXISTS. That means it already existed.
3. There was an error and the mutex handle is not valid. That also means it already existed.

DWORD retVal = NO_ERROR;

m_GlobalMutex = CreateMutex(NULL, TRUE, globalMutexName.c_str());
retVal = GetLastError();
if (m_GlobalMutex.IsValid() && ERROR_ALREADY_EXISTS == GetLastError()) {
    m_GlobalMutex.CloseHandle();
}

m_LocalMutex = CreateMutex(NULL, TRUE, localMutexName.c_str());
retVal = GetLastError();
if (m_LocalMutex != NULL && ERROR_ALREADY_EXISTS == GetLastError()) {
    m_LocalMutex.CloseHandle();
}

At the end of that segment, having a valid mutex for a given scope will be evidence that the instance was the first in that scope.

If the application wants to use that information to make a decision, it can use the following methods:

//Is this the first instance in this particular session?
bool CAppInstance::IsFirstInSession() {
    return m_LocalMutex.IsValid();
}

//Is this the first instance on this particular computer?
bool CAppInstance::IsFirstOnComputer() {
    return m_GlobalMutex.IsValid();
}

Using the CAppInstance Class

Using the class is trivial:

try {
    CAppInstance g_AppInstance(L"406B6F5D-4A7B-43A7-8CF8-1E44B3C938BE", false);

    wcout << L"Started process as user " << w32_GetCurrentUserName()
        << L" in session " << WTSGetActiveConsoleSessionId() << endl;
    cout << "App is first in session: " << g_AppInstance.IsFirstInSession() << endl;
    cout << "App is first on computer: " <<
             g_AppInstance.IsFirstOnComputer() << endl;
    cout << "Press any key to end the application." << endl;
    string input;
    getline(cin, input);
}
catch (exception& ex) {
    cout << ex.what() << endl;
}

For our application, we use GUIDGen to create a GUID and paste that as instance name. We also indicate that we do not want copies of the application on disk to be identified as different applications.

Note that the format of the GUID doesn't matter. It all goes into the hash algorithm. As long as the GUID is in there, any whitespace or special characters do not detract from the 'unique-ness'.

When I tested the program, I first ran it as admin:

And then two times as myself:

Points of Interest

Aside from having the option to restrict instances this way, there is the possibility for implementing additional restrictions no matter how farfetched they seem. It all depends on what we push into the hash. If we add the year and the day of the year, then we can implement a policy where one instance can start per day of the year and multiple instances can run concurrently as long as they start on different days.

It wouldn't be very useful of course, but I just wanted to indicate that you can define any sort of uniqueness that you want and make runtime decisions based on that definition of uniqueness.

It should be noted that this sort of construction could potentially lead to a vulnerability. The global and local namespace can be inspected and if an attacker knows the name of the mutex that is being used, it could potentially lead to a situation where they create the mutex in order to prevent your application from running (denial of service). Some mitigation is possible but after the first time a mutex with that name has been created, it is no longer guaranteed to be a secret.

History

• ^10th November, 2022: Added note about possible denial of service.
• 21^st October, 2022: First version