Introduction
The new Kernel Transaction Manager (KTM) in Windows Vista allows you to perform a number of file and/or registry operations, and have them take effect in an indivisible ("all or nothing") fashion called a "transaction".
Although invisible to the end-user, adding transaction support to the Windows registry and file system was a significant step along the road to increased robustness, especially in the face of extreme circumstances (e.g. electricity failures). If you can't even "imagine" using a database that didn't support transactions, or wouldn't dream of formatting your disk with FAT instead of NTFS, then the KTM is for you …
Article Table of Contents
This article is broken into the following sections:
Definition of New Terminology
No new technology would be complete without adding a few more acronyms to the lexicon.
- TxF (Transactional NTFS) - TxF allows transacted file system operations within the NTFS file system.
- TxR (Transactional Registry) - TxR allows transacted registry operations.
- KTM (Kernel Transaction Manager). "The KTM enables the development of applications that use transactions. The transaction engine itself is within the kernel, but transactions can be developed for kernel- or user-mode transactions, and within a single host or among distributed hosts. …The KTM is used to implement TxF and TxR".
- RM (Resource Manager) – In the context of transactions on Windows, think of a resource manager as the system that makes sure a transaction works as promised (e.g. takes care of the rolling-back if the power fails). TxF, TxR and SQL Server are all examples of resources managers.
- DTC (Distributed Transaction Coordinator) – A system that coordinates resource managers, so you can use multiple resource managers in a single transaction (e.g. update a table in SQL Server and a file write using TxF).
- ACID (Atomicity, Consistency, Isolation, and Durability) - Not a new term, but relevant to the topic of transactions.
Overview of Transactions
The fundamental benefit that the use of transactions brings to the table, is reducing the number of possible outcomes you have to consider to two (i.e. complete success or total failure); while letting the transaction resource manager (RM) handle the difficult job of ensuring that your operations occur in an "all or nothing" fashion.
A Brief History of Transactions ...
The use of transactions originated in databases. As time has gone by, the use of transactions has slowly spread around the world of computer science:
- 1970s Databases – Databases have had transactions for decades, and were the original widespread use of transactions in computer systems.
- 1993 NTFS File System – Introduced in Windows NT 3.1, NTFS was a huge improvement in reliability over the previous FAT file system used by DOS and Windows 9x.
- 2007 KTM (see rest of this article)
- 2007 Transacted Web Services – As the reliability of your system decreases, the value of being able to use transactions increases. Because of the low reliability of inter-computer communication (compared to operations on a single computer), transactions are very important in this area. Windows Communication Foundation (WCF, previously "Indigo") brings distributed transactions in your Web service application and Windows.
- 20?? Transacted memory – If your database, registry, file system and web services all support transactions, then why not use transactions on your memory operations too? Although still in the research stage, transactional memory operations are an area of much interest today, as it is hoped that one of its benefits will be to alleviate some of the synchronization issues that make multi-threaded programming so difficult and error-prone. This was also the focus of an January 2006 MSDN Magazine article.
Should you use Transactions?
Transactions sound sort of interesting, but should I be using them?
In almost all cases, using transactions will increase the robustness to your software. Even if your software is completely free of bugs (just pretend) your application can still fail while executing any line of code (e.g. the electricity goes off or a poorly written driver crashes the entire system). Here are a few common scenarios that would benefit greatly from transactions:
- Windows Update and System Protection on Vista use transactions. This is especially important since a partial installation of systems components can make the entire OS unstable.
- Installers that use the KTM functionality will give a huge boost to the quality of the user experience, as partially installed applications are one of the worst user experiences. The application often starts up, but then you can experience strange problems at a later time.
- Save All - Imagine you were writing the MFC portion of Visual Studio. When the user hits the "Save All" button, you want to write the .rc file and the resource.h file at the same time, (since the .rc file references the contents of the resource.h file). This applies to any Document Management System with cross-referenced items.
In general, transactions are appropriate any time you are performing multiple registry or file operation, particularly if those files or registry entries reference one another. Essentially, transactions ensure that the data your application deals with (i.e. registry keys, files, etc) are left in a consistent state when something goes wrong.
Transaction Functionality Provided in Windows Vista
Windows Vista introduces a transacted file system and a transacted registry, allowing you to commit your changes to the file system and/or registry in one atomic operation. If anything interrupts the transaction (e.g. your application crashes or exits, the power fails, Windows crashes) while the transaction is in progress (i.e. a transaction that has been created, but not committed) the transaction will be rolled-back. At your discretion, you can also rollback the transaction within your code. You should also be aware that a transaction can be cancelled by the system at any time (e.g. if the system runs low on resources).
Essentially, a transaction is performed as in the following pseudo-code:
HANDLE hTrans = ::CreateTransaction(...)
::MoveFileTransacted(oldPath, newPath, ..., hTrans);
::CommitTransaction(hTrans);
::RollbackTransaction(hTrans);
An MSDN article lists all of the new transacted file functions. It also has a list of functions whose signature is the same, but the behaviour changes because of transactions (e.g. FindNextFile will work on the transacted "view" of the file system, if you started your file-finding operation with a call to FindFirstFileTransacted).
Resource Managers
In the context of Windows transactions, a resource manager is a "system" that provides the transaction support. A resource manager takes care of the roll-back procedure when things go wrong, and makes sure all commits occur atomically. TxF, TxR and SQL Server are all examples of resources managers.
Writing a resource manager is a very advanced topic, and is beyond the scope of this article. More information can be found in the MSDN articles Writing a Resource Manager and Programming Considerations For Writing Resource Managers.
The History of Transaction Support on Vista
You know those times when you've got a "brilliant" idea; you're all enthusiastic and start coding it up, and then an hour into coding you realise that although your idea will be very elegant for the simple case, it has the potential to cause big problems - so you hit "undo checkout" …
The original implementation of transactions on Vista (i.e. during development, when it was called Longhorn) followed an "implicit" model. Essentially, you created a transaction, called all of the file and registry functions you wanted, and then committed (or rolled back) the transaction, as in the following pseudo-code:
::CreateTransaction
::SetCurrentTransaction
::MoveFile(oldPath, newPath);
::CommitTransaction ::RollbackTransaction
Somewhere around Q3 of 1996, Microsoft decided to, figuratively, rollback their implementation of transaction support, and they moved to the current "explicit" model. In the explicit model, you call the create and (commit or rollback) functions as before. The significant difference is that the explicit model introduces a whole set of new functions, which are essentially duplicates of the file and registry functions, except they also take a handle to a transaction as a parameter.
Why the change?
According to Surendra Verma (Development Manager on the Vista Kernel team, in a post to the Channel 9 forum, Oct 27 2006):
"...Previously, you could have an ambient transaction and all your NTFS and registry operations would be a part of the transaction. That was a very easy model to get used to and start programming with! The problem with it was that it was also easy to accidentally include operations you didn't really intend to include in the transaction. This was particularly true programming at higher ends of the programming stack (COM+ and .Net) since many of those components themselves use the Windows registry itself, and it was very difficult for a programmer to know when/if lower level registry operations were getting included. So, we decided to make the model explicit..."
This author would also add that the implicit model would not have allowed you to have two transactions open simultaneously (which could certainly be the case if you were calling some library code).
Because this change was fairly late in the development cycle, the unfortunate side effect is that there are numerous outdated articles floating around the internet. (e.g. July 2006 MSDN article, Because we can blog, etc) The moral of the story? Take those "This article is based on a prerelease version of Windows Vista. All information herein is subject to change" warning banners at the top of the MSDN article seriously.
Isolation - with an Example
Just as in the world of databases, the effect of a transaction is easy to understand when there is only one process modifying the items covered by the transaction. When there are multiple processes attempting to access the object covered by a transaction, the rules for what happens gets slightly more messy.
With respect to interactions, it can be helpful to think of transactions as an advanced form of sharing permissions. Just as the way you open a file (e.g. with FILE_SHARE_READ and/or FILE_SHARE_WRITE) effects how other processes can access the file, so do transactions. Looking at an example may help clarify the situation...
In the following image, we can see the effect of our transaction, both within our own program, and interaction with outside processes...
Here are the operations, in order, that created the image:
- Before the image was taken, the file was named File0000.
- In the KTM Demonstration program, we created a transaction.
- In the KTM Demonstration program, we renamed the file. Note that even within our application, the transaction view of the Folder has the renamed file, whereas the non-transacted view still has the original file name. We retrieve the non-transacted view using the traditional function name(i.e.
FindFirstFileEx), and retrieve the transacted view using the transacted function name (i.e. FindFirstFileTransacted, passing in the HANDLE to the transaction we received when we called CreateTransaction in the previous step). - We opened the file in Notepad. Note that this operation succeeded. The contents of the file are the contents of file0000 before the transaction (i.e. we do not see the file renaming that is occurring within the scope of the transaction). If we were modifying the contents of file within the scope of the transaction, we would not see these changes either.
- We try to save the file in Notepad. This operation fails when we try to save (since the file File0000 has been locked by the transaction). If we had tried to save to another file name (e.g. file0001) the operation would have succeeded.
- In Windows Explorer, we attempt to rename File0000 to FileTryToRenameInExplorer. This operation fails, and explorer gives us an error dialog with the reason (i.e. the file has been locked by the transaction).
The Code...
New APIs are always subject to the chicken-or-the-egg problem. Because there isn't enough of an installed base of Windows Vista, making a separate "Vista build" won't be worth the trouble for most people, so the new functionality is not used for many years.
The code provided includes a C++ class KTMTransaction (in the files KTM.h and KTM.cpp) The goal of this class is to enable the best of both worlds. Your application will work on previous versions of Windows, but it will work "better" on Windows Vista. Without any changes to the code you write, the function calls in this class will form a transaction if the application is running on Windows Vista, and will not be transacted if run on earlier versions of Windows (which isn't THAT bad - it's how your code works right now anyway).
The class wraps all of the new transacted functions, and provides an interface that is identical to the existing Win32 functions you are familiar with. All of the functions in the class take the exact same parameters (in the same order) as the regular Win32 functions of the same name, so you can just use MSDN help for details on the parameters. (e.g. you just call the class's DeleteFile function and the class will call either ::DeleteFileTransacted or ::DeleteFile, as appropriate).
The class has no #include dependencies and all function calls to the transacted functions are done via LoadLibrary and GetProcAddress so that:
- You don't need Vista to run your application, and
- You can build your app without needing the latest SDK.
The code supports both UNICODE and ANSI builds.
The relevant public portions of the class are listed here:
class KTMTransaction
{
public:
KTMTransaction();
~KTMTransaction();
bool RollBack(); bool Commit();
HANDLE GetTransaction();
BOOL CopyFile( ... );
BOOL CopyFileEx( ... );
BOOL CreateDirectoryEx( ... );
BOOL CreateHardLink( ... );
HANDLE CreateFile( ... );
BOOL DeleteFile( ... );
HANDLE FindFirstFileEx( ... );
DWORD GetCompressedFileSize( ... );
BOOL GetFileAttributesEx( ... );
DWORD GetFullPathName( ... );
DWORD GetLongPathName( ... );
BOOL MoveFileEx( ... );
BOOL MoveFileWithProgress( ... );
BOOL RemoveDirectory( ... );
BOOL SetFileAttributes( ... );
LONG RegCreateKeyEx( ... );
LONG RegDeleteKey( ... );
LONG RegOpenKeyEx( ... );
};
Here is an example of how to use the class:
KTMTransaction trans();
trans.DeleteFile( "a.txt" );
trans.DeleteFile( "b.txt" );
trans.DeleteFile( "c.txt" );
trans.Commit();
What happens when the electricity comes back on? On Windows XP (and earlier) files a.txt and b.txt will be deleted, and c.txt will still be present. On Windows Vista none of the files will be deleted (the KTM will rollback the transaction on startup).
An example application is provided (built as a Visual Studio 2005 project). The application should run on versions of Windows before Vista (e.g. 2000 or XP), however the transaction buttons will not have any effect on these Operating Systems.
Links to other KTM Articles
Here are some great links for more information on transactions:
Article History
- 2007-03-07, Version 1.0, Initial release
