List of contents
1. Introduction
1.1 Why to process requests in user mode?
1.2 Article structure
1.3 Technologies used
1.4 Project structure
1.5 Project directory structure
2. Windows and disks
3. Implementation
3.1 Stage 1. Initialization
3.2 Stage 2. Disk mounting
3.3 Stage 3. Processing requests
3.4 Stage 4. Unmounting
4. How to build this Solution
4.1 Testing
4.2 Supported Windows Versions
5. Conclusion
Introduction
This article is oriented to Windows-developers and considers how to create virtual disk in Windows system.
In Windows virtual disks are implemented by means of kernel mode drivers. Some necessary information concerning disk drivers for Windows is given in the section “Windows and disks”.
The implementation of virtual disk for Windows can also be found in the widely known project with open sources FileDisk (http://www.acc.umu.se/~bosse/ ). The main difference of the solution, which is given in this article, is that FileDisk processes requests to the virtual disk in kernel mode while the proposed solution processes them in user mode.
Why to process requests in user mode?
This approach can be useful when there is already developed code of user mode that provides access to the data source (there can be any data source, e.g. disk image in memory, remote disk, or cash register) and it’s hard to port this code to the kernel mode; or there is no source code at all, for example when performing network access or using specific encryption library.
As an example of such user mode library I used SparseFile (file-container with some data; as the data is accumulated it will size up to the maximal size).
Article structure
In section «Windows and disks» we will discuss the way the Windows interacts with the disks; possible variants of virtual disk creation; necessary information for virtual disk driver development.
In section «Implementation» the solution architecture is considered as well as key implementation aspects and main stages of our disk’s life cycle.
The section «How to build this Solution» contains a couple of phrases about how to build and test the project.
Technologies used
Project structure
Solution includes several projects:
- CoreMntTest (user mode, executable) – creates disk image and mounts it using the code from CoreMnt_user.
- CoreMnt_user (user mode, library) – receives requests to virtual disks from the CoreMnt driver and serves them.
- UsrUtils (user mode, library) – contains auxiliary code of the interaction with drivers using DeviceIoControl.
- CoreMnt (kernel mode, executable) – implements OS requirements for the disks; performs request transformation; sends them to CoreMnt_user for serving.
- drvUtils (kernel mode, headers only library) – auxiliary code of kernel mode, for example synchronization tools.
The scheme below represents the projects’ relations.
Project directory structure
.\bin - folder with binary files
.\lib - folder with library files
.\obj - folder with object files
.\src - folder with source files
|
|-> .\CoreMnt - Kernel mode driver.
|-> .\CoreMnt_user - User mode mount library.
|-> .\CoreMntTest - User mode mount test.
|-> .\drvCppLib - Kernel Library to develop driver in C++.
|-> .\drvUtils - Kernel Library with utils for kernel mode projects.
|-> .\mnt_cmn - Files that are shared between projects.
|-> .\STLPort - Directory with STLPort 4.6 ported for utilizing in windows drivers.
|-> .\usrUtils - Win32 Library with utils for user mode projects.
Windows and disks
If you are acquainted with driver development for Windows and have developed at least one even the simplest driver then you can skip this section.
I’m glad to appease the rest – everything is very simple here. Windows sends a request “write” or “read” to the disk. The disk returns read data or error code. And that’s all.
Surely there are some nuances, how it can be without them.
Let’s consider a simplified scheme of processing of requests to the disk. So, what is going on after an application calls, for example, ReadFile function. First the read file request is received by the file system driver (for example ntfs.sys). The scheme illustrates this process:
File system driver detects where the requested file is exactly situated on the disk (with what offset) and forms the read disk request. A file can be divided into parts, which can be located at different places on the disk. In this situation several requests will be formed. Just those very requests will be received by our virtual disk driver from the file system driver. And by the way, virtual disk can be also implemented at the file system level, see details in the article fs-filter-driver-tutorial.aspx .
Terms listed below will be used in this article.
- IRP (I/O Request Packet) is a structure of Windows kernel that stores request parameters. For example, if we want to read data from device we will need to indicate request type, buffer to read data to, size and offset. With some reservations we can say that IRP is a request to some device. At least in this article when we speak about IRP we always speak about a request. More details can be found here http://www.microsoft.com/whdc/driver/kernel/IRPs.mspx.
- STATUS_PENDING is a special return code that notifies the request initiator about that IRP cannot be processed right now and will be processed later on. There is a termination event for this situation, device will set it when complete the request processing. Below we will consider the code that uses this return code.
- Device is an object of Windows kernel representing any device. It stress an information about this device, for example its name. It also contains DeviceExtension.
- DeviceExtension is a field in Device structure, which can be used by the device creator in his own way. Below we will consider the code that uses DeviceExtension.
Implementation
The solution itself is a driver (CoreMnt.sys) and application (CoreMntTest.exe). The general scheme is as follows.
Driver provides the service of disk mounting. Application creates data source and mounts it as a disk using the service. Driver receives IRP, process them in user mode and returns the result. General scheme of driver work is represented on the figure.
Application (CoreMntTest.exe) serves the requests from OS to the virtual disks. Structural scheme is represented on the figure.
Now let’s consider how it looks in the source code, stage by stage.
Stage 1. Initialization
On this stage we start CoreMnt driver by means of the command:
c:\>sc start CoreMnt
We should create the management device in DriverEntry as an access point for CoreMntTest from user mode:
NTSTATUS DriverEntry(IN PDRIVER_OBJECT DriverObject,
IN PUNICODE_STRING RegistryPath)
{
...
NTSTATUS status;
status = IoCreateDevice(DriverObject,
0,
&gDeviceName,
FILE_DEVICE_NULL,
0,
FALSE,
&gDeviceObject);
if (status != STATUS_SUCCESS)
return STATUS_FAILED_DRIVER_ENTRY;
status = IoCreateSymbolicLink(&gSymbolicLinkName,&gDeviceName);
if (status != STATUS_SUCCESS)
return STATUS_FAILED_DRIVER_ENTRY;
Next step is the register the driver request handler. We will use a single handler for all request types:
for (size_t i = 0; i <= IRP_MJ_MAXIMUM_FUNCTION; ++i)
DriverObject->MajorFunction[i] = IrpHandler;
And finally let’s create MountManager:
gMountManager = new MountManager(DriverObject);
return STATUS_SUCCESS;
}
Stage 2. Disk mounting
On this stage we start application CoreMntTest.exe. It sends management message CORE_MNT_MOUNT_IOCTL to the driver:
CORE_MNT_MOUNT_REQUEST request;
request.totalLength = totalSize;
request.mountPojnt = mountPoint;
DWORD bytesWritten = 0;
CORE_MNT_MOUNT_RESPONSE response;
if(!m_coreControl.DeviceIoGet(CORE_MNT_MOUNT_IOCTL,
&request,
sizeof(request),
&response,
sizeof(response),
&bytesWritten))
{
throw std::exception(__FUNCTION__" DeviceIoGet failed.&);
}
Function DispatchMount deserialize request parameter and calls MountManager::Mount:
if(inputBufferLength < sizeof(CORE_MNT_MOUNT_REQUEST) ||
outputBufferLength < sizeof(CORE_MNT_MOUNT_RESPONSE) )
{
throw std::exception(__FUNCTION__" buffer size mismatch");
}
DISK_PROPERTIES diskProperties;
CORE_MNT_MOUNT_REQUEST * request = (CORE_MNT_MOUNT_REQUEST *)buffer;
diskProperties.totalLength.QuadPart = request->totalLength;
CORE_MNT_MOUNT_RESPONSE * response =
(CORE_MNT_MOUNT_RESPONSE *)buffer;
response->deviceId = gMountManager->Mount(&diskProperties);
In MountManager::Mount we create an object of the class MountedDisk and save it. MountedDisk includes LogicIrpDispatcher. Its constructor creates disk device. OS will sends requests to this device:
LogicIrpDispatcher::LogicIrpDispatcher(PDISK_PROPERTIES diskProperties,
PDRIVER_OBJECT DriverObject,
MountManager* mountManager)
{
...
status = IoCreateDevice(DriverObject,sizeof(InThreadDeviceExtension),
&deviceName,FILE_DEVICE_DISK,
0,
FALSE,&deviceObject_);
if (!NT_SUCCESS(status))
throw std::exception(__FUNCTION__" can't create device.");
After the device is created we have to initialize DeviceExtension. We want to use it for storing device identifier. Thus when we get IRP we will easily find the corresponding MountedDisk:
InThreadDeviceExtension* devExt =
(InThreadDeviceExtension*)deviceObject_->DeviceExtension;
memset(devExt, 0, sizeof(InThreadDeviceExtension));
devExt->mountManager = mountManager;
devExt->deviceId = diskProperties->deviceId;
So at the moment MountManager has created an instance of MountedDisk and saved it to the container. Initialization stage is finishing in the user mode. A thread is creates for each disk and all its requests are served there. A thread sends IOCTL RequestExchange to the driver and proceeds to the request awaiting mode:
while(true)
{
int type = 0;
int size = 0;
__int64 offset = 0;
drvCtrl->RequestExchange(deviceId, lastType, lastStatus, lastSize, &dataBuf[0], dataBuf.size(),
&type, &size, &offset);
DispatchImageOperation(image, type, size, offset, &dataBuf[0], dataBuf.size(), &lastStatus);
lastType = type;
lastSize = size;
}
Performance note: serving requests in one thread is surely the «bottle neck». There will be certainly a thread pool in the real project.
Stage 3. Processing requests
So, our virtual disk is ready to process requests. Let’s follow the complete sequence in request processing. It all begins with the function IrpHandler, which is registered by our driver as the procedure of IRP processing. Here we get the device identifier from DeviceExtension (we saved it there on initialization stage) and transmit IRP to MountManager:
NTSTATUS IrpHandler( IN PDEVICE_OBJECT fdo, IN PIRP pIrp )
{
...
InThreadDeviceExtension* devExt =
(InThreadDeviceExtension*)fdo->DeviceExtension;
return gMountManager->DispatchIrp(devExt->deviceId, pIrp);
MountManager receives IRP, finds the corresponding MountedDisk by device identifier, and redirects IRP to it. The code below decides if it is possible to process this request at once or it should be processed in the user mode:
NTSTATUS MountedDisk::DispatchIrp(PIRP irp)
{
IrpParam irpParam(0,0,0,0);
irpDispatcher_.getIrpParam(irp, &irpParam);
if(irpParam.type == directOperationEmpty)
{
...
irpDispatcher_.dispatch(irp);
...
NTSTATUS status = irp->IoStatus.Status;
IoCompleteRequest(irp, IO_NO_INCREMENT);
return status;
}
IoMarkIrpPending( irp );
irpQueue_.push(irp);
return STATUS_PENDING;
}
Decision making is simple: if it is IRP_MJ_READ or IRP_MJ_WRITE then it should be processed in user mode. Driver can process all other requests itself. For example IOCTL_DISK_GET_LENGTH_INFO: our driver knows the disk size, and it knows also that the disk size cannot be changed. The complete list of requests, which Windows can send to the disk, can be found in LogicIrpDispatcher::dispatchIoctl.
The thread that serves this disk selects requests from the list:
void MountedDisk::RequestExchange(UINT32 lastType, UINT32 lastStatus, UINT32 lastSize, char* buf, UINT32 bufSize,
UINT32 * type, UINT32 * length, UINT64 * offset)
{
...
NTSTATUS status = KeWaitForMultipleObjects(sizeof(eventsArray)/sizeof(PVOID), eventsArray, WaitAny,
Executive, KernelMode, FALSE, NULL, 0);
...
IrpParam irpParam(0,0,0,0);
irpDispatcher_.getIrpParam(lastIrp_, &irpParam);
*type = irpParam.type;
*length = irpParam.size;
*offset = irpParam.offset;
If it is IRP_MJ_WRITE, data to write will be copied to the buffer. Then this buffer will be passed to the user mode code:
if(*type != directOperationEmpty
&& opType2DirType(directOperationTypes(*type)) == directOperationWrite)
{
IrpParam irpParam(0,0,0,0);
irpDispatcher_.getIrpParam(lastIrp_, &irpParam);
if(irpParam.buffer)
memcpy(buf, irpParam.buffer, *length);
After returning from RequestExchange function we will get to the cycle of request processing (DispatchImage) again:
while(true)
{
int type = 0;
int size = 0;
__int64 offset = 0;
drvCtrl->RequestExchange(deviceId, lastType, lastStatus, lastSize, &dataBuf[0], dataBuf.size(),
&type, &size, &offset);
DispatchImageOperation(image, type, size, offset, &dataBuf[0], dataBuf.size(), &lastStatus);
lastType = type;
lastSize = size;
}
The variables type, size, offset now contain the new request to be processed. It’s a task for DispatchImageOperation function:
void DispatchImageOperation(IImage * image,
int type, int size, __int64 in_offset, char* buf,
int bufsize,
int* status)
{
switch(type)
{
...
case directOperationRead:
{
image->Read((char*)buf, in_offset, size);
*status = 0;
break;
}
case directOperationWrite:
{
image->Write((const char*)buf, in_offset, size);
*status = 0;
break;
}
After the request is served the function RequestExchange will be called again and the thread will proceed to the new request awaiting mode.
Stage 4. Unmounting
This stage starts in user mode with the call of UnmountImage function. Code below checks if the disk is being used at the moment:
void UnmountImage(int devId, wchar_t mountPoint, DriverControl * drvCtrl)
{
...
if (!DeviceIoControl(hVolume,FSCTL_LOCK_VOLUME,NULL,
0,NULL,0,&BytesReturned,NULL))
{
throw std::exception("Unable to lock logical drive");
}
else if (!DeviceIoControl(hVolume,FSCTL_DISMOUNT_VOLUME,
NULL,0,NULL,0,&BytesReturned,NULL))
{
throw std::exception("Unable to dismount logical drive");
}
else if (!DeviceIoControl(hVolume,FSCTL_UNLOCK_VOLUME,NULL,
0,NULL,0,&BytesReturned,NULL))
{
throw std::exception("Unable to unlock logical drive");
}
Then we destroy the connection of mounting point with our device:
if (UndefineLogicDrive(mountPoint))
throw std::exception("Unable to undefine logical drive");
And then we send a message for all those components that store the disk list in the system (for example explorer.exe or any other file manager):
SHChangeNotify(SHCNE_DRIVEREMOVED, SHCNF_PATH, root, NULL);
And finally we notify our driver that the device can be deleted:
drvCtrl->Unmount(devId);
}
MountManager::Unmount simply deletes the corresponding MountedDisk form the container that causes its destructor call:
MountedDisk::~MountedDisk()
{
We set the stop event for the thread of processing requests for this disk:
stopEvent_.set();
And we terminate all IRP, which were not served and are in a queue at the moment:
if(lastIrp_)
CompleteLastIrp(STATUS_DEVICE_NOT_READY, 0);
while(irpQueue_.pop(lastIrp_))
CompleteLastIrp(STATUS_DEVICE_NOT_READY, 0);
}
The thread of request processing, which has been in the MountedDisk::RequestExchange in the awaiting state, reacts on stopEvent_ set and throws an exception:
NTSTATUS status = KeWaitForMultipleObjects(sizeof(eventsArray)/sizeof(PVOID), eventsArray, WaitAny,
Executive, KernelMode, FALSE, NULL, 0);
if(status != STATUS_SUCCESS)
{
throw std::exception("MountedDisk::RequestExchange - mount stop.");
}
We will get the thrown exception in catch block of DispatchException function and return STATUS_UNSUCCESSFUL to the user mode:
NTSTATUS DispatchExchange(PVOID buffer, ULONG inputBufferLength, ULONG outputBufferLength)
{
try
{
...
gMountManager->RequestExchange(request->deviceId,
request->lastType,
request->lastStatus,
request->lastSize,
request->data,
request->dataSize,
&response.type,
&response.size,
&response.offset);
...
}
catch(const std::exception & ex)
{
KdPrint((__FUNCTION__" %s\n", ex.what()));
return STATUS_UNSUCCESSFUL;
}
}
Returned error state will be then processed by user mode code in DriverControl::RequestExchange function, and will also throw an exception in its turn:
void DriverControl::RequestExchange(int deviceId,
int lastType,
int lastStatus,
int lastSize,
char * data,
int dataSize,
int *type,
int *size,
__int64 * offset)
{
...
if(!m_coreControl.DeviceIoGet(CORE_MNT_EXCHANGE_IOCTL,
&request,
sizeof(request),
&response,
sizeof(response),
&bytesWritten))
{
throw std::exception(__FUNCTION__" DeviceIoGet failed.");
}
...
}
This exception in its turn will be caught by catch block in SyncMountmanager::mountDispatchThread:
void SyncMountManager::mountDispatchThread(void* pContext)
{
...
try
{
DispatchImage(dispContext->devId,
image->GetMountPoint(),
dispContext->image,
dispContext->mountManager->GetDriverControl());
}
catch(const std::exception& ex)
{
dispContext->mountManager->OnUnmount(dispContext->image, ex.what());
}
...
}
It will cause the request processing thread termination and also the IImage destructor call.
How to build this Solution
- Install Windows Driver Developer Kit 2003. (http://www.microsoft.com/whdc/devtools/ddk/default.mspx)
- Set global environment variable "BASEDIR" to path of installed DDK.
Computer -> Properties -> Advanced -> Environment variables ->System Variables -> New
Like this: BASEDIR -> c:\winddk\3790
- Download and install boost (tested with 1.40 version) (http://www.boost.org/users/download/ )
- Set global environment variable "BOOST" to path of installed boost. (You have to restart your computer after this.)
- Use Visual Studio 2008 to build the solution.
Testing
- Build the solution using the instructions above.
- Copy CoreMnt.sys to %windir%\system32\drivers.
- Register the driver in the system by the command:
sc create CoreMnt type= kernel binPath= system32\drivers\CoreMnt.sys
- Start driver by the command:
sc start CoreMnt
- Start CoreMntTest.exe.
If everything was ok then CoreMntTest.exe will display the message:
Image was mounted. Press any key for unmount.
Disk Z will appear in the system.
Now we can format it.
A file «tst_img» will appear in the directory along with CoreMntTest.exe.
Supported Windows Versions
Conclusion
In this article we considered only one from the possible approaches to the virtual disk creation. Described approach can be called logical virtual disk. The following approaches can be also mentioned:
- Virtual disk of file system. Driver receives requests
CreateFile, ReadFile, WriteFile and so on. - Physical virtual disk. This approach is semantically similar to the logical one but it has some differences at driver level. The requests to the virtual disk have other format, disk device should be PNP.
This article on the Apriorit website
