Introduction
This article deals with a subject which is comparatively less discussed. People new to the programming world mostly start with samples which create, read, or write files. In Windows, all file related calls finally end in CreateFile, ReadFile, or WriteFile. We are all familiar with these APIs. Technically, the ReadFile and WriteFile APIs are synchronous APIs by default (when the OVERLAPPED parameter is not specified). That is, these APIs return only after the requested data is read or written (by default). Now, let's come to the article's topic. There are purely asynchronous versions of these APIs: ReadFileEx and WriteFileEx. Actually, ReadFile and WriteFile can also behave asynchronous. But our discussion is going to be on the explicitly asynchronous APIs, ReadFilEx and WriteFilEx.
Background
Normally, when we need asynchronous behaviour or parallelism, we create threads. The asynchronous file I/O APIs allows us to leverage asynchronous behaviour without introducing threads. So we can issue a ReadFileEx or WriteFileEx and perform other operations. Then finally, when the application needs the result of I/O, it can check the status of the asynchronous operation with the help of some APIs which we will discuss shortly. The file should be opened with FILE_FLAG_OVERLAPPED for using the asynchronous APIs.
See the prototype for ReadFileEx below. It is the same for WriteFileEx.
BOOL WINAPI ReadFileEx(
__in HANDLE hFile,
__out_opt LPVOID lpBuffer,
__in DWORD nNumberOfBytesToRead,
__inout LPOVERLAPPED lpOverlapped,
__in_opt LPOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine
);
Here, we are interested in the last two parameters: lpOverlapped and lpCompletionRoutine. We need to specify the OVERLAPPED structure filled with offset from where the data is to be read or written. There is an hEvent member in this structure. The MSDN documentation says that we are free to use this for our purposes. But there is asynchronous behaviour exhibited by ReadFile and WriteFile when its last parameter OVERLAPPED pointer is provided. In this case, some MSDN documentation says that we need to supply separate event handles for each instance of the OVERLAPPED structure. If we check the documentation for FileIOCompletionRoutine (http://msdn.microsoft.com/en-us/library/aa364052(v=vs.85).aspx), in the remarks, it asks to call GetOverlappedResult even if dwErrorCode is zero. But, if we check the documentation for GetOverlappedResult (http://msdn.microsoft.com/en-us/library/ms683209(v=vs.85).aspx), it mentions about a manual reset event to be specified for each OVERLAPPED pointer passed. Also, GetOverlappedResult waits for the event to be signaled. But it makes sense to think that it is applicable only for the asynchronous versions of ReadFile and WriteFile. In the case of the overlapped operation started by ReadFileEx or WriteFileEx, the system may have another mechanism to execute GetOverlappedResult other than the event. In the case of ReadFileEx and WriteFileEx, lpCompletionRoutine is the callback function to be called by the system when the specified asynchronous operation completes. It will be invoked per ReadFileEx/WriteFileEx call. There is a side track for this. This callback routine is invoked only when the thread enters an “alertable” wait state. This state can be set with the help of SleepEx, WaitForSingleObjectEx, WaitForMultipleObjectsEx, MsgWaitForMultipleObjectsEx, etc. A thread can issue asynchronous I/O operations and freely do other jobs. Then, once all its other tasks are completed, it can enter the alertable wait state. The callback routines are then called in the context of such APIs. The status of an overlapped operation can be checked with the help of the GetOverlappedResult API.
Using the Code
I have wrapped all this knowledge into a small class AsyncFile. The functions are well commented. So it is easy to understand the purpose of each function.
class IAsyncOperationCompletionNotifier {
public:
virtual void OnAsyncOperationComplete(BOOL bRead,DWORD dwErrorCode)=0;
};
class AsyncFile{
public:
AsyncFile(LPCTSTR lpctszFileName,BOOL bCreate,DWORD dwDesiredAccess,
DWORD dwShareMode, IAsyncOperationCompletionNotifier* pINotifier,
BOOL bSequentialMode=FALSE, __int64 nStartOffset=0,
BOOL bInUIThread=FALSE);
BOOL IsOpen();
BOOL Write(LPVOID pvBuffer,DWORD dwBufLen,DWORD dwOffsetLow=0,DWORD dwOffsetHigh=0);
BOOL Read(LPVOID pvBuffer,DWORD dwBufLen,DWORD dwOffsetLow=0,DWORD dwOffsetHigh=0);
BOOL IsAsyncIOComplete(BOOL bFlushBuffers=TRUE);
BOOL AbortIO();
DWORD GetFileLength(DWORD* pdwOffsetHigh);
__int64 GetLargeFileLength();
VOID Reset(BOOL bSequentialMode=FALSE,__int64 nStartOffset=0);
operator HANDLE()
{
return m_hAsyncFile;
}
BOOL SeekFile(__int64 nBytesToSeek,__int64& nNewOffset,DWORD dwSeekOption);
~AsyncFile(void);
private:
....
};
The class object can be created specifying the file name, whether we need to create a new file, the access mode (READ/WRITE), the share mode (same as CreateFile), the callback interface to register for I/O completion notifications, sequential mode or not, the start offset, and finally whether the thread is hosting a UI or not. For a UI thread, the MsgWaitForMultipleObjectEx API is used for the I/O completion wait (to set the alertable wait state). In the case of a non-UI thread, WaitForSingleObjectEx is used. An event m_hIOCompleteEvent is used to synchronize the wait. It is a manual reset event and it should be as per the MSDN documentation. This is because, if the event is auto-reset, the wait functions can alter its state. This can cause un-intended deadlocks if calling GetOverlappedResult with its wait for completion flag set to TRUE. The m_hIOCompleteEvent is set when the completion routines are invoked for each asynchronous request made. Actually a counter is kept for each asynchronous call issued. This counter is decremented when the completion routine is invoked. When it reaches zero, the event is signaled.
The Read/Write functions perform the overlapped I/O operations. If the sequential mode flag is TRUE, the offset will be incremented by the buffer length requested to read/write. After the Read/Write requests are issued, the IsAsyncIOComplete function can be called to trigger the alertable wait state and get the queued I/O completion routines to be executed. This function checks the status of the overlapped operations by calling GetOverlappedResult for each asynchronous operation requested. Also, if the bFlushBuffers flag is TRUE, it will flush the file buffers to disk with the help of the FlushFileBuffers API. This may be necessary, because the system writes the data to an intermediate cache for performance. These cached contents are flushed to disk at a later time. Such a mechanism improves the application performance, but at the risk of data loss (if an unexpected shutdown occurs due to power failure or so).
The asynchronous I/O operations can be aborted by issuing the CancelIo API. This will succeed only if the API is called by the same thread which performs the asynchronous I/O. The AbortIO call can be used to cancel the IO.
Also, I have prepared a demo application (VS2010 solution and VC6.0 dsp provided). This application accepts a source file and copies the contents to a target file. It uses an instance of AsyncFile to read the source file and another instance to write to the target file. The progress of copy is visible in the UI. Also, the user can abort the operation in between. I have tested the same in Win7 and to some extent in XP. The debug trace can be seen in DebugView. That's all for now! Thank you!
Points of Interest
As I mentioned in the beginning, asynchronous file I/O is not a well discussed topic. So the design of the AsyncFile class was fixed after some experiment with these APIs. I have tried to use it with a more than 700 MB file (and it worked!). In the demo application, I have used an instance of AsyncFile to read and another instance to write. I have not tried using the same instance for reading and writing. Looking forward to see how this class will be used.
Update
- Fixed the pointer initialization issue
- Updated solution to VS2019 (community)
