Redirecting an Arbitrary Console's Input/Output

• Download demo project - 43.2 KB
• Download source files - 3.6 KB

Introduction

To redirect the input/output of a console application is interesting and useful. You can display the child's output in a window (just like Visual Studio's output window), or search some keywords in the output string to determine if the child process has completed its work successfully. An old, 'ugly' DOS program could become a useful component of your fancy Win32 GUI program.

My idea is to develop a simple, easy to use redirector class which can redirect an arbitrary console, and won't be affected by the behavior of the child process.

Background

The technique of redirecting the input/output of a console process is very simple: The CreateProcess() API through the STARTUPINFO structure enables us to redirect the standard handles of a child console based process. So we can set these handles to either a pipe handle, file handle, or any handle that we can read and write. The detail of this technique has been described clearly in MSDN: HOWTO: Spawn Console Processes with Redirected Standard Handles.

However, MSDN's sample code has two big problems. First, it assumes the child process will send output at first, then wait for input, then flush the output buffer and exit. If the child process doesn't behave like that, the parent process will be hung up. The reason of this is the ReadFile() function remains blocked until the child process sends some output, or exits.

Second, it has a problem to redirect a 16-bit console (including console based MS-DOS applications.) On Windows 9x, ReadFile remains blocked even after the child process has terminated; On Windows NT/XP, ReadFile always returns FALSE with error code set to ERROR_BROKEN_PIPE if the child process is a DOS application.

Solving the Block Problem of ReadFile

To prevent the parent process from being blocked by ReadFile, we can simply pass a file handle as stdout to the child process, then monitor this file. A simpler way is to call PeekNamedPipe() function before calling ReadFile(). The PeekNamedPipe function checks information about data in the pipe, then returns immediately. If there's no data available in the pipe, don't call ReadFile.

By calling PeekNamedPipe before ReadFile, we also solve the block problem of redirecting a 16-bit console on Windows 9x.

The class CRedirector creates pipes and launches the child process at first, then creates a listener thread to monitor the output of the child process. This is the main loop of the listener thread:

for (;;)
{
    // redirect stdout till there's no more data.
    nRet = pRedir->RedirectStdout();
    if (nRet <= 0)
        break;

    // check if the child process has terminated.
    DWORD dwRc = ::WaitForMultipleObjects(
        2, aHandles, FALSE, pRedir->m_dwWaitTime);
    if (WAIT_OBJECT_0 == dwRc)      // the child process ended
    {
        ...
        break;
    }
    if (WAIT_OBJECT_0+1 == dwRc)    // m_hEvtStop was signalled, exit
    {
        ...
        break;
    }
}

This is the main loop of the RedirectStdout() function:

for (;;)
{
    DWORD dwAvail = 0;
    if (!::PeekNamedPipe(m_hStdoutRead, NULL, 0, NULL,
        &dwAvail, NULL))    // error, the child process might ended
        break;

    if (!dwAvail)           // no data available, return
        return 1;

    char szOutput[256];
    DWORD dwRead = 0;
    if (!::ReadFile(m_hStdoutRead, szOutput, min(255, dwAvail),
        &dwRead, NULL) || !dwRead)
             // error, the child process might ended
        break;

    szOutput[dwRead] = 0;
    WriteStdOut(szOutput);          // display the output
}

WriteStdOut is a virtual member function. It does nothing in CRedirector class. However, it can be overridden to achieve our specific target, like I did in the demo project:

int nSize = m_pWnd->GetWindowTextLength();
         // m_pWnd points to a multiline Edit control
m_pWnd->SetSel(nSize, nSize);
m_pWnd->ReplaceSel(pszOutput);
       // add the message to the end of Edit control

To Redirect DOS Console Based Applications on NT/2000/XP

MSDN's solution is to launch an intermediate Win32 Console application as a stub process between the Win32 parent and the 16-bit console based child. In fact, the DOS prompt program (on NT/XP, it's cmd.exe, on 9x it's command.com) is a natural stub process we just need. We can test this in RedirDemo.exe:

1. Input 'cmd.exe' in Command Editbox, then press Run button.
2. Input the name of the 16-bit console based application (dosapp.exe for example) in the Input Editbox, then press Input button. Now we can see the output of the 16-bit console.
3. Input 'exit' in the Input Editbox, then press Input button to terminate cmd.exe.

Apparently, this is not a good solution because it's too complicated. A more effective way is to use a batch file as the stub. Edit stub.bat file like this:

%1 %2 %3 %4 %5 %6 %7 %8 %9

Then, run a command like 'stub.bat dosapp.exe', then the 16-bit DOS console application runs OK.

License

This article has no explicit license attached to it, but may contain usage terms in the article text or the download files themselves. If in doubt, please contact the author via the discussion board below. A list of licenses authors might use can be found here.