Introduction
I wanted a trick console output pane just like that of Developer Studio for my program � for launching commands and receiving their output in an orderly fashion, instead of having DOS prompts popping up here and there. I found a couple of relevant classes in CodeProject [1, 2] which generally did what I wanted but shared a common limitation: they only work for NTxx not 9x.
As you do, I did some research and ended up writing a new class from scratch. It is mainly written in straight Win32 API except for some mild dependencies on WTL (CString class). I have tested it on both 9x & NT (including UNICODE) and it works fine. I hope you'll find it to be of "industrial-strength" and hence reliable to include in your programs.
Console redirection for all Windows platforms
Windows processes can share a single console � that's how DOS command prompts work. CreateProcess API has options that allow you to specify whether a process you launch will inherit the parent's console or will get its own. Consoles have traditionally been associated with command processors but this is only one particular type of use. In general, they are windows where standard I/O handles of plain (non-GUI) processes are bound to, enabling simple input and output with library functions like printf, cout etc.
If you have a GUI process more often than not you don't need a console, so any console child processes launched need to get their own. You could contemplate creating and offering a console in these cases, but this is cumbersome at best. Console windows are in a class of their own, you can't easily copy text from them, you can't even use the standard subclassing/hooking mechanics on them. You're better off without them.
The usual solution is to launch the child process giving it a hidden console and redirect its standard handles to anonymous pipes. If you are like me and never used consoles or pipes directly before, then there's a lot of new buzzwords, but the procedure is described pretty well in Q190351 - HOWTO: Spawn Console Processes with Redirected Standard Handles. In summary, here are the main steps:
- Create 2 pipes for the standard input and output of the child, ensuring that the child pipe end handles are inheritable.
- Spawn the child using
CreateProcess passing to it its end of the pipes using the STARTF_USESTDHANDLES option.
- Create a listener thread which waits on the "upstream" side of the
stdout pipe. Whenever the child writes in its stdout, our thread is awakened from its ReadFile blocking call and retrieves the child output.
- When the child terminates, the pipe is broken and the thread can terminate.
In a similar fashion, we can send input to the child, if necessary, using the stdin pipe. We use WriteFile on our end and the child is accepting it just as if it was reading from its (invisible) console. Note that we don't need a thread for this task. That's all there is to it for establishing 2 way communication between the parent and child processes. Or is it?
Complications for Windows 9x
Developers' life would be dull without all these platform-specific quirks, no? :) The above procedure works fine in Win9x but only if the spawned process uses a 32-bit console. It so happens that the type of child you are more likely to spawn, the DOS command processor, uses a 16-bit console which breaks the calls to ReadFile our listener thread is waiting on to receive output from the child. Either we receive no output or we are not notified when the child terminates (or both). Splendid!
To add insult to injury, Microsoft offers a really daft workaround in their knowledge base article Q150956 - INFO: Redirection Issues on Windows 95 MS-DOS Applications. This unashamedly recommends spawning an intermediate stub Win32 console process and launching the intended child from it (!) overlooking the "slight" efficiency problem this entails. You could leave this to your PR office trying to come up with soundbites like "launch one, get one free", in a desperate cover-up attempt. But chances are this is hardly going to fool/impress any Win9x users � and believe me there are still loads of them around.
The workaround in Q150956 got me thinking that the whole issue was to use a parent process with a 32-bit console to spawn 16-bit console children. I don't know why the heads at Microsoft didn't think of it first, but I gathered that if the original process had a console then the need for the stub console process disappears. So all you need is to create a console for your GUI process and then launch the DOS processor making sure it shares the parent console. From then on the piping redirection works a treat.
Obviously you don't want the users to see an ugly console popping up next to your cool skinned dialog window, so that has to remain hidden. Unfortunately AllocConsole API doesn't take a parameter for creating a hidden console the way CreateProcess does. I have discovered that WH_CBT hooks won't work either for console windows. So I had to resort to a lo-tech alternative of searching for the new window with FindWindow after its creation, so as to hide it with ShowWindow. But all in all, it is a small price to pay; if any readers have a better workaround please let me know.
Using CConsolePipe class
The CConsolePipe class is meant for GUI processes that want to spawn child processes using the DOS command processor (cmd.exe or command.com depending on the Windows platform), and read their output. It encapsulates all the pipe work, threading and platform differences, exposing a simple interface as follows:
CConsolePipe(BOOL bWinNT, CEdit wOutput, DWORD flags). This class should be created on the heap with new, since it self-destructs after the child terminates. The constructor expects a boolean for the platform type (NT or 9x), some flags and an edit control window handle where the redirected output is sent.
int Execute(LPCTSTR pszCommand). The argument is the command to execute without the command processor. It can contain environmental variables like %WINDIR% which are automatically substituted for you. It returns some CPEXEC_xxx constant depending on the success or otherwise of the spawning. In case of failure, you are responsible for delete-ing the object, unless you want to have another go.
virtual void OnReceivedOutput(LPCTSTR pszText). If Execute succeeds returning CPEXEC_OK then this virtual member will be called each time child output is captured. The default action is to print the received pszText buffer in the edit window specified in the constructor. You may override this function in a derived class to do something different. When the child terminates OnReceivedOutput is called one last time with NULL as its argument, hinting of this special event. Shortly after this the object will commit suicide via delete this.
static void Term(). Just before your application ends (e.g. within WinMain) you should call CConsolePipe::Term() to release any consoles allocated for Win9x. This has only got to be called once not each time you use the CConsolePipe object.
Those are the main methods you're most likely to use. For each command you want to launch, you must create a new instance on the heap and use the Execute method. This was convenient for my particular project requirements where I just wanted to "launch and forget".
There are a few more methods that you may want to use, please see the class definition within consolePipe.h. The code is well commented. You'll also discover that I rather like to use ATLASSERT-ions frequently. This may seem over the top but that's the kind of guy I am :) and in 99% of the cases I came to thank the powers that be for their existence, especially when lazily modifying code 6 months down the road.
Building your project
You just need to include the sole header file consolePipe.h in your stdafx.h. The intended platform is the one I'm working on, a mixture of WTL7 and ATL3. You don't need any special initializations (e.g. CoInitialize or common controls) to use the class. Both UNICODE and MBCS builds are supported. I am still using VS6 but I hope it will build ok with VS.NET too.
The mild dependency on ATL/WTL can be removed by redefining the ATLTRACE macros to plain MFC-style TRACE equivalents or even OutputDebugString API calls. From WTL it just uses the CString class but I suppose � although untested � that you could as easily compile it with MFC's version of CString. Finally in MFC, you may have to change the CEdit wrapper references to pointers. I haven't used MFC for a while so I can't remember if you can pass whole window objects by value.
WTL test application
I have created a simple dialog-based application which demonstrates the use of CConsolePipe class. To build it, you'll need Windows Template Library (WTL) installed, the templated classes for improved ATL windowing. Just take a look at the executable size and you'll realize what's the fuss all about. If you don't have WTL you may download it and add its path to your Include directories. You'll be glad you did so.
The main action occurs within CMainDlg::OnRun handler, which reads the input field, creates the pipe object and executes the command, taking evasive action if things go pear-shaped:
TCHAR buf[128];
GetDlgItemText(IDC_COMMAND, buf, sizeof(buf)/sizeof(buf[0]));
CEdit out(GetDlgItem(IDC_OUTPUT));
m_pLastCommand = new CMyConsolePipe(m_bIsWindowsNT, out, 0);
int status = m_pLastCommand->Execute(buf);
if(CPEXEC_OK == status) {
...
}
else {
delete m_pLastCommand;
m_pLastCommand = 0;
...
}
You may have noticed that I am using a derived class CMyConsolePipe. This is for overriding the virtual OnReceivedOutput member to send a message to the dialog when the child process terminates, so as to update the activation of the Run/Break buttons of the GUI. Note that OnReceivedOutput is called within the context of the background thread; this may limit the things you can do within this handler.
You can test the Break button using a command that is guaranteed not to terminate by normal means, like "more <CON:". Clicking on the button forcibly terminates the child using the Break method. This should be only used when things get desperate since it acts like a right bully. For Win9x it may even cause Windows to crash, hence should be avoided at all costs.
Update (December 2002)
As promised, here are a few improvements to the original class. Extra tweakability comes in the form of various CPF_xxx flags that can be specified in the CConsolePipe constructor. The following constants can be ORed together:
CPF_REUSECMDPROC. Forces the creation of a persistent command processor (via the /K option) which doesn't stop after the first command terminates. This is somehow optimized behavior since you don't have to create a new object every time you want to run something. A single object with a single command processor and listener thread takes care of all input. Subsequent commands can be issued either with Execute or SendChildInput methods. This works by sending input at the child process' stdin handle just like as if someone was typing in a console window. When this flag is set, the child process won't terminate unless the StopCmd member is called � which just sends the "exit" command. A slight disadvantage with this flag is that one cannot tell whether the command processor is actually busy or just standing there. Commands are just queued for execution.
CPF_NOAUTODELETE. Nothing spectacular, it just switches off the default self-destruction (via delete this) of the object. The class may thus be reused but the benefits aren't that great. Perhaps it could be used in conjunction with CPF_REUSECMDPROC to create a stack-based object, to protect against leaks when the program terminates.
The demo project has been updated to use CPF_REUSECMDPROC. Moreover it now demonstrates a procedure to send input to the child process for those commands that require it (e.g. Y/N answers for "del /P *.*" etc.). Note that the response can be typed either in the output window or in the box where main commands are specified. It is all rather clunky but it works. The main class CConsolePipe is predominantly for commands that just generate output though.
Update #2 (November 2003)
This looks like the final round of refinements. Major additions:
SendCtrlBrk() method. This sends a Ctrl-Break event to the running console, suitable for stopping some programs like more etc. Unfortunately this method doesn't seem to have any effect for Win9x, but at any rate it is preferable to the bullying tactics of the old Break() method. Note that this functionality requires a hidden console window even for Windows NT.
- OEM-awareness. All standard command processors work with the OEM code page. Assuming that your program deals with "Windows" strings (ANSI or Unicode) some conversions are required both for outgoing commands and for received output. These ensure that filenames with "funny" characters (code >= 128) will be encoded properly. There's a flag for turning off this default OEM assumption (
CPF_NOCONVERTOEM) but it is unlikely that you'd ever want to use it, unless you communicate with a custom command processor.
The demo project has been updated with an extra button highlighting the new "soft break" capability. If a launched program appears to be stuck, try the new break button first before resorting to an abrupt killing.
