Background
While developing a database-dependent free web product, we had to write database-specific jobs in C#, which would otherwise be written in T-SQL, PL/SQL or DB2 SQL. Since our company did not want to invest in yet another big machine, we needed to run these jobs simultaneously from many different workstations. The database is hosted on a SMP machine and is capable of handling thousands of concurrent connections by using the load balancing failover feature of the Oracle 9i server. We wrote a racing multi-threaded C# batch program to maximize the CPU's ability to process the transactions. This program was run on several workstations which accessed a database server. The number of workstations was adjusted to keep the database server CPU utilization close to, but not greater than, 80%. We wrote an NT service to run these batch programs from different workstations. On a single workstation, several processes were started from the NT service which used different connection strings to the Oracle load balancing server. We had to explicitly specify the instance name in the processes as the Oracle load balancing server, since the server is good at balancing loads for on-line transactions, but not for batch jobs. In other words, instead of a big machine talking to another big machine, we were making several small machines to pound the big guy.
Introduction
The System.Diagnostic.Process class provides the mechanism to run a program in a separate process. We needed to spawn multiple processes of the same program or NT job script. We need to know when a job finishes and its performance, so we wrote a method to capture the standard output to a job specific file, which can be reviewed by operators. It was important to load balance the workstations through batch configuration files to optimize the time taken by each process and the number of transactions per process.
If we just want to execute a single process, this is a simple thing to do by attaching a hookup method to the Process.Exited event handler and capture the standard output for the process. The Process.Exited event handler raises the OnExit event when a process completes. The sample code here illustrates this:
Process p = new Process();
p.StartInfo.FileName = @"C:\WINDOWS\system32\ping.exe";
p.StartInfo.RedirectStandardOutput = true;
p.StartInfo.CreateNoWindow = true;
p.StartInfo.Arguments = @"www.google.com";
p.StartInfo.WorkingDirectory = @"C:\WINDOWS\system32";
p.EnableRaisingEvents = true;
p.StartInfo.UseShellExecute = false;
p.Exited += new EventHandler(captureStdout);
p.Start();
private void captureStdout(object sender, EventArgs e)
{
Process ps = (Process) sender;
Console.WriteLine(ps.StandardOutput.ReadToEnd());
}
The object sender, when type cast to the process type, will tell you which process has exited. This is not even necessary if you run a single process, but for most cases, if you are handling multiple processes through an ArrayList in a loop, you would like to know the process ID or name that has completed its work.
What we really need is a little more than this, as we want to identify not only the exiting process, but also a few external parameters which are not related to the process attributes, for example, the connection string of the Oracle load balancing server and the batch name that we want to identify for each process that we spawn. When a process exits, we should be able to capture the time and other external attributes and log them to the database so that our Process Manager can synchronize the works in such a way that there is no conflict when the same service is run by different workstations.
The MSDN documentation suggested writing our own event handlers to pass data to events. We attempted to override the OnExited API method in our derived class from Process but we discovered that this API method is not marked virtual, abstract or override in its base class. I confirmed this by looking at the source code of this base class. I do not know the reason why there is a discrepancy in MSDN documentation, or am I missing something very trivial here? I am not an expert in C#, so if you know about it, please do let me know. We wrote a separate event handler and attached it to the Exited event handler so that our custom event handler comes into play when the parent process exits.
This custom event handler allows you to access the index of the process in the hookup method of the event handler. Without this, you will have to write a workaround in the code to cache the index of the process, and then flush it out when the process exits. We originally tried the second method, but this was cumbersome and ugly.
The steps that we followed are given below:
-
Create a Custom EventArgs Class to pass event data.
public class ProcessEventArgs : EventArgs
{
private readonly int index;
public ProcessEventArgs (int p_index)
{
this.index = p_index;
}
public int Index
{
get {return index;}
}
}
-
Create a Delegate.
public delegate void
ProcessEventHandler(object sender, ProcessEventArgs e);
-
Create a derived class to hijack when parent class raises OnExited event.
public class ProcessManager : System.Diagnostics.Process
{
private int i;
ProcessEventHandler onProcessExited;
public ProcessManager()
{}
public ProcessManager(int i)
{
this.i = i;
this.Exited += new
EventHandler(processManagerCapture);
}
public event ProcessEventHandler ProcessExited
{
add
{
onProcessExited += value;
}
remove
{
onProcessExited -= value;
}
}
public void OnProcessExited()
{
if (onProcessExited != null)
onProcessExited(this,new ProcessEventArgs(i));
}
private void processManagerCapture(object o, EventArgs e)
{
OnProcessExited();
}
}
In the above derived class, we need a mechanism to attach our event when the parent event OnExit is raised. When we instantiate the class, we subscribe to the base class Exited handler and attach it to the processManagerCapture method. This method raises our custom OnProcessExited event. Through the above implementation, you hide the functionality of the custom event handlers and let developers use the derived class.
-
Implementation of the derived class.
We use the ProcessExited event handler to pass the index of our ArrayList to the exit method. The implementation class shows this. We inherit the new class from ProcessManager, which has the implementation of our custom ProcessExited event handler. The BatchManager class runs the ping program with two different arguments in a separate process and captures the standard output along with the external attributes that we need.
public class BatchManager : ProcessManager
{
ArrayList processArrayList;
private string[] batchName = {"Batch 1","Batch 2"};
private string[] pingSites = {"www.google.com","www.yahoo.com"};
public BatchManager()
{
}
public void Run()
{
try
{
RunPingProcess();
}
catch (Exception ex)
{
Console.WriteLine(" Error while trying to " +
"Run ping process. Msg = " + ex.Message);
}
}
private void RunPingProcess ()
{
processArrayList = new ArrayList();
for (int i = 0; i < pingSites.Length; ++i)
{
processArrayList.Add(new ProcessManager(i));
((ProcessManager)processArrayList[i]).StartInfo.FileName =
@"C:\WINDOWS\system32\ping.exe";
((ProcessManager)processArrayList[i]).StartInfo.RedirectStandardOutput
= true;
((ProcessManager)processArrayList[i]).StartInfo.CreateNoWindow = true;
((ProcessManager)processArrayList[i]).StartInfo.Arguments = pingSites[i];
((ProcessManager)processArrayList[i]).StartInfo.WorkingDirectory =
@"C:\WINDOWS\system32";
((ProcessManager)processArrayList[i]).EnableRaisingEvents = true;
((ProcessManager)processArrayList[i]).StartInfo.UseShellExecute = false;
((ProcessManager)processArrayList[i]).ProcessExited +=
new ProcessEventHandler(NewCapture);
((ProcessManager)processArrayList[i]).Start();
}
for (int i = 0; i < pingSites.Length; ++i)
{
if (!((ProcessManager)processArrayList[i]).HasExited)
{
((ProcessManager)processArrayList[i]).Refresh();
((ProcessManager)processArrayList[i]).WaitForExit();
}
}
}
private void NewCapture(object sender, ProcessEventArgs e)
{
ProcessManager pm = (ProcessManager) sender;
Console.WriteLine("Batch Job " + batchName[e.Index] + ": Ping on " +
pingSites[e.Index] + " has completed.\nThe output of ping is:\n");
Console.WriteLine(pm.StandardOutput.ReadToEnd());
Console.WriteLine();
}
}
Our sample program will work without a problem, since our standard output is not large. But if you have a batch process with a large standard output, the above hookup method will not work and will need to be modified. The reason for this is that, in the main loop we are waiting for the process to exit, but the hookup method reads all the standard output. Since the internal implementation is through a pipe, when the buffer gets filled, there will be a deadlock. The MSDN documentation suggests that you read the standard output before WaitForExit.
-
Run as a Standalone Program.
public class Test
{
public Test()
{}
static void Main()
{
BatchManager bm = new BatchManager();
bm.Run();
}
}
