Introduction
I've come across a requirement on a number of occasions to produce a Windows application that interacts with a remote web server. The Windows application may be dealing
with a web-service, or simply automating form input or screen scraping - what is common is that there is one side of the equation that is web based and therefore can potentially
handle multiple requests, thus allowing us to complete the process faster. This article is a project that consists of two parts: a threaded Windows client, and a simple
MVC application that it interacts with. As not all remote services allow multiple connections, the project gives the option for running the process in sequential
or parallel (threaded) mode. The source code of both projects is attached.
Background
The main concept being introduced in this article is Windows multi-threading. The approach I have taken is one of many open to developers.
The technologies being demonstrated here are:
- using an HTTPClient in async mode in a thread
- XML messages
- interacting with a Windows application main operating thread to update objects on the user interface in a thread safe manner
The threading concept is as follows:
- create a thread and set its various attributes
- when the thread completes its work, have it call back into a work-completed method in the main form thread and update the user
as required.
Setting things up
The simple server - An MVC app
In order to test our work, and not trigger a denial of service warning (large numbers of multiple threads can do that!), we will create a test harness. In this case, a simple MVC
application will suffice. We will create a controller method GetXML
that takes in an ID sent by the Windows application, and returns an XML response.
The GetXML
method is called like this: http://localhost:4174/home/GetXML?ItemID=23.
And returns output XML like this:
<response type="response-out" timestamp="20130804132059">
<itemid>23</itemid>
<result>0</result>
</response>
NB: for the purposes of this test, a "result" of 0 = failure, 1 = success.
(1) Create a new MVC app, and add a new controller GetXML
. We are also going to put in a small "sleep" command to slow things down a bit and emulate
delay over the very busy Interwebs.
public ContentResult GetXML()
{
string ReceivedID = Request.Params["ItemID"];
Random rnd = new Random();
int SleepTime = rnd.Next(1, 2) * 1000;
System.Threading.Thread.Sleep(SleepTime);
return Content(TestModel.GetXMLResponse(ReceivedID), "text/xml");
}
(2) Create a model method that takes care of the logic of constructing the XML response to send back. This will take as parameter an ID (int
) that represents the identifier
of a list of objects/queries that the user is working with. These could be credit cards, websites to scrape, account numbers, etc. In reality you can send in any data you need
to work with, and return then to the main calling application.
public static string GetXMLResponse(string ItemID)
{
Random rnd = new Random();
string ResultCode = rnd.Next(0, 2).ToString();
string TimeStamp = GetTimeStamp();
XmlDocument doc = new XmlDocument();
XmlNode rootNode = doc.CreateElement("response");
XmlAttribute attr = doc.CreateAttribute("type");
attr.Value = "response-out";
rootNode.Attributes.Append(attr);
attr = doc.CreateAttribute("timestamp");
attr.Value = TimeStamp;
rootNode.Attributes.Append(attr);
doc.AppendChild(rootNode);
XmlNode dataNode = doc.CreateElement("itemid");
dataNode.InnerText = ItemID;
rootNode.AppendChild(dataNode);
dataNode = doc.CreateElement("result");
dataNode.InnerText = ResultCode;
rootNode.AppendChild(dataNode);
return doc.OuterXml;
}
The threaded client - A Windows form app
The client is visually quite simple. It contains two edit boxes for input variables, a listview to show the user what is happening, and a checkbox to tell the program if it should
run in sequential or threaded mode.
We will go through the overall logic first by examining the sequential process, and then look at the threading part.
At the top of the form class we keep track of some variables:
private int RunningThreadCount;
private int RunTimes;
private int TimeStart;
Everything is kicked off by the RunProcess
button click event:
TimeStart = System.Environment.TickCount;
InitProcess();
if (chkRunThreaded.Checked)
RunProcessThreaded();
else RunProcess();
We keep track of the start time, and update this when all processes are complete to test how long the process took. We also at this stage call an
Init
method that sets things
up for us, assigning some variables and filling the ListView with values.
public void InitProcess()
{
btnExit.Enabled = false;
btnRunProcess.Enabled = false;
chkRunThreaded.Enabled = false;
RunTimes = int.Parse(edtTimesToRun.Text);
FillListView();
RunningThreadCount = 0;
}
public void FillListView()
{
lvMain.Items.Clear();
for (int i = 0; i < RunTimes; i++)
{
ListViewItem itm = new ListViewItem();
itm.Text = (i+1).ToString();
itm.SubItems.Add("Pending");
itm.SubItems.Add("-");
itm.SubItems.Add("-");
lvMain.Items.Add(itm);
}
}
Let's now look at the sequential RunProcess
method. This
controls the main body of work for each web request. The number of times to run
the process is set by the value of edtTimesToRun.Text
which is
assigned to the variable RunTimes
.
The RunProcess
method has a keyword of async
- this is important as we are using the
await
keyword within the RunProcess
method. The
important part of this code is SendWebRequest
- this takes the input, queries the web server, and returns a value that we use to update the UI for the user.
public async void RunProcess()
{
for (int i = 0; i < RunTimes; i++) {
updateStatusLabel("Processing: " + (i + 1).ToString() +
"/" + RunTimes.ToString());
lvMain.Items[i].Selected = true;
lvMain.Items[i].EnsureVisible();
lvMain.Items[i].SubItems[1].Text = "Processing...";
SimpleObj result = await Shared.SendWebRequest(
new SimpleObj()
{ ItemID = i.ToString(),
WebURL = edtTestServer.Text}
);
lvMain.Items[i].SubItems[1].Text = result.ResultCode;
if (result.ResultCode == "ERR")
lvMain.Items[i].SubItems[2].Text = result.Message;
}
CleanUp();
}
SendWebRequest
is located in a separate shared.cs file as it
is used from two different places. The shared.cs file also contains a
simple object called SimpleObj
.
This is used to carry data between methods.
public class SimpleObj
{
public string WebURL;
public string ResultCode;
public string XMLData;
public string Message;
public string ItemID;
}
The SendWebRequest
method is again, flagged as async. This will be explained later.
In the SendWebRequest
method, we set up an HTTPClient, calling its
PostAsync
method. Here we are telling the HTTPClient to perform a "POST" action against
the server. If you have done web programming before, you will recall setting up a form:
<form action="somedomain.com/someaction?somevalue=134" method="post">
<input type="text" id="ItemID">
<input type="submit" value="send">
</form>
That is in effect we are doing here. We create the client object, send in as
parameters the URL of the website we want to send the data to, together with the
"content", which is the data packet to send. In this case the content simply
consists of the parameter ItemID
and its value.
HttpResponseMessage response = await httpClient.PostAsync(rec.WebURL, content);
The await
keyword tells the code to sit there until with the HTTPClient comes back with a response, or an exception is raised. We examine the response
to ensure it is valid (response.IsSuccessStatusCode
), and assuming it is, we proceed to take the response content stream and process its XML result. Note the outer Try/Except
wrapper - this will catch any HTTP connection errors and report these separately and allow the application to continue working smoothly.
public static async Task<SimpleObj> SendWebRequest(SimpleObj rec)
{
SimpleObj rslt = new SimpleObj();
rslt = rec;
var httpClient = new HttpClient();
StringContent content = new StringContent(rec.ItemID);
try
{
HttpResponseMessage response =
await httpClient.PostAsync(rec.WebURL, content);
if (response.IsSuccessStatusCode)
{
HttpContent stream = response.Content;
Task<string> data = stream.ReadAsStringAsync();
rslt.XMLData = data.Result.ToString();
XmlDocument doc = new XmlDocument();
doc.LoadXml(rslt.XMLData);
XmlNode resultNode = doc.SelectSingleNode("response");
string resultStatus = resultNode.InnerText;
if (resultStatus == "1")
rslt.ResultCode = "OK";
else if (resultStatus == "0")
rslt.ResultCode = "ERR";
rslt.Message = doc.InnerXml;
}
}
catch (Exception ex)
{
rslt.ResultCode = "ERR";
rslt.Message = "Connection error: " + ex.Message;
}
return rslt;
}
So, that is the basic sequential work-flow. Take a list of work items, iterate through them in sequence, call the web server, and parse back the xml response.
As the processes are run sequentially, the overall time taken to complete can be high.
Now let's run through the RunProcessThread
code and see the difference.
Here is our outer wrapper method in the main form:
public void RunProcessThreaded()
{
updateStatusLabel("Status: threaded mode - watch thread count and list status");
lblThreadCount.Visible = true;
for (int i = 0; i < RunTimes; i++)
{
updateStatusLabel("Processing: " + (i + 1).ToString() + "/" + RunTimes.ToString());
lvMain.Items[i].Selected = true;
lvMain.Items[i].SubItems[1].Text = "Processing...";
SimpleObj rec = new SimpleObj() { ItemID = i.ToString(), WebURL = edtTestServer.Text };
CreateWorkThread(rec);
RunningThreadCount++;
UpdateThreadCount();
}
}
The critical change is that instead of carrying out the WebRequest
task on each loop sequentially, we are passing that task off to the method CreateWorkThread
,
passing in the required parameters.
public void CreateWorkThread(SimpleObj rec){
ThreadWorker item = new ThreadWorker(rec);
item.Completed += WorkThread_Completed;
item.DoWork();
}
This small method creates a new object ThreadWorker
, and tells
it to call WorkThread_Completed
when it is finished. The WorkThread_Completed
method in the form simply updates the form UI
*in the context of the form thread* and performs some cleanup.
private void WorkThread_Completed(object sender, WorkItemCompletedEventArgs e)
{
lvMain.Items[int.Parse(e.Result.ItemID)].SubItems[1].Text = e.Result.ResultCode;
if (e.Result.ResultCode == "ERR")
lvMain.Items[int.Parse(e.Result.ItemID)].SubItems[2].Text = e.Result.Message;
RunningThreadCount--;
UpdateThreadCount();
if (RunningThreadCount == 0)
{
CleanUp();
}
}
I have created a separate class/file ThreadWorker
to manage the thread work - this keeps things separate and clean.
The class has some private members and a public event:
class ThreadWorker
{
private AsyncOperation op;
private SimpleObj ARec;
public event EventHandler<WorkItemCompletedEventArgs> Completed;
public ThreadWorker(SimpleObj Rec)
{
ARec = Rec;
}
}
You will recall that in our main form, when we are creating each
thread worker, we set up the thread object, then tell it to DoWork
so this is the main kick-off method for the thread object.
public void DoWork()
{
this.op = AsyncOperationManager.CreateOperation(null);
ThreadPool.QueueUserWorkItem((o) => this.PerformWork(ARec));
}
The reason I am using a ThreadPool
object is that creating
threads is a very expensive operation therefore using the pool means that on
completion, threads can be put into a pool to be reused. After being added to
the pool, we tell the thread to kick off the method PreformWork
.
This method calls our shared method SendWebRequest
,
and when that is finished, calls PostCompleted
which gets picked up
by the WorkThread_Completed
method in the main form class.
private void PostCompleted()
{
op.PostOperationCompleted((o) =>
this.OnCompleted(new WorkItemCompletedEventArgs(ARec)), ARec);
}
protected virtual void OnCompleted(WorkItemCompletedEventArgs Args)
{
EventHandler<WorkItemCompletedEventArgs> temp = Completed;
if (temp != null)
{
temp.Invoke(this, Args);
}
}
Our main form "WorkThread_Completed
" method watches each incoming terminated thread, and when they have all completed, runs some cleanup code.
private void WorkThread_Completed(object sender, WorkItemCompletedEventArgs e)
{
lvMain.Items[int.Parse(e.Result.ItemID)].SubItems[1].Text = e.Result.ResultCode;
if (e.Result.ResultCode == "ERR")
lvMain.Items[int.Parse(e.Result.ItemID)].SubItems[2].Text = e.Result.Message;
RunningThreadCount--;
UpdateThreadCount();
if (RunningThreadCount == 0)
{
CleanUp();
}
}
And that is it, as you can see running threaded adds a bit more code, but dramatically improves performance.
As I stated at the start of this article, this is but one way of handling a threaded application. Thread pools have their advantages and disadvantages,
you need to weigh up your goals and granular needs against the ease of use. If you are interested in this area you should also look at Background worker and if you want to harness the power that is in multi-core
CPUs while threading the Task Parallel Library is a great way to go.
(PS: If you found this article useful or downloaded the code please let me know by giving a rating below!)