Introduction
While doing a code review for a friend of mine, I came across an interesting technique for getting a web service to “push” an event or message back to the client when something occurs. We all know that the HTTP is a request/response protocol. The client makes a request, a socket is opened, the server allocates a thread for the request, the request is processed, the results are returned, and the socket is closed. In the code that I saw, and reworked/improved for this example, web services are used to communicate events or messages between the request and the response.
Think of a chat room application entirely handled by web services. The various “events” or messages that would have to be passed between users include “user has logged in”, “user has logged out”, and the actual text messages sent.
Background
Normally, this would be handled by writing some TCP socket based application that maintains sockets between the server and the clients. The advantage to using a web service is that you don’t have to manage the sockets or the thread pool yourself. You also don’t have to worry about configuring every firewall on every client machine to open a port for you; HTTP runs on port 80, and is not blocked. This is known as HTTP Tunneling. Your clients can communicate with each other through HTTP, without firewalls and anti-viral applications treating them like some Trojan horse.
The major disadvantage of this technique, and we’ll see this further in the article, is that it requires holding on to a thread pool thread on the server. Although the tuning and performance aspects of this problem are beyond the scope of this article, suffice it to say that this technique may not scale very well. Still, if the application is served by a dedicated server with a predetermined amount of clients, and the thread pool configuration on the server is properly done, this solution can be compelling.
Besides practical usage, this article showcases some of the challenges and techniques in writing multi-threaded applications. Some of the .NET threading classes that I have used in the example can be replaced with others. I believe the ones chosen best fit the example for both functional and performance reasons, but feel free to experiment with various constructs. At the end of this article, I’ve included links to various sites that deal with multi-threading.
Using the Code
OK, so how do I get a web service to notify the client when something occurs on another client or on the server itself? As I mentioned earlier, web services use a request/response protocol. How do we get it to push back a message? The answer is, we don’t. We simulate a push back by stopping the request on the server before it returns to the client. This process simulates “listening”. When an event occurs, such as another client sending a message through another web method, our listening web method is released, returning an “event” object that represents the type of event/message that occurred and containing any data that pertains to it.
Initial state – Everyone’s listening for an event from someone
An event occurs – A listen request is released and a response occurs
This is possible since a web service can be called asynchronously from a form without locking up the form’s main thread. A callback event handler is used to process the incoming event from the server.
Calling a web service asynchronously is made easy in .NET 2.0 since every web method you create generates two methods and an event on the client proxy.
For instance, if you were to create a web method called HelloWorld
, you’ll see that the client proxy for the web service has a HelloWorld
method, a HelloWorldAsynch
method, and a HelloWorldCompleted
event. The first method, as you probably know, relays the execution request to the server and blocks the calling client thread until a response occurs. It is said to be synchronous. HelloWorldAsynch
acts as a fire-and-forget method. You call it from your client code, but it never blocks the thread. Your form is free to continue processing without having to wait for the web method to return a value or a response. When the response arrives, it fires off the HelloWorldCompleted
event which can be handled by your client code.
The Listen
method described above works in this manner. When the form loads, it calls for a Listen
using the asynchronous version of the method. When an event occurs on the server that needs to be reported to the client, the ListenCompleted
event fires off. This event is handled with a method that processes the “event”. It then calls Listen
again to be ready for the next event.
The ListenCompleted
event comes complete with a ListenCompletedEventArgs
object, also generated by the IDE. It contains an object of the same type as returned by the Listen
web method.
Since the Listen
web method returns an “EventObject
” array defined by me, the event argument’s Result
property will contain an object of that type. EventObject
is the ancestor of three event classes that my client application is interested in listening to and processing. They are as follows: LoginEvent
, LoggedOutEvent
, and MessageEvent
.
Since my sample application is a simple chat room, I want to display all users that are currently logged in, and being told in real time when a new user has logged in, when a logged in user has logged out, and when a message has been sent by someone in the chat room.
I made the Listen
web method return an array of events because various events can occur while we were not listening. I.e., while we were processing an event on the client, and before calling the ListenAsync
web method, various other events can occur on the server that we may have missed (I’ll get into the caching mechanism of these events further down the line). When we listen again, we want to receive all the missed events in one shot. Hence, an array of EventObject
s.
The code for the ListenCompleted
event handler looks like this:
private void _ws_ListenCompleted(object sender, ListenCompletedEventArgs e) {
bool getAllUsers = false;
ChatClient.TwoWayWS.EventObject[] events = e.Result;
System.Diagnostics.Trace.WriteLine("_ws_ListenCompleted - " +
"Made it in - _formClosing=" + _formClosing);
System.Diagnostics.Trace.WriteLine("_ws_ListenCompleted - " + events.Length);
if (_formClosing)
return;
foreach (EventObject eventObj in events) {
System.Diagnostics.Trace.WriteLine("_ws_ListenCompleted - " +
"event Object type " + eventObj.ToString());
_myLastEventID = eventObj.EventID;
_myLastEventsListResetId = eventObj.EventsListResetID;
switch (eventObj.ToString()) {
case "ChatClient.TwoWayWS.LoginEvent":
if (lvLoggedInUsers.Items.Count == 0)
getAllUsers = true;
else {
LoginEvent le = eventObj as LoginEvent;
try {
if (!lvLoggedInUsers.Items.ContainsKey(le.UserName))
lvLoggedInUsers.Items.Add(le.UserName, le.UserName, 0);
}
catch { }
}
break;
case "ChatClient.TwoWayWS.LoggedOutEvent":
LoggedOutEvent lo = eventObj as LoggedOutEvent;
lvLoggedInUsers.Items.RemoveByKey(lo.UserName);
break;
case "ChatClient.TwoWayWS.MessageEvent":
MessageEvent msg = eventObj as MessageEvent;
string fromPart = "From " + msg.From + ":";
rtChatRoom.SelectionBackColor = Color.LightGray;
rtChatRoom.SelectionColor = Color.Crimson;
rtChatRoom.SelectedText = fromPart;
rtChatRoom.SelectionBackColor = Color.White;
rtChatRoom.SelectionColor = Color.Black;
rtChatRoom.SelectedText = " " + msg.Message;
rtChatRoom.SelectedText = "\n";
break;
default:
int nonEventCnt = int.Parse(lblNonEventCnt.Text);
nonEventCnt++;
lblNonEventCnt.Text = nonEventCnt.ToString();
break;
}
if (getAllUsers) {
LoginEvent[] allLoginEvents = _ws.GetAllUsers();
foreach (LoginEvent le in allLoginEvents) {
try {
lvLoggedInUsers.Items.Add(le.UserName, le.UserName, 0);
}
catch { }
}
getAllUsers = false;
}
}
Application.DoEvents();
int milliSecdelay = string.IsNullOrEmpty(txtListenDelay.Text) ? 0 :
int.Parse(txtListenDelay.Text);
System.Threading.Thread.Sleep(milliSecdelay * 1000);
txtListenDelay.Text = "0";
System.Diagnostics.Trace.WriteLine("_ws_ListenCompleted -" +
" Calling _ws.ListenAsync() for " + txtUsername.Text);
_ws.ListenAsync(_myLastEventID, _myLastEventsListResetId);
}
Server Pushback and Thread Management
Let’s examine how the web service acts as a server for the chat application by processing the above mentioned events and pushing them back to all the listening clients.
Core Objective’s Mechanism
The primary mechanism in the Listen
method is to use a WaitHandle
of type AutoResetEvent
to cause the Listen
event to block, and to register that WaitHandle
to a global list.
.
.
.
waithandle = new AutoResetEvent(false);
Global.WaitingListenerList.Add(waithandle);
Global.EventBeingDelivered = false;
}
System.Diagnostics.Trace.WriteLine("Listen - Aquired waithandle and blocked IIS thread");
listenTimedOut = !waithandle.WaitOne(Global.EventWaitTimeout, true);
The list will be traversed upon the arrival of an event, and each WaitHandle
will be released with a Set
method on the WaitHandle
. So, if we peek at a snippet of code from the SendObjectEvent
method that occurs with each event, we’ll see this:
foreach (AutoResetEvent waitingListner in Global.WaitingListenerList) {
waitingListner.Set();
}
SendObjectEvent
then goes into its own wait state that’s released only when all the Listen threads have been released so that it can do some post-event-release housekeeping. This is achieved by the Listen
method by decrementing a listener count right before returning the event object to the client. When the count is zero, it signals the SendObjectEvent
method through a global waithandle called AllListenersReleased
.
That’s the concept in a nutshell. Of course, no multi-threaded solution is ever that simple. But, before I dig deeper into the issues and the solution that cropped up while writing this sample application, it's important to understand the core objective’s mechanism:
- Have the
Listen
method block the thread on its own so that it doesn’t return a response until some event occurs and data pertaining to that event is ready to be returned to the client. - Retain the blocking
WaitHandle
object for each client application running the Listen
web method on a web server thread. The list of WaitHandle
s are retained in a global list. SendObjectEvent
releases each listening thread when an event occurs that needs to be pushed back to all the listening clients, by issuing a Set
on each WaitHandle
that we’ve accumulated in the global list. It then blocks/waits until all the Listeners have been released.- The
Listen
method signals the waiting SendObjectEvent
that all listing threads have been released so that it can do some housekeeping such as un-registering the Listener WaitHandle
s (the client will re-register a WaitHandle
on subsequent calls to Listen
).
Digging Deeper
Ancillary Issues
The following issues and challenges must be addressed before the core objective is complete:
- Synchronization - The
Listen
web method and the SendEventObject
method, which all events call to release the Listeners and pass event specific data, must be synchronized and thread safe.
- Can’t start to
Listen
while sending an event. - Can’t send an event while the
Listen
method registers its waithandle. - Can’t process more than one event at a time.
- Don’t want to register two listener waithandles at the same time.
- Event Caching - The web service must cache events until it’s certain that all clients received them. This becomes an issue when events are pushed back while one or more clients aren’t listening due to being busy processing a prior event.
- If outstanding events exists for a client, they should be returned immediately upon the subsequent
Listen
call. - The web service must have a reasonable way to determine if all events have been delivered to all clients. Once that has been determined, the events cache can be cleared. The list of waithandlers can be cleared at this time as well.
- Request Timeouts - Request timeouts must be addressed. Before a timeout occurs, we must release a dummy event back to the client. The client in turn recognizes that it received a “non-event” event and simply re-executes a
Listen
.
Synchronization
There are various locks in the code to ensure that multiple threads do not modify global variables simultaneously. I’ll focus on the locks that synchronize access between the Listen
web method and the SendEventObject
method that all the “application events” use to push back an event.
Conceptually speaking, listening for, and sending, an event needs to be synchronized. I don’t want to send an event while a Listen
method is trying to register its waithandle to the global list. This could cause a race condition were the Listen
method hasn’t registered itself for receiving events, but an event has already been released to all the listening threads. Since all registered listening threads are assumed to have received the event, the event cache can be cleared and a client can lose an event.
Furthermore, I don’t want two events processed simultaneously either. Although I can push back more than one event to a client, I want to manage "one event/one release" to all listening clients at a time.
Below are the first few lines of the Listen
method. A generic lock object (Global.LockObj
) is used in a code block to protect various global variables that are being changed - that has to be done atomically. Within that code block, I enter a loop that checks if an event is being delivered, and blocks the current thread if it is.
[WebMethod]
[XmlInclude(typeof(EventObject))]
[XmlInclude(typeof(LoginEvent))]
[XmlInclude(typeof(LoggedOutEvent))]
[XmlInclude(typeof(MessageEvent))]
public EventObject[] Listen(int myLastEventID, int myLastEventsListResetId) {
AutoResetEvent waithandle;
bool listenTimedOut = false;
EventObject[] returnedEvents;
lock (Global.LockObj) {
while (Global.EventBeingDelivered)
Monitor.Wait(Global.LockObj);
Global.EventBeingDelivered = true;
returnedEvents = EventUtils.PrepareEventsForSending(myLastEventID,
myLastEventsListResetId);
if(returnedEvents.Length > 0) {
Global.EventBeingDelivered = false;
return returnedEvents;
}
waithandle = new AutoResetEvent(false);
Global.WaitingListenerList.Add(waithandle);
Global.EventBeingDelivered = false;
}
I use a global state variable called EventBeingDelivered
to determine if a client is in the process of registering a Listen, or if an event is in the process of being sent to all the clients. Since I’m using a “blocking condition flag”, as mentioned in Joseph Albahar’s excellent e-book on Threading in C#, I’ve chosen the Monitor
class as the synchronization mechanism between the Listen
and the SendEventObject
methods.
This is essential since a Monitor.Pulse
may have executed indicating that an application event has been processed successfully, but another thread running a SendEventObject
may have received execution control prior to the Listen code executing. We, therefore, check the blocking condition flag for its state prior to continuing with the registration. If the flag is still true, we loop into another wait.
The same code exists on the SendEventObject
method to ensure that another thread is not sending out an event prior to processing the current thread’s event.
public class EventUtils {
public static void SendEventObject(EventObject eo) {
lock (Global.LockObj) {
while (Global.EventBeingDelivered)
Monitor.Wait(Global.LockObj);
Global.EventBeingDelivered = true;
System.Diagnostics.Trace.WriteLine("Event processing commenced");
Global.NewEventID++;
Global.EventsList.Add(eo);
Global.ListenerCount = Global.WaitingListenerList.Count;
foreach (AutoResetEvent waitingListner in Global.WaitingListenerList) {
waitingListner.Set();
}
}
Global.AllListenersReleased.WaitOne();
lock (Global.LockObj) {
Global.LastEventID = Global.NewEventID;
if (Global.loggedInUsers.Count == Global.WaitingListenerList.Count) {
Global.EventsList.Clear();
Global.NewEventID = 0;
Global.EventsListResetId++;
}
Global.WaitingListenerList.Clear();
Global.EventBeingDelivered = false;
Monitor.PulseAll(Global.LockObj);
}
System.Diagnostics.Trace.WriteLine("SendEventObject - " +
"Released listeners and cleared lists");
}
If you’re wondering why the Global.LockOb
j isn’t enough to synch the two methods, that’s because both methods exit the lock block and go into a wait state. The Listen
method waits for an application event, and the SendEventObject
waits for all Listen
methods to complete before continuing to do some housekeeping.
Event Caching
As I mentioned above, we need a mechanism to cache events until we can ensure that all clients have received them. This is achieved logically, not technologically.
All events are cached in a global list of application event objects called EventList
. By retaining an incremental EventId
for each event, and what I call an EventsListResetID
, both globally and on the event object being pushed back, I can determine if events occurred since my last Listen.
The PrepareEventForSending
method returns the events to the client from the point it last received an event.
internal static EventObject[] PrepareEventsForSending(int myLastEventID,
int myLastEventsListResetId) {
if (myLastEventsListResetId == Global.EventsListResetId) {
if (myLastEventID > Global.NewEventID)
myLastEventID = 0;
}
else
myLastEventID = 0;
int numberOfEventsImMissing = Global.NewEventID - myLastEventID;
int startIndex = Global.EventsList.Count - numberOfEventsImMissing;
int returnIndex = 0;
EventObject[] eventToReturn = new EventObject[numberOfEventsImMissing];
for (int missingEventIndex = startIndex;
missingEventIndex < Global.EventsList.Count;
missingEventIndex++) {
eventToReturn[returnIndex] = Global.EventsList[missingEventIndex];
eventToReturn[returnIndex].EventID = missingEventIndex + 1;
eventToReturn[returnIndex].EventsListResetID = Global.EventsListResetId;
returnIndex++;
}
return eventToReturn;
The EventsListResetId
gets incremented every time the EventList
gets cleared. The EventList
gets cleared after all registered clients receive an event.
You’ll notice that when the client’s last event list ID doesn’t match the global one on the server, we return the entire list to the client. That’s because the mismatch indicates the client’s last EventID
(which acts as an index on the list) is no longer valid because the list has been cleared since the last time the client was listening for events.
In other words, a mismatch between the client's last EventsListResetId
and the server’s EventsListResetId
means that the last time you got an event, everybody got the event, and we cleared the EventList
. So, your EventID
is no longer valid, and can now start from zero, meaning, give me the events from the beginning of the new list.
Clearing the list is done at the end of the SendEventObject
method by comparing the number of logged on users to the number of registered waithandles that are blocking a listener. If both are equal, then it’s assumed that all logged-on users received the last event and any outstanding events, so we can clear the list and increment the EventsListResetID
.
Global.AllListenersReleased.WaitOne();
lock (Global.LockObj) {
Global.LastEventID = Global.NewEventID;
if (Global.loggedInUsers.Count == Global.WaitingListenerList.Count) {
Global.EventsList.Clear();
Global.NewEventID = 0;
Global.EventsListResetId++;
}
Global.WaitingListenerList.Clear();
Request Timeouts
This part is easy. In the Global.asa, I determine the request time, and take 90% of it to make up my EventWaitTimeout
.
protected void Application_Start(object sender, EventArgs e)
{
Global.EventWaitTimeout =
(int)(HttpContext.Current.Server.ScriptTimeout * .90) * 1000;
}
In the Listen
web method, a block is set with this value as a time limit. When the limit is reached, the method stops blocking, and a dummy event is set to everybody using the same mechanism we used to send real events (that’s why everyone will be notified). The event class used for dummy events is the parent class to all my event classes. The client knows to ignore these events, and re-requests a Listen.
listenTimedOut = !waithandle.WaitOne(Global.EventWaitTimeout, true);
System.Diagnostics.Trace.WriteLine("Listen - Waithandle released");
if (listenTimedOut) {
if (Global.EventsList.Count == 0) {
EventObject dummyEvent = new EventObject();
lock (Global.LockObj) {
dummyEvent.EventID = myLastEventID;
dummyEvent.EventsListResetID = myLastEventsListResetId;
}
SendDummyEventDelegate sendEventDel =
new SendDummyEventDelegate(EventUtils.SendEventObject);
sendEventDel.BeginInvoke(dummyEvent, null, null);
waithandle.WaitOne();
}
}
Conclusion
With a minimal amount of code, you can create servers that serve up real-time events to your client applications. Although I haven’t tried it, I’m sure this same technique can be used for web-clients using AJAX and page methods, or at least web service methods.
Note: If you’re using the ASP.NET development server to run the sample app, you’ll need to stop it between runs.
References