The classic way of multithreading
The ability to have parallel execution paths within a process is implemented in Microsoft Windows since the
first days of Windows NT. Most programmers know this feature from avoiding it. Multithreaded applications are hard to debug and errors
can manifest themselves in multiple ways. Nevertheless it's one of the most powerful features for building responsive user interfaces
or server applications that scale well on multi-CPU/multi-core systems.
There are many threading libraries that try to make the implementation of threaded applications easier. But mostly
they focus on how a thread is created and how its lifetime is managed. Many of them suffer from the same problem: which function
can be called from which thread is mainly a convention. For example, you can use BeginInvoke in C# to call functions on another thread.
But no one hinders you to call the function directly. The client is responsible to call all methods of a library from the right thread.
COM changed that by introducing different threading models for each apartment. This was coarse but it allowed a library to declare how
threading should be handled. The framework took care of ensuring the right behaviour. In the post COM era, this approach vanished and
again the client became responsible for making sure the threading conventions of the callee are respected.
This article describes a new way of writing multithreaded applications. To clarify my point, let's assume that we implement a class called FooBarBaz:
class FooBarBaz
{
public void Foo() {}
public void Bar(int numberOfBars) {}
public void Baz(String howToBaz, int numberOfBaz) {}
}
The documentation states that Bar and Baz are not thread-safe and must be called from the same thread that instantiated the class.
Foo is said to be thread-safe and can be called from any thread.
Now, while implementing release 2.0 of the assembly, implementing Foo in a thread-safe way becomes more and more complicated. You want to drop this feature for the next release.
Simply put: you can't. There may be many applications that use Foo from different threads and all of them need to be changed. I would say this violates the principle of isolation.
The caller and the callee are tightly coupled by their threading behaviour.
Wouldn't it be better to simply declare how the different methods of FooBarBaz should be called and have a framework
that takes care of ensuring that they are called in the right thread? This would decouple the caller from the callee because the framework
takes care of the threading. If the threading ability of the callee changes, the framework figures out how to call the assembly
in the correct way.
Putting post-its on your code
The ThreadBinding library is my approach to implement declarative multithreading on top of the ContextBoundObject
class of the .NET Framework. The following code snippet shows how the FooBarBaz class would look if it used declarative threading:
[ThreadBound(ThreadBoundAttribute.ThreadBinding.WorkerContext)]
class FooBarBaz :
ThreadBoundObject
{
[FreeThreaded]
public void Foo() {}
public void Bar(int numberOfBars) {}
public void Baz(String howToBaz, int numberOfBaz) {}
}
Now the thread binding system will create the class on its own thread (WorkerContext) and marshal all method calls to Bar and Baz to this thread.
This makes sure that all of these calls take place on the same thread. The method Foo, marked with the FreeThreaded attribute, will be executed on the calling thread.
Now that we have a worker thread for our class, we can call Bar asynchronously from our main thread. This way we can
continue with drawing our user interface without waiting for the function to finish.
[ThreadBound(ThreadBoundAttribute.ThreadBinding.WorkerContext)]
class FooBarBaz :
ThreadBoundObject
{
[FreeThreaded]
public void Foo() {}
[AsyncThreaded]
public void Bar(int numberOfBars) {}
public void Baz(String howToBaz, int numberOfBaz) {}
}
Now the class declares that Bar can be called asynchronously. The framework takes care of marshalling the call
parameters to the worker thread. Speed is ensured by using a thin wrapper that does not serialize the data. If there are asynchronous
methods mixed with synchronous calls, the framework executes them in order. In the “worst case”, an synchronous call following ten
asynchronous ones has to wait for all of them to finish before executing. But this way the caller and the callee can be sure that
the order of calls is not influenced by the threading preferences of each side.
Simply put: The class/assembly only declares how it wants to be called and how threads should be used. The rest is done by the framework. The client does not need to know from which
thread the methods should be called. The framework intercepts the calls and marshals them to the correct thread – or executes them right away.
Bring on some examples
To go a little deeper into the details of the ThreadBinding library, let's create an example project. It's using a very “sophisticated” service that generates an increasing series
of numbers. The delay between the numbers can be configured when calling the method on the service class. We will use declarative
threading to do some asynchronous calls and to make sure that the GUI updates are done within the right thread (the GUI thread).
First we create a form with some buttons and a list control that receives the numbers.
using System;
using System.Windows.Forms;
using ThreadBound;
namespace FormsTest
{
public interface IUpdateInterface
{
void NewNumber(int number);
}
public partial class MainForm :
Form,
IUpdateInterface
{
private NumberWorker numWorker;
private IUpdateInterface threadBoundUpdater;
public MainForm()
{
InitializeComponent();
numWorker= new NumberWorker();
threadBoundUpdater = this.BindInterfaceToThread<IUpdateInterface>();
numWorker.NewNumber += threadBoundUpdater.NewNumber;
}
private void BtnStart_Click(object sender, EventArgs e)
{
numWorker.CreateNumbers(10, 500, new Test());
}
private void BtnCancel_Click(object sender, EventArgs e)
{
numWorker.Cancel();
}
public void NewNumber(int number)
{
LBNumbers.Items.Add(number.ToString());
}
private void BtnDispose_Click(object sender, EventArgs e)
{
numWorker.Dispose();
}
}
}
This is relatively straightforward. Creating the class and defining everything needed. The only “strange” thing is
the IUpdateInterface interface. Let's ignore it for a little while. I'll explain this in a minute. First we create the worker class:
using System;
using ThreadBound;
using System.Threading;
namespace FormsTest
{
[ThreadBound(ThreadBoundAttribute.ThreadBinding.WorkerContext)]
class NumberWorker : ThreadBoundObject, IDisposable
{
private volatile bool CancelFlag=false;
public delegate void NewNumberHandler(int number);
public event NewNumberHandler NewNumber;
[AsyncThreaded]
public void CreateNumbers(int limit, int delay)
{
CancelFlag = false;
for (int i = 0; i <= limit; i++)
{
Thread.Sleep(delay);
if (NewNumber!=null)
NewNumber(i);
if (CancelFlag)
break;
}
}
[FreeThreaded]
public void Cancel()
{
CancelFlag = true;
}
public void Dispose()
{
}
}
}
As you can see, the service implementation does not contain much code. Let's start our analysis with the service class. It must be derived from ThreadBoundObject
or ContextBoundObject. This is necessary for the whole thread binding mechanism to work. The difference between
ThreadBoundObject and ContextBoundObject will also be explained later.
The next thing to note is the ThreadBound-attribute. It declares the threading style as ThreadBinding.WorkerContext. The ThreadBinding library will create
a separate thread for each instance of the class. There are currently three threading styles implemented:
ThreadBinding.WorkerContext: Executes all method calls that are not marked with the FreeThreaded-attribute
on a separate thread. Each instance of the class has its own thread.ThreadBinding.CurrentContext: Executes all method calls that are not marked with the FreeThreaded-attribute on the current thread.
This can only be used if the current thread has an execution context assigned. Currently only WPF or WinForms GUI-threads support instantiating classes with this attribute.ThreadBinding.PoolContext: Executes all method calls that are not marked with the FreeThreaded-attribute on a thread-pool thread.
Every method call (regardless if it's synchronous or asynchronous) may be executed on different thread-pool threads. The ThreadBound library does not guarantee any thread affinity
besides that the executing thread will be a thread-pool thread.
The AsyncThreaded-attribute marks the CreateNumbers method as asynchronous. The ThreadBinding library will call this class on the worker thread. The call will return
immediately and the method will execute on the worker thread for this instance. It's obvious that an asynchronous method can't return a value. Therefore all asynchronous
methods must return void. If they don't, a ThreadBoundException will be thrown. As we'll see later, there is an exception to this rule.
The Cancel method is marked with the FreeThreaded attribute. This method will always execute within the calling thread's context.
It sets the volatile variable CancelFlag to true. The worker method periodically checks the cancel flag and stops executing if the flag is set.
The NewNumberHandler event is called from the worker thread. If there is no thread binding in place on the receiving side, the event will be executed within the worker thread.
But this can be changed declaratively as well.
Up to this point everything is easy. But there is one more thing that needs a little extra attention. How does the thread binding mechanism know when to terminate the worker thread?
You may have guessed it already. The Dispose method is responsible for terminating the worker thread. It has to be there, even if it's – like in this example – empty.
The ThreadBinding library detects the call to Dispose and terminates the thread after the Dispose method has been executed on the worker thread.
If the object is not derived from IDisposible, the worker thread will eventually terminate at the end of the process. If you've a singleton object that has
to exist throughout the runtime of your application, you may omit implementing IDisposible.
Now back to the main form. The NewNumberHandler method is the method that is assigned to the NewNumber event. But if we'd assign it directly,
the method will be called on our worker thread. This is not desired because GUI updates have to be done on the GUI thread. It's a special coincidence that
the Form class is derived from ContextBoundObject. But the WPF classes are not. Let's assume that the Form class is not derived from
ContextBoundObject so I can show how the ThreadBinding library handles this case.
As stated earlier, the magic of the ThreadBinding library is based on the ContextBoundObject class. Every class that should be bound to a thread has to be derived from it.
To allow binding methods of classes, that are not derived from ContextBoundObject, to a thread, a little trick has to be used. The BindInterfaceToThread extension method
can bind any interface to the current thread as long as the current thread has a current context.
But how does it work? BindInterfaceToThread creates a wrapper that implements all methods of the specified interface. This wrapper class is derived from
ContextBoundObject so all method calls can be intercepted and transferred to the correct thread. All of the method implementations eventually call the methods
of the real interface. This way any interface implementation can be bound to a thread. You just have to create the interface wrapper via this.BindInterfaceToThread
and use the returned reference instead of the this reference.
This is why the MainForm class implements the IUpdateInterface.
This interface is bound to the current thread and then the NewNumber method of the wrapped interface is assigned to the event of the NumberWorker.
As you can see, NumberWorker and MainForm are specifying how threading should be handled. MainForm hands over
the NumberWorker a NewNumber callback that will automatically transfer the call back to the GUI thread. NumberWorker has declared
the CreateNumbers method to be asynchronous and this way the GUI thread won't be blocked while the number worker does its job.
That's the primary idea behind declarative threading: Just declare what you want and let the framework do the rest.
Managing asynchronous method calls
The ThreadBinding library makes sure the method calls are always executed in order. This way the caller and the callee are coupled as loosely as possible. But this creates some
additional problems. A simple cancel flag like the one used in the first example can not be used to cancel an asynchronous method that's still queued.
How could the caller cancel an asynchronous call regardless of the call being currently executed or if it's still waiting for execution?
For this case, the ThreadBinding library allows an asynchronous method to return an IAsyncCallState interface.
using System;
using ThreadBound;
using System.Threading;
namespace FormsTest
{
[ThreadBound(ThreadBoundAttribute.ThreadBinding.WorkerContext)]
class NumberWorker : ThreadBoundObject, IDisposable
{
public delegate void NewNumberHandler(int number);
public event NewNumberHandler NewNumber;
[AsyncThreaded]
public IAsyncCallState CreateNumbers(int limit, int delay)
{
for (int i = 0; i <= limit; i++)
{
Thread.Sleep(delay);
if (NewNumber != null)
NewNumber(i);
if (WasCanceled())
break;
}
return null;
}
public void Dispose()
{
}
}
}
This implementation of NumberWorker resembles the previous one. The main difference is that the Cancel method has been removed and that CreateNumbers
now returns an IAsyncCallState interface.
“But CreateNumbers always returns null”, you might say. This is right if you call it without the AsyncThreaded attribute. As previously stated,
the ThreadBinding library can't provide a return value for an asynchronous call. But if the call returns IAsyncThreaded, a synthetic return value is created. The returned interface
can be used to cancel the method call.
If the call is currently not executing, execution is simply cancelled. The method is never called. If the method is currently executing,
the WasCanceled member of the ThreadBoundObject base class returns true. That's the difference between ContextBoundObject and
ThreadBoundObject. ThreadBoundObject implements WasCanceled. On a synchronous method, WasCanceled always returns false.
If the method call is asynchronous, the ThreadBinding library makes sure that the correct cancel state for the currently executing method is shown.
This way every method in the queue can be cancelled regardless of its current execution state.
Let's update the example to use the new NumberWorker:
using System;
using System.Windows.Forms;
using ThreadBound;
namespace FormsTest
{
public interface IUpdateInterface
{
void NewNumber(int number);
}
public partial class MainForm :
Form,
IUpdateInterface
{
private NumberWorker numWorker;
private IUpdateInterface threadBoundUpdater;
private ThreadBound.IAsyncCallState callState;
public MainForm()
{
InitializeComponent();
numWorker= new NumberWorker();
threadBoundUpdater = this.BindInterfaceToThread<IUpdateInterface>();
numWorker.NewNumber += threadBoundUpdater.NewNumber;
}
private void BtnStart_Click(object sender, EventArgs e)
{
callState = numWorker.CreateNumbers(10, 500, new Test());
}
private void BtnCancel_Click(object sender, EventArgs e)
{
if (callState != null)
callState.Cancel();
}
public void NewNumber(int number)
{
LBNumbers.Items.Add(number.ToString());
}
private void BtnDispose_Click(object sender, EventArgs e)
{
numWorker.Dispose();
}
}
}
Note that the BtnCancel_Click method checks if callState is null. This can happen if the call is not marked asynchronous.
Then the ThreadBinding library will not create a synthetic return value and the real null value will be returned. For the best decoupling between your code and the called assembly,
you should be prepared to receive null as the return value of an asynchronous call.
In a real application, you'd queue the returned IAsyncCallState interface references to allow cancelling each call independently.
Internal affairs
There is one thing the interception mechanism can't do: It can't intercept calls within your own class. This should be no problem, because within your class, you control the execution
flow. But it may happen that you call a thread bound method from a free threaded one. Because the framework can not intercept this
method call, the thread bound method will be called from the current thread of the free threaded method.
This allows you to call internal methods from differently threaded source methods. But you can also corrupt your internal state by accidentally crossing thread boundaries.
You should be extra cautious if you use the FreeThreaded attribute and call other class methods.
Maybe later versions of this library will incorporate a safe “cross threading style” call mechanism.
Under the hood
Within this chapter, I'll explain the structure of the source code and how the different parts work together. I don't want to bore you with function calls and function
descriptions, it's more a high altitude view of the source code to get you started. For a more detailed description, refer to the comments within the source code.
The ThreadBound attribute
For the ThreadBinding library to work and do its magic, it's necessary to be able to intercept method calls made to an object. This way the thread binding mechanism can decide what to do
with the function call.
ThreadBoundAttribute is derived from ProxyAttribute. If a class is marked with ProxyAttribute or a derived attribute, the instantiation of the
class is intercepted and CreateInstance of the ProxyAttribute is called.
That's the hook used by the ThreadBinding library. The CreateInstance method then checks the thread binding type passed within the constructor and instantiates
or fetches the required SynchronizationContext.
The next step is creating a new ThreadBoundProxy instance and fetching its transparent proxy implementation.
This transparent proxy is returned to the caller instead of the real instance.
The calling application does not notice the interception. It simply uses the returned transparent proxy as if it was the real instance created by the new call.
This way the developer using this type of interception has not to worry about how to create a new instance. Everything works like before: Call new on the class and you're done.
Handling intercepted method calls
After a method call is intercepted, there are the following possibilities to process the call further:
- We're inside the correct context (thread): Just forward the method call directly to the
InvokeMethod method. A SyncRemoteMethod instance is created
to transport the parameters and the return value. - The method is marked with the
FreeThreaded attribute: Just forward the method call directly to the InvokeMethod method.
A SyncRemoteMethod instance is created to transport the parameters and the return value.
- We're inside the wrong context (thread): In this case, we've two additional options:
- The method is marked with the
AsyncThreaded attribute (which is derived from the OneWay attribute): Do some checks to make sure the method
conforms to the rules for asynchronous methods. Create an instance of AsyncRemoteMethod to wrap the method call. Then call InvokeMethod
via the Post method of the execution context. The AsyncRemoteMethod instance is passed as the “state” parameter. If the methods
returns an IAsyncCallState interface, the AsyncRemoteMethod instance will be cast to IAsyncCallSate and returned as a synthetic return value. - The method is not marked with the
AsyncThreaded attribute: Create a SyncRemoteMethod instance to wrap the method call. Then use the
Send method of the execution context to call InvokeMethod. The SyncRemoteMethod instance is passed as the “state” parameter.
The Send call will not return until the method call has completed. This way the return value of the method call can be put into the passed SyncRemoteMethod
instance. The stored return value is returned by the proxy. This way a synchronous method call can return a value.
The CheckForShutdown method performs a special task. It is called after the method call has been processed. It implements the shutdown logic for the execution context.
Some execution contexts implement an IDispoable interface. The ThreadBinding library needs to know if the execution context is no longer needed.
That's the purpose of the CheckForShutdown method. It detects if Dispose is called on the proxied instance and checks if the execution context implements
IDisposable. If the context is disposable, Dispose is called to tear down the execution context.
Synchronization contexts
If you use the ThreadBinding library within a WinForms or WPF application, there is already an execution context that is automatically created by the .NET Framework.
It is used if you set the CurrentContext flag for the ThreadBound attribute. For all the other types of thread binding,
the ThreadBinding library implements its own synchronization contexts:
WorkerSynchronizationContext: Used for the ThreadBinding.WorkerContext flag. Implements a worker thread and a FIFO buffer for the work items.
This guarantees the method call order is preserved. The Worker class implements the background worker thread. The implementation bears nothing spectacular
besides the global done event. Every thread has its own done event that is stored within a thread local variable. Details are explained within the comments.PoolSynchronizationContext: Really simple execution context. Every method call is executed on a thread pool thread. Synchronous calls wait for their completion.
Asynchronous calls are dispatched onto the worker threads. This execution context does not guarantee any order of execution!
Implementation of AsyncCallState
The implementation of IAsyncCallState – especially the cancellation of asynchronous method calls – demands special attention. The currently running method must be able to query
its cancellation state. Because all classes are derived from ThreadBoundObject, the easiest way is to supply a WasCanceled method
via the base class. If you do this, the base class must be able to get the state information for the currently running method. To avoid
the overhead of extracting the class pointer from the method call information, a different approach is used:
The IRemoteMessage interface of the currently running AsyncRemoteMethod instance is saved into the thread static value
ThreadStatic.currentMethod by the InvokeMethod method. This way a call to ThreadBoundObject::WasCanceled() is able to access
the WasCanceled method of the IRemoteMessage interface assigned to the current thread. Because a thread can only run one method at a time,
it is sufficient to have one thread static value to store the IRemoteMessage interface of the currently running method. After ExecuteMethod has finished
running the method stored inside the AsyncRemoteMethod instance, the thread static value is reset to null.
The CallStates.Canceled member is marked as volatile to allow setting its value via the IAsyncCallState.Cancel()
method and querying it from the thread that is executing the method. If the call state is set to CallStates.Canceled,
the WasCanceled() method of AsyncRemoteMethod will return true.
If you call IAsyncCallState.Cancel() before the method starts executing, the method call is simply skipped. This is
done by the InvokeMethod method of the ThreadBoundProxy. This method tries to set the current state of the AsyncRemoteMethod instance to
CallStates.Running. If this fails (because a canceled method can't put into the running state), the call is not executed.
The interface proxy generator
The class InterfaceProxyGenerator implements the magic to create a class that is derived from ThreadBoundObject and implement a specific interface.
As previously described, all method calls are just forwarded to the real interface. The InterfaceProxyGenerator class is a static class that is shared
by all intercepted interfaces. The static constructor of the class generates an appdomain that is the home of all dynamically generated wrapper classes.
InterfaceProxyGenerator uses a cache for the generated wrapper classes. This reduces the penalty of the dynamic class creation to the first creation
of a wrapper for a specific interface. Wrapping the interface another time is much faster because the class is already cached and only needs to be instantiated.
A call to CreateProxy creates the wrapper and returns the corresponding MarshalByRefObject. The method gets the reference to the real interface as a parameter.
The wrapper class is generated using the MethodBuilder and ILGenerator classes. The interface is scanned via Reflection and then the necessary methods are generated.
Every wrapper method works the same:
- Get the real interface's reference and push it onto the stack.
- Push all the parameters that were passed to the original method onto the stack.
- Call the original method.
The details of the wrapper generator can be found within the GenerateProxyMethod method. The generation of the constructor is handled
by the GenerateProxyCtor method. It creates a constructor that saves the passed real interface reference into a private field of the wrapper class.
After the wrapper class is generated or pulled from the cache, Activator.CreateInstance is called to create an instance of the wrapper class. This instance is returned to the
caller for further usage.
Stay tuned
This proof of concept library is very useful to provide easy threading for common cases. It allows you to build responsive GUIs without much effort. But while testing this library,
I noticed some shortcomings:
- The current state of an asynchronous operation can not be determined.
- It's not possible to wait for an asynchronous operation to finish.
- Handling uncaught exceptions within synchronous or asynchronous methods.
- A safe mechanism for calling private methods with a different threading style.
I think these limitations can be removed by extending IAsyncCallState a little bit.
Summary
I hope this article was able to clarify the idea of declarative threading. But even though this library makes threading a lot easier, it comes with the same warning that I think
every threading library should come with:
You have to understand how this library works and what it can and can't do. Humans are not designed to grasp all
aspects of multithreading easily because our brain is generally single threaded. We can only focus on one task at a time. Be careful
when implementing parallel execution paths. This library makes shooting ourselves in the foot easy and elegant. But it won't relieve the pain...
Because of the call interception technique used, the thread binding library is surely not the right choice for high
performance server applications. That's the domain of other threading models already implemented within the .NET Framework.
Revision history
- 22.11.2011: First release.
- 23.11.2011: Updated the source code and translated the missing comments.
- 13.03.2012: Found and fixed wrong indentation within "Handling intercepted method calls"
1 OK, I admit that this is a little bit oversimplified to get the point across.
If the threading behaviour changes unexpectedly, deadlocks may occur and other strange things may happen. The application using an assembly must always respect the threading behaviour
of the called methods. To write a truly threading agnostic client might be a real challenge (or an impossible task).
2 Remember: A WinForms form is derived from ContextBoundObject and could be thread bound right away.
But I pretend that this is not possible, to show you how to bind any interface with BindInterfaceToThread.
