Fluent Method and Type Builder

• Download sample - 47.2 KB
• You can contribute to this library by using CodePlex

NuGet

Thanks to Mackenzie Zastrow, there's now a NuGet Package. You can find it at http://www.nuget.org/packages/Pfz.TypeBuilding/[^]

Example

Even if the following example looks like Basic code, it is actually C# code used to generate a dynamic method that will read a text file and write all non-empty lines to the console:

string fileName = null;
string line = null;
StreamReader streamReader = null;

var method = 
  new FluentMethodBuilder(typeof(void)).
  AddParameter(() => fileName).
  AddLocal(() => streamReader).
  AddLocal(() => line).
  Body.
    Using(() => streamReader, () => new StreamReader(fileName)).
      Loop().
        Assign(() => line, () => streamReader.ReadLine()).
        If(() => line == null).
          Break().
        EndIf().
        If(() => line.Length > 0).
          Do(() => Console.WriteLine(line)).
        EndIf().
      EndLoop().
    EndUsing().
  EndBody().
  Compile<Action<string>>();

method(@"Test.txt");

Background

I love dynamic code, and I am not talking about the dynamic keyword, which allows us to access existing objects in a dynamic manner, I am talking about code that is generated at run-time. This opens a lot of possibilities, in special to implement classes that have a hard to follow or big pattern, also opening the possibility to do Aspect Oriented Programming easily.

I can say that I used the following solutions to generate dynamic code:

• Generate a unit at run-time, save it to the disk, call the compiler to do the job and finally load the library (I did that in Delphi and C++ Builder 10 years ago);
• Generate a unit at run-time, use CodeDOM to compile it (in fact, that's using CodeDOM in the inverse sense of its purpose, but it works) and finally load the library;
• Reflection.Emit to create dynamic methods and modules that are directly loaded without having to write a file to disk (but using much harder code);
• And using Linq expressions that can be compiled.

And to avoid to continue repeating myself I wrote some solutions to make it easier to generate a new type at run-time, be it by implementing an interface and calling a proxy object to process the call or by using the DelegatedTypeBuilder, which allows new methods to be created having their implementations given by a delegate.

Yet there were still many problems: The solution that implements an interface only redirects interface declared methods to the proxy object, forbidding users from creating new properties at run-time and also requiring a single method to be capable of dealing with any of the run-time generated methods, using parameters in an untyped manner. The DelegatedTypeBuilder required a delegate that was already capable of doing all the code, which can be fine for small patterns but it is extremely limited if new fields are required or even if the generated method can be written in different manners.

Also, there are the natural limitations of the Microsoft solutions. LINQ expressions are only compiled as dynamic methods, not entire types. Reflection.Emit dynamic modules can't call private members of other libraries, not even the library that created it, which forced me to make many things public when they shouldn't be. So, considering all the limitations, none of the existing solutions is really helpful when I simply want to create a new dynamic method fast and without constraints or a new type that had to deal with 2 or more fields per property. So, in a moment of inspiration (or maybe delirium, as I was sick and with fever) I decided to create something new and easier to use.

And that something new is a fluent type and method builder.

How it works - Very short explanation

I am not going to explain in details the inner workings of the library, but the main points are:

• LINQ expressions are already compilable but the C# compiler don't support all the constructs when parsing expression (like assigning a variable) so I decided to add methods to make that easier.
• LINQ expressions are also immutable and they receive all their parameters in the constructor. But what I usually want (and I think most people too) is to build a method step by step, effectively adding statements, maybe doing some tests and adding more statements. And that's how the FluentMethodBuilder works. Also, with the Fluent API it can look more natural if you want to build the entire method at once (like I did on the first example).
• LINQ expressions are a single method, not a type, but they can access private fields (and the local variables used in LINQ become private fields). Dynamic modules can create entire types but they can't access private fields. But it is possible to compile a LINQ expression, creating a delegate, and build a dynamic module that calls such delegate.

Performance

If you are worried about performance, well, don't be. The generated code is pretty fast. As it compiles expressions directly in memory, it avoids losing time generating a text file, saving it to disk and then invoking a large compiler (which is what usually happens if you use CodeDOM or similar solutions). Then, if you are compiling an isolated method it will have a single virtual call to be invoked or, if you are compiling an entire type and accessing it through an interface you will have two virtual calls. Even this can be reduced to a single virtual call if you set the respectVisibility parameter of the AddMethod to true, but in such case the generated method will not be able to access internal or private members of its generator.

Using the code

There are three classes that are the heart of this library:

• FluentMethodBuilder: This is the class that you must use, independent if you want to create a single and independent method or a full type;
• FluentTypeBuilder: If you need to create an entire type instead of a single independent method you should start by the FluentTypeBuilder. Each method added to it (and you must remember that property get and set are methods) will use a FluentMethodBuilder;
• FluentExpression: This type is not really mandatory, but it makes it much easier to mutate expressions at run-time and, so, it will make things easier for you.

So, let's view each one of them:

FluentMethodBuilder

If you want to create a method that is not bound to a specific type you should instantiate a FluentMethodBuilder directly. You simply need to give the return type of the resulting method to its constructor.

With such method builder you have the following possibilities:

• Add parameters;
• Add local variables;
• Add external variables;
• Fill its body;
• Compile it.

Adding parameters

To add a parameter you should declare it as a local variable in your code and set the default value to it. This is needed only to make it possible to access such parameter by all the different expressions that will be built by the fluent method calls and the default value is only a validation requirement as we can't let variables without a value when building expression (because the C# compiler will complain) and I opted to check for the default value only to make it clear that it is not receiving the value declared in your local variable.

Then, in the call to AddParameter() you should give an expression that will access such local variable. So, to declare a parameter named name of type string, you should do:

string name = null;
method.AddParameter(() => name);

From that point on, you can make your dynamic method use that parameter by doing something like:

method.Body.Do(() => Console.WriteLine("The name is: " + name));

And when you compile the method it will be compiled as a method that receives a name parameter, of type string, and all accesses that used your local name will, in fact, use that parameter.

Adding local variables

When we write our own code we can usually access the same member twice, execute the same get method twice or we can access it only once, putting it into a local variable, and then access the local variable many times.

Well, in a fluent method you can also add local variables. They work in the exact same manner as the parameters. You should declare a local variable in the builder method, setting it to its default value, only to make such variable accessible by all the expressions that will be used to build the method, then you call AddLocal() with an expression that directly access such variable and that's enough to make that variable acessible to your calls.

Adding external variables

This one was done simply to avoid bugs. When you do something like:

method.Body.Assign(() => variable, () => 5);

The generated expression will naturally have access to such "variable", but it will share such variable with all threads and all calls to the generated method. This is usually not what you want, so you will probably need to call AddLocal().

But what if you want to share such variable? That's how it naturally works, but in my initial tests I was many times forgetting to declare a variable as a variable. The code was working and the errors were only visible if I created other instances. So, to avoid errors, I decided to generate exceptions if you access a field (that's what a local variable used by an expression becomes) without telling how it should be used.

So, if you want it to be shared by the generator code and by all calls of the generated code, you should register such variable with the AddExternal() call. It has exactly the same syntax of the AddParameter() and AddLocal(), but it will only avoid exceptions, as such variable don't need to be compiled as something else.

Fill its body

This is where the magic really happens and this is where this code most deviate from LINQ expressions, even if it uses a lot of them.

The truth is: The Body will become a BlockExpression from LINQ to be compiled. The big difference here is how we can build it. When using normal LINQ expressions we are forced to create the inner parts before the outer parts. In small cases that's not a big deal, but the more complex the expression becomes, the harder it gets to write it (and later to read it).

With the fluent API we have the Body and we can keep "adding" new actions do be done. If you already have an expression (or if you can directly write it in C#) it is enough to call the Do() method, which will incorporate such expression in the method body.

As a fluent API, you can do things like:

method.
  Do(() => Console.WriteLine("Action1")).
  Do(() => Console.WriteLine("Action2"));

Or you can do:

method.Do(() => Console.WriteLine("Action1"));
method.Do(() => Console.WriteLine("Action2"));

So, if you have some conditions before deciding what you will add to your method, you are free to do one thing at a time. You don't need to build the entire expression in a single step.

Also there are many different methods to make things easier or simply overcome the natural limits of the C# compiler expression generation. For example, the C# compiler does not allow us to do this:

method.Do(() => value = 5);

It understands such command but it says that assignment expressions are not supported (well, at least in C# 4, I don't know about version 5 or even about future versions).

So, to solve that problem there are other methods, like Assign(). With it you can do this:

method.Assign(() => value, () => 5);

Note: We are using two expressions here, one to access the variable and one to get the value, yet those two expressions will become one AssignExpression and will be incorporated in a into a single block expression during the compile phase.

And yet more interesting than the Assign() are the methods that create new blocks, like the If(), Try(), Loop(), While() and For().

Talking about the loop methods, that's something I consider bugged in LINQ expressions. I can't add a break or continue directly. I should give them the "target" to the beginning or to the end of the loop and, if I give the wrong target a break may end-up doing a continue and a continue may end-up doing a break. So, they are only doing gotos and shouldn't be different expressions. But I solved that problem, so you can call a Break() without any parameters and it will exit its enclosing loop.

Compiling the method

To compile the method it is enough to call the Compile() method of the FluentMethodBuilder.

If you don't give a delegate type to it, a default one will be chosen. If you give a delegate type, it must be compatible with the method. The validation part is, in fact, done by the LINQ expressions.

You can use the generic Compile<T>() method to avoid giving the type and also doing a cast to the same type on the obtained result if you know the delegate type at compile time.

FluentTypeBuilder

The FluentTypeBuilder does not have a fluent API, yet each added method will use a FluentMethodBuilder and so, the most important part will be fluent.

When creating the FluentTypeBuilder you must give a generic parameter. That is the "base" type that will be used for the generated type. If you don't have any preference, it should be object.

In the constructor you can also pass all the interfaces that the type will implement.

Then, you can do the following:

• Add fields - This uses the same syntax used to add local variables and parameters to methods;
• Add methods;
• Add properties;
• Add events;

And after you add all the code you need you can:

• Compile the type. This will return a Type instance, so you will need to use reflection;
• Call the GetConstructorDelegate(). This will return a delegate that creates instances of the generated type without using reflection, which is much faster and is the ideal way of creating instances of the generated type.

In fact there is not much to explain about the FluentTypeBuilder, as the important part is the method generation (be it real methods or the properties that use get/set methods) and that's the job of the FluentMethodBuilder.

FluentExpression

The FluentMethodBuilder has the Body property, where you can use a fluent API to add new statements. The problem is: When doing an If(), for example, it is possible that you don't have the entire expression that you should give as the If() parameter.

You may want to combine different And() and Or() conditions at run-time, but combining LINQ Expressions is not very easy (and in fact adding any condition using a Lambda expression instead of its inner Body usually generates strange bugs later instead of immediate exceptions).

So, to avoid those problems you can use the FluentExpression.

With it, you can do things like:

var expression = FluentExpression.Create(() => firstCondition);

if (mustCheckSecondCondition)
  expression.And(() => secondCondition);

if (useOrCondition)
  expression.
    Or(() => firstOrCondition).
    Or(() => secondOrCondition);

You can very well do everything you want by combining expressions by hand, but I really consider this much easier to use.

Method Summary

I already gave a brief description of what a FluentMethodBuilder can do. Now, I want to present a brief list of the constructs you can use:

block.Do(() => Console.WriteLine("Test"));

block.Return(() => 5);

block.Assign(() => fieldOrVariable, () => value);

block.
  If(() => x < 10).
    Do(() => Console.WriteLine("x is less than 10.")).
  Else().
    Do(() => Console.WriteLine("x is at least 10.")).
  EndIf();

block.
  Using(() => variable, () => new DisposableObject()).
    Do(() => Console.WriteLine("Here we can use the disposable object.")).
  EndUsing();

block.
  Loop().
    Do(() => Console.WriteLine("This is an infinite loop.")).
  EndLoop();

block.
  While(() => Application.IsRunning).
    Console.WriteLine("New messages will be written while the application is running.").
  EndWhile();

block.
  For(() => i, () => i<10, FluentExpression.Increment(() => i)).
    Do(() => Console.WriteLine(i)).
  EndFor();

block.
  Loop().
    Assign(() => line, () => streamReader.ReadLine()).
    If(() => line == null).
      Break().
    EndIf().
    If(() => line.Length == 0).
      Continue().
    EndIf().
    Do(() => Console.WriteLine(line)).
  EndLoop();

block.
  Try().
    Do(() => MethodThatMayThrowExceptions()).
  Catch(() => ioException).
    Do(() => Console.WriteLine("An IOException was thrown.")).
  Catch(() => exception).
    Do(() => Console.WriteLine("An exception was thrown and it was not an IOException.")).
  Finally().
      Do(() => Console.WriteLine("Finalizing independently if expections were thrown or not.")).
  EndTry();

block.Do(() =>anotherFluentMethod.Call("argument 0", 57, ...));

block.Inline(anotherFluentMethod);

Note: all block statements that I used in the samples refer to an enclosing block. The default one it the Body of the FluentMethodBuilder, but when you do a Loop(), for example, it generates another block, the same way the If, Try, Catch etc generate blocks. Also, at the end of each action it is always possible to replace the ; by a . so you can add a new statement.

Additional Features

The heart of this library is the FluentTypeBuilder. With it, it is real easy to create new types at run-time and add any methods, properties, events and fields you want.

Yet I should say that what I usually do is to implement interfaces at run-time with an specific pattern and so, to avoid doing always the same kind of code that look for all properties, events and methods that should be implemented, I decided to make that easier.

If your purpose is to implement an interface or abstract class at run-time, see the AbstractTypeImplementer class. You should inherit it to do the work and it has abstract methods that you should override that will be called to implement methods, events and properties, and they will use the right types on properties and events so you can build your expression easily.

For example, this is the entire code of the NotifyPropertyChangedImplementer:

using System;
using System.ComponentModel;
using System.Reflection;
using System.Linq.Expressions;
namespace Pfz.TypeBuilding.AbstractTypeImplementers
{
  public sealed class NotifyPropertyChangedImplementer<TAbstract>:
    AbstractTypeImplementer<TAbstract>
  {
    private readonly Expression<Func<PropertyChangedEventHandler>> 
                     _eventHandlerFieldExpression;

    public NotifyPropertyChangedImplementer():
      base(typeof(INotifyPropertyChanged))
    {
    }
    public NotifyPropertyChangedImplementer(
      Expression<Func<PropertyChangedEventHandler>> eventHandlerFieldExpression):
      base(typeof(INotifyPropertyChanged))
    {
      _eventHandlerFieldExpression = eventHandlerFieldExpression;
    }

    private NotifyPropertyChangedGenerator<TAbstract> _generator;
    protected override FluentTypeBuilder<TAbstract> CreateTypeBuilder(Type[] additionalInterfaces)
    {
      var result = base.CreateTypeBuilder(additionalInterfaces);
      _generator = new NotifyPropertyChangedGenerator<TAbstract>(result, _eventHandlerFieldExpression);
      return result;
    }

    protected override void ImplementProperty<T>(
      FluentTypeBuilder<TAbstract> typeBuilder, PropertyInfo property)
    {
      _generator.AddProperty<T>(property.Name);
    }

    protected override void ImplementEvent<T>(
      FluentTypeBuilder<TAbstract> typeBuilder, EventInfo eventInfo)
    {
      if (eventInfo.EventHandlerType != typeof(PropertyChangedEventHandler))
        throw new NotSupportedException();
    }

    protected override void ImplementMethod(
      FluentTypeBuilder<TAbstract> typeBuilder, MethodInfo method)
    {
      throw new NotSupportedException();
    }

    protected override void ImplementIndexerUntyped(
      FluentTypeBuilder<TAbstract> typeBuilder, PropertyInfo indexer)
    {
      throw new NotImplementedException();
    }
  }
}

This class is capable of implementing the abstract properties of classes (or interfaces) so they will follow the pattern that a get is a direct field access and a set should call the PropertyChanged event if the new value is different from the old value.

As you can see, it will throw NotSupportedExceptions if there is any abstract method or indexer or if there's an event that is not the PropertyChanged one. You can also notice that to add the property there is a call to _generator.AddProperty.

Such generator (the NotifyPropertyChangedGenerator generic class) has the only purpose of creating properties that follow the INotifyPropertyChanged pattern, but it has the following capabilities:

• It can add the PropertyChanged event if it is implementing an interface or an abstract class that does not have such event;
• If you are adding new properties to an abstract class that already has such event, you can give an expression to access the event handler field, so the new properties will trigger the already existing event;
• You can tell that it should notify the change of other properties when this one changes (useful if you have calculated properties);
• You can add a property giving the get and set methods. This is not an error, you give get and set methods that don't generate the event and the generator will be capable of implementing the set to verify if the value was changed, call your set and then generate the event. This is useful if you want to create calculated properties, if you want to put a validate on the set or if you want to create a ViewModel that that redirects all gets and sets to a Model adding the change notification.

So, here are some basic examples of how it can be used.

// The following generator does not tells how to access the
// field that stores the PropertyChanged handler, so it is considered
// that such event does not exist and must be declared.
var generator = NotifyPropertyChangedGenerator.Create(typeBuilder);

// The next statement adds a property named Name, which have its 
// own backing field and will only notify its own change.
generator.AddProperty<string>("Name");

// The previous line could be written as:
generator.AddProperty("Name", typeof(string));

// The next line is an example of a property that notifies that more
// than one property was changed:
generator.AddProperty<DateTime>("Birthdate", "Age");

// And again, it could be written as:
generator.AddProperty("Birthdate", typeof(DateTime), "Age");


// And the next one is the most complex case: The get and set
// will be given. In the set, there is a validation before
// actually accepting the value:
int positiveIntField = 0;
typeBuilder.AddField("positiveIntField", () => positiveIntField);

int value = 0;
var setAction = FluentMethodBuilder.CreateAction().
  Body.
    If(() => value < 0).
      Throw(() => new ArgumentException("Value must be greater than 0.")).
    EndIf().
    Assign(() => positiveIntField, () => value).
  EndBody();

generator.AddProperty("PositiveInt", () => 
        positiveIntField, setAction, () => value, "Age");

dynamic keyword

At least in .NET 4, the dynamic keyword is not capable of acessing the types generated at run-time. At first I thought it was my error, as I was generating all methods as private methods, but even when they are public they aren't acessible. I consider this really bizarre, because reflection sees all the properties and methods, WPF is capable of binding on the properties of the run-time generated objects, I can access them through interfaces but the dynamic keyword simply fails to access them.

Even if this is bizarre, I don't see a big problem with this. If you read a property named X you know such property exists, so why not create an interface with such property and ask the run-time generated type to implement it? And, if you really want run-time access to the object (like the one done by DataGrid) it will work too. But I am commenting about this problem because the first thing I tried to do with a dynamically created object was access it through the dynamic keyword, and that failed.

Roslyn

I know that Microsoft created the Roslyn project to expose the C# and VB compilers as services. I've never really used it but I believe someone might think my project is doing the same.

I don't know if Roslyn has a fluent API to create dynamic methods but I believe my library is much smaller and focused than Roslyn. I am not trying to create an entire compiler or exposing components that can be used by a compiler, I am only creating a small library that allows dynamic types and methods to be created easily. If you are using Roslyn and it is working fine for you, I really don't think this library will be useful to you. But if you want something small that will do the job, then this library is probably what you need.

The Sample

The downloadable solution has the library and a small sample. You can consider the sample more of a guide on how to do things than an application. If you want to run it and see something beautiful, well, you will get frustrated. But if you are interested in knowing how to use the library, it may give you some examples.

Different from other articles, the library in this article does not include any other of my personal libraries. It is simply a small library that you can use anywhere you want.

An overview of my tests and decisions

I am definitely not going to write a tutorial on how to write this library, but I will explain some details on how it was implemented.

In the beginning of the article I said that Expressions only compile single methods and that Reflection.Emit dynamic modules can't call private members declared on other libraries (not even the one that created it).

So, to solve the problem, I decided to combine both: I use expressions to compile single methods and then I use Reflection.Emit to create a dynamic module with all the methods I need, to simply redirect to the methods created by the expressions.

I thought that will be directly possible, but the module in which the Expression compiles its method is also not accessible to the dynamic module. Seeing that there is the CompileToMethod() method in the LambdaExpression I tried it, believing that maybe that will solve my problem. But there are two problems with that method:

1. It was simply throwing an exception that didn't told me what was wrong. After searching in the internet I discovered that it can only compile to static methods;
2. I tried to create the static method and redirect from the instance one to the static one, yet the generated code continued to have the limitation of not being able to call the private fields on other modules. This may look like a violation of the visibility traits, but all the local variables that we may use directly in the expressions are naturally compiled as private fields.

But I found a solution: The method in the dynamic module can call the delegates generated by the expressions. So, at the first moment, I was compiling the expressions (each FluentMethodBuilder will generate a single expression and compile it), putting all the delegates into an array and the dynamic module was compiled with a code to access such array, get the right delegate and invoke it.

This worked fine for the first tests, but then the big problem came: How do I access the fields that I will add to my type?

Even if I call DefineField() before compiling the expression, the type is not yet compiled, and the Expressions can only call already compiled types. On the first moment, instead of trying to change the logic I was considering each field to be an access to a ConditionalWeakTable. That worked, but aside from the performance impact of doing that, I didn't like the fact that if I debugged the type, it simple didn't had any fields.

But I was doing one thing wrong: I was compiling the expressions to be able to get their delegate types (so the dynamic module could call it). But the dynamic module only needs to know the delegate types, not the actual delegates.

Considering that the delegate type is defined by the expression parameters and return type, I decided to create a "pre compile". When generating the dynamic module I only generate an empty expression with all the parameters and the return type. With this I can get the delegate type, without trying to really compile the expression that will access fields that don't exist yet. With the delegate types I am able to compile the dynamic module.

Then I visit and replace the expressions that represent a field access with the access to the real field (after all, it is compiled now) and everything works.

So, that's how I build a type composed of many individually compiled methods that are well capable of accessing the same fields. It is important to know that even if the expression methods are "static" it is always possible to give the instance on which they will work as a parameter, which I surely do.

The Fluent interface

The fluent interface of the library was another problem on its own. I started with a "block" of code. It had only some methods (like Do()) with added the expression and returned this.

By returning this the type of the return was of the block itself (the FluentBlockBuilder).

The problem came when I added the If() method, which is also a block. First, I wanted that by calling EndIf() it returned to the right block. Second, as it was a block I thought about inheriting from the FluentBlockBuilder, but as a call to Do() was already returning this cast as FluentBlockBuilder I was losing the Else() method.

Here I had many solutions to the problem:

• I could put all the methods (like Else(), Catch() etc) on the base block, even if they only work on the specific blocks. This will work great when the expression is well written, but then I can only have an EndBlock() method instead of EndIf(), EndTry() etc that are more specific. Also, users will see those Catch(), Else() etc at all times, which can be confusing.
• I could create all the FluentBlockBuilder methods as protected and without a return. Then each inheritor should create all the same methods to call the real method and return the rightly typed this. This will surely work, but having to revisit all the possible block constructs each time a new method is added is terrible and error-prone.
• I could create all the FluentBlockBuilder methods as extension methods. This is something I do in the CTM Model, for example. Extension methods can be generic and so the result-type can be the same of the input type. That is, a method like T Do<T>(T instance) will have a result type of int if instance is int, a result type of string if instance is string and, for my problem, will have a type of FluentIfBuilder if the instance is of type FluentIfBuilder. But I personally try to avoid extension methods as if you are accessing an object without the right using added the editor will not show the extension methods and the compiler will not find them either.
• And finally, the solution that I used. I made the FluentBlockBuilder receive two generic parameters. One of them is the type to which it should return to. That is, if I have a Loop() and then an If(), a call to EndIf() should know it will return to a Loop block. The other one is the actual child type. I should say that in normal situations I would say that someone doing that does not understand OOP or has a terrible understanding of generics, but it works. That's why the FluentIfBuilder has a generic parameter (the type to which it will return) and inherits from FluentBlockBuilder<TParentBuilder, FluentIfBuilder<TParentBuilder>>. It may look that the "if" inherits from "if", but what it is saying is: I inherit from the FluentBlockBuilder, and I want that all results of this to be cast as this specific If block, not as some less precise type. Of course this could be a problem if I created a type X that gives a different type as the second generic argument, but as the FluentBlockBuilder can't be inherited from another library, that's fine.

I am sure there are other things that may be of interest, but those are the most important ones in my opinion.

Version History

• 16, October, 2013. Changed the license to Apache and added a codeplex link;
• 09, June, 2013. Added the respectVisibility parameter to the AddMethod() method and added the sub-topic Performance;
• 07, June, 2013. Initial Version.