Build Lambda Expressions Dynamically

• Download source - 156.6 KB

Introduction

Have you ever tried to provide your users with a way to dynamically build their own query to filter a list? If you ever tried, maybe you found it a little complicated. If you never tried, believe me when I say it could be, at least, tedious to do. But, with the help of LINQ, it does not need to be that hard (indeed, it could be even enjoyable).

I developed something like this with Delphi when I was at college (almost twelve years ago) and, after reading this great article from Fitim Skenderi, I decided to build that application again, but this time, with C# and empowered by LINQ.

Background

Let us imagine we have classes like this:

public enum PersonGender
{
    Male,
    Female
}

public class Person
{
    public int Id { get; set; }
    public string Name { get; set; }
    public PersonGender Gender { get; set; }
    public BirthData Birth { get; set; }
    public List<Contact> Contacts { get; private set; }
    
    public class BirthData
    {
        public DateTime? Date { get; set; }
        public string Country { get; set; }
    }
}

public enum ContactType
{
    Telephone,
    Email
}

public class Contact
{
    public ContactType Type { get; set; }
    public string Value { get; set; }
    public string Comments { get; set; }
}

...and we have to build the code behind a form like this one to filter a list of Person objects:

Apart from the UI specific code to fill this dropdown lists, to add another line with a new set of controls, and so on; most of the work would reside in building the query to filter your application's database. As, nowadays, most database drivers offer support for LINQ, I think it is reasonable we resort to this resource.

The LINQ expression result of the example from the image above would be similar to this:

p => (p.Id >= 2 && p.Id <= 4)
    && (p.Birth != null p.Birth.Country != null)
    && p.Contacts.Any(c => c.Value.EndsWith("@email.com")
    || (p.Name != null && p.Name.Trim().ToLower().Contains("john")))

As it is not the goal of this article to develop the user interface for the proposed problem, but to build the LINQ expression that will query the database, I will not focus on the UI. The code I am providing has the implementation for those things, but it is neither well organised nor optimised. So, if you need that code for some other reason and you find it a little messy, please let me know so I can clean it up later.

https://github.com/dbelmont/ExpressionBuilder

Having said that, let us see how to use the real code.

Using the Code

First of all, let us get acquainted with some parts of the expressions that will appear later in this article:

• ParameterExpression (x): This is the parameter which is passed to the body
• MemberExpression (x.Id): This is a property or field of the parameter type
• ConstantExpression (3): This is a constant value

And to build an expression like this one, we would need a code like this:

using System.Linq.Expressions;

var parameter = Expression.Parameter(typeof(Person), "x");
var member = Expression.Property(parameter, "Id"); //x.Id
var constant = Expression.Constant(3);
var body = Expression.GreaterThanOrEqual(member, constant); //x.Id >= 3
var finalExpression = Expression.Lambda<Func<Person, bool>>(body, param); //x => x.Id >= 3

We can see here that the body of this expression has three basic components: a property, an operation, and a value. If our queries are a group of many simple expressions as this one, we may say that our Filter would be a list of FilterStatements similar to this:

/// <summary>
/// Defines a filter from which a expression will be built.
/// </summary>
public interface IFilter<TClass> where TClass : class
{
    /// <summary>
    /// Group of statements that compose this filter.
    /// </summary>
    List<IFilterStatement> Statements { get; set; }
    /// <summary>
    /// Builds a LINQ expression based upon the statements included in this filter.
    /// </summary>
    /// <returns></returns>
    Expression<Func<TClass, bool>> BuildExpression();
}

/// <summary>
/// Defines how a property should be filtered.
/// </summary>
public interface IFilterStatement
{
    /// <summary>
    /// Name of the property.
    /// </summary>
    string PropertyName { get; set; }
    /// <summary>
    /// Express the interaction between the property and the constant value 
    /// defined in this filter statement.
    /// </summary>
    Operation Operation { get; set; }
    /// <summary>
    /// Constant value that will interact with the property defined in this filter statement.
    /// </summary>
    object Value { get; set; }
}

public enum Operation
{
    EqualTo,
    Contains,
    StartsWith,
    EndsWith,
    NotEqualTo,
    GreaterThan,
    GreaterThanOrEqualTo,
    LessThan,
    LessThanOrEqualTo
}

Back to our simple expression, we would need a code like this to set up our filter:

var filter = new Filter();
filter.Statements.Add(new FilterStatement("Id", Operation.GreaterThanOrEqualTo, 3));
filter.BuildExpression(); //this method will return the expression x => x.Id >= 3

And, here, all the fun begins. This would be the first implementation of the BuildExpression method:

public Expression<Func<TClass, bool>> BuildExpression()
{
    //this is in case the list of statements is empty
    Expression finalExpression = Expression.Constant(true);
    var parameter = Expression.Parameter(typeof(TClass), "x");
    foreach (var statement in Statements)
    {
        var member = Expression.Property(parameter, statement.PropertyName);
        var constant = Expression.Constant(statement.Value);
        Expression expression = null;
        switch (statement.Operation)
        {
            case Operation.Equals:
    			expression = Expression.Equal(member, constant);
    			break;
    		case Operation.GreaterThanOrEquals:
    			expression = Expression.GreaterThanOrEqual(member, constant);
    			break;
    		///and so on...
    	}
    	
    	finalExpression = Expression.AndAlso(finalExpression, expression);
    }
    
    return finalExpression;
}

Besides the incomplete switch statement, there are some issues not covered by this method:

1. It does not handle inner classes properties
2. It does not support the OR logical operator
3. The cyclomatic complexity of this method is not good at all
4. The Contains, StartsWith and EndsWith operations do not have an equivalent method in the System.Linq.Expressions.Expression class
5. The operations with strings are case-sensitive (it would be better if they were case-insensitive)
6. It does not support filtering list-type properties

To address these issues, we will need to modify the GetExpression method and also take some other attitudes:

1. To handle inner classes properties, we will also need a recursive method (a method that calls itself):

   MemberExpression GetMemberExpression(Expression param, string propertyName)
   {
       if (propertyName.Contains("."))
       {
           int index = propertyName.IndexOf(".");
           var subParam = Expression.Property(param, propertyName.Substring(0, index));
           return GetMemberExpression(subParam, propertyName.Substring(index + 1));
       }
       
       return Expression.Property(param, propertyName);
   }

2. To support the OR logical operator, we will add another property to the IFilterStatement that will set how a filter statement will connect to the next one:

   public enum FilterStatementConnector { And, Or }
   
   /// <summary>
   /// Defines how a property should be filtered.
   /// </summary>
   public interface IFilterStatement
   {
       /// <summary>
       /// Establishes how this filter statement will connect to the next one.
       /// </summary>
       FilterStatementConnector Connector { get; set; }
       /// <summary>
       /// Name of the property (or property chain).
       /// </summary>
       string PropertyName { get; set; }
       /// <summary>
       /// Express the interaction between the property and
       /// the constant value defined in this filter statement.
       /// </summary>
       Operation Operation { get; set; }
       /// <summary>
       /// Constant value that will interact with
       /// the property defined in this filter statement.
       /// </summary>
       object Value { get; set; }
   }

3. To decrease to cyclomatic complexity introduced by the long switch statement (we have nine cases, one for each operation) inside the foreach loop, we will create a Dictionary that will map each operation to its respective expression. We will also make use of that new property from the IFilterStatement, and that new GetMemberExpression method:

   //Func<Expression, Expression, Expression> means that this delegate expects two expressions 
   //(namely, a MemberExpression and a ConstantExpression) and returns another one. 
   //This will be clearer when you take a look at the complete/final code.
   readonly Dictionary<Operation, Func<Expression, Expression, Expression>> Expressions;
   
   //this would go inside the constructor
   Expressions = new Dictionary<Operation, 
   Func<Expression, Expression, Expression>>();
   Expressions.Add(Operation.EqualTo,
       (member, constant) => Expression.Equal(member, constant));
   Expressions.Add(Operation.NotEqualTo,
       (member, constant) => Expression.NotEqual(member, constant));
   Expressions.Add(Operation.GreaterThan,
       (member, constant) => Expression.GreaterThan(member, constant));
   Expressions.Add(Operation.GreaterThanOrEqualTo,
       (member, constant) => Expression.GreaterThanOrEqual(member, constant));
   Expressions.Add(Operation.LessThan,
       (member, constant) => Expression.LessThan(member, constant));
   Expressions.Add(Operation.LessThanOrEqualTo,
       (member, constant) => Expression.LessThanOrEqual(member, constant));
   
   public Expression<Func<TClass, bool>> BuildExpression()
   {
       //this is in case the list of statements is empty
       Expression finalExpression = Expression.Constant(true);
       var parameter = Expression.Parameter(typeof(TClass), "x");
       var connector = FilterStatementConnector.And;
       foreach (var statement in Statements)
       {
           //*** handling inner classes properties ***
           var member = GetMemberExpression(parameter, statement.PropertyName);
           var constant = Expression.Constant(statement.Value);
           //*** decreasing the cyclomatic complexity ***
           var expression = Expressions[statement.Operation].Invoke(member, constant);
           //*** supporting the OR logical operator ***
           if (statement.Conector == FilterStatementConector.And)
           {
               finalExpression = Expression.AndAlso(finalExpression, expression);
           }
           else
           {
               finalExpression = Expression.OrElse(finalExpression, expression);
           }
   
           //we must keep the connector of this filter statement to know 
           //how to combine the final expression with the next filter statement
           connector = statement.Connector;
       }
       
       return finalExpression;
   }

4. To support the Contains, StartsWith and EndsWith operations, we will need to get their MethodInfo and create a method call:

   static MethodInfo containsMethod = typeof(string).GetMethod("Contains");
   static MethodInfo startsWithMethod = typeof(string)
       .GetMethod("StartsWith", new [] { typeof(string) });
   static MethodInfo endsWithMethod = typeof(string)
       .GetMethod("EndsWith", new [] { typeof(string) });
   
   //at the constructor
   Expressions = new Dictionary<Operation, 
   Func<Expression, Expression, Expression>>();
   //...
   Expressions.Add(Operation.Contains,
       (member, constant) => Expression.Call(member, containsMethod, expression));
   Expressions.Add(Operation.StartsWith,
       (member, constant) => Expression.Call(member, startsWithMethod, constant));
   Expressions.Add(Operation.EndsWith,
       (member, constant) => Expression.Call(member, endsWithMethod, constant));

5. To make the operations with strings case-insensitive, we will use the ToLower() method and also the Trim() method (to remove any unnecessary whitespace). So, we will have to add their MethodInfo as well. Additionally, we will change the BuildExpression method a little.

   static MethodInfo containsMethod = typeof(string).GetMethod("Contains");
   static MethodInfo startsWithMethod = typeof(string)
       .GetMethod("StartsWith", new [] { typeof(string) });
   static MethodInfo endsWithMethod = typeof(string)
       .GetMethod("EndsWith", new [] { typeof(string) });
   
   //We need to add the Trim and ToLower MethodInfos
   static MethodInfo trimMethod = typeof(string).GetMethod("Trim", new Type[0]);
   static MethodInfo toLowerMethod = typeof(string).GetMethod("ToLower", new Type[0]);
   
   public Expression<Func<TClass, bool>> BuildExpression()
   {
       //this is in case the list of statements is empty
       Expression finalExpression = Expression.Constant(true);
       var parameter = Expression.Parameter(typeof(TClass), "x");
       var connector = FilterStatementConnector.And;
       foreach (var statement in Statements)
       {
           var member = GetMemberExpression(parameter, statement.PropertyName);
           var constant = Expression.Constant(statement.Value);
           
           if (statement.Value is string)
           {
               // x.Name.Trim()
               var trimMemberCall = Expression.Call(member, trimMethod);
               // x.Name.Trim().ToLower()
               member = Expression.Call(trimMemberCall, toLowerMethod);
               // "John ".Trim()
               var trimConstantCall = Expression.Call(constant, trimMethod); 
               // "John ".Trim().ToLower()
               constant = Expression.Call(trimConstantCall, toLowerMethod); 
           }
           
           var expression = Expressions[statement.Operation].Invoke(member, constant);
           finalExpression = CombineExpressions(finalExpression, expression, connector);
           connector = statement.Connector;
       }
       
       return finalExpression;
   }
   
   Expression CombineExpressions(Expression expr1,
       Expression expr2, FilterStatementConnector connector)
   {
       return connector == FilterStatementConnector.And ? 
   			Expression.AndAlso(expr1, expr2) : Expression.OrElse(expr1, expr2);
   }

6. Last but not least, the support to filter by properties of objects inside of list-type properties was the most difficult requirement to handle (at least, it was for me). The best way I found was to come up with a convention to deal with those properties. The convention adopted was to mention the property inside of brackets right after the name of the list-type property, eg. Contacts[Value] would point to the property Value of each Contact in Person.Contacts. Now the heavy work is going from Contacts[Value] Operation.EndsWith "@email.com" to x.Contacts.Any(i =>i.Value.EndsWith("@email.com"). That is when the method below enters the scene.

   static Expression ProcessListStatement(ParameterExpression param, IFilterStatement statement)
   {
       //Gets the name of the list-type property
       var basePropertyName = statement.PropertyName
           .Substring(0, statement.PropertyName.IndexOf("["));
       //Gets the name of the 'inner' property by 
       //removing the name of the main property and the brackets
       var propertyName = statement.PropertyName
           .Replace(basePropertyName, "")
           .Replace("[", "").Replace("]", "");
   
       //Gets the type of the items in the list, eg. 'Contact'
       var type = param.Type.GetProperty(basePropertyName)
           .PropertyType.GetGenericArguments()[0];
       ParameterExpression listItemParam = Expression.Parameter(type, "i");
       //Gets the expression 'inner' expression:
       // i => i.Value.Trim().ToLower().EndsWith("@email.com".Trim().ToLower())
       var lambda = Expression.Lambda(GetExpression(listItemParam, statement, propertyName),
           listItemParam);
       var member = GetMemberExpression(param, basePropertyName); //x.Contacts
       var tipoEnumerable = typeof(Enumerable);
       //MethodInfo for the 'Any' method
       var anyInfo = tipoEnumerable
           .GetMethods(BindingFlags.Static | BindingFlags.Public)
           .First(m => m.Name == "Any" && m.GetParameters().Count() == 2);
       anyInfo = anyInfo.MakeGenericMethod(type);
       //x.Contacts.Any(i => 
       //    i.Value.Trim().ToLower().EndsWith("@email.com".Trim().ToLower())
       //)
       return Expression.Call(anyInfo, member, lambda); 
   }

    

That's all folks! I hope you had just as much fun reading this as I had writing it. Please feel free to leave any comments, suggestions and/or questions.

Points of Interest

As if it wasn't enjoyable enough, I decided to make another improvement by implementing a fluent interface in the Filter:

/// <summary>
/// Defines a filter from which a expression will be built.
/// </summary>
public interface IFilter<TClass> where TClass : class
{
    /// <summary>
    /// Group of statements that compose this filter.
    /// </summary>
    IEnumerable<IFilterStatement> Statements { get; }
    /// <summary>
    /// Adds another statement to this filter.
    /// </summary>
    /// <param name="propertyName">
    /// Name of the property that will be filtered.</param>
    /// <param name="operation">
    /// Express the interaction between the property and the constant value.</param>
    /// <param name="value">
    /// Constant value that will interact with the property.</param>
    /// <param name="connector">
    /// Establishes how this filter statement will connect to the next one.</param>
    /// <returns>A FilterStatementConnection object that 
    /// defines how this statement will be connected to the next one.</returns>
    IFilterStatementConnection<TClass> By<TPropertyType>
        (string propertyName, Operation operation, TPropertyType value, 
        FilterStatementConnector connector = FilterStatementConnector.And);
    /// <summary>
    /// Removes all statements from this filter.
    /// </summary>
    void Clear();
    /// <summary>
    /// Builds a LINQ expression based upon the statements included in this filter.
    /// </summary>
    /// <returns></returns>
    Expression<Func<TClass, bool>> BuildExpression();
}

public interface IFilterStatementConnection<TClass> where TClass : class
{
    /// <summary>
    /// Defines that the last filter statement will 
    /// connect to the next one using the 'AND' logical operator.
    /// </summary>
    IFilter<TClass> And { get; }
    /// <summary>
    /// Defines that the last filter statement will connect 
    /// to the next one using the 'OR' logical operator.
    /// </summary>
    IFilter<TClass> Or { get; }
}

By doing this, we are able to write more readable filters, as follows:

var filter = new Filter<Person>();
filter.By("Id", Operation.Between, 2, 4)
      .And.By("Birth.Country", Operation.IsNotNull)
      .And.By("Contacts[Value]", Operations.EndsWith, "@email.com")
      .Or.By("Name", Operaions.Contains, " John");

Update - July 2017

Besides some minor UI improvements, these are the main changes to the code:

• Seven new operations added: IsNull, IsNotNull, IsEmpty, IsNotEmpty, IsNullOrWhiteSpace, IsNotNullNorWhiteSpace, and Between;
• Added Globalization support for properties and operations: now you can map the properties' and operations' descriptions into resource files to improve the user experience;
• Some operations were renamed: Equals => EqualTo; NotEquals => NotEqualTo; GreaterThanOrEquals => GreaterThanOrEqualTo; LessThanOrEquals => LessThanOrEqualTo;
• Added support for Nullable properties;
• Code dettached from the UI: what enabled the convertion of the project into a NuGet package.

History

• 26.02.2016 - Initial publication.
• 29.02.2016 - Little adjustment in the penultimate code snippet, some lines were wrongly commented.
• 21.03.2017 - Added a downloadable version of the source code.
• 10.07.2017 - New features
• 23.08.2017 - Added configuration support in order to enable the use of types other than the default ones.