This article explains Linq Expression programming by providing simple examples, making it easy to switch the context from C# to Expression programming when needed.
Introduction
One of the most useful and at the same time poorly documented C# built-in libraries is System.Linq.Expressions. It allows creating complex C# code, compiling it dynamically into lambda expressions and then running those lambda expressions at compiled code speed - many times faster than achieving the same using System.Reflection library.
Expression coding is very different from writing usual C# code and requires a drastic context switch.
I've been working with expressions now and then, not very often, and every time I work with expressions, I have to spend some time to remember the basics of Expression coding.
The purpose of this article is to provide information and samples that will make the context switch into expression coding easier for myself and others in the future.
The code samples for this article are located under Expressions folder of NP.Samples repository at NP.Samples/Expressions on Github.
Simple vs Block Expressions
Simple expressions are those that are built without Expression.Block(...) method. Such expressions are simpler and are often sufficient for basic purposes, but have the following limitations:
- Composed non-block (simple) expressions are build by combining simpler expressions with
static methods from Expression class (aside from Expression.Block(...) method). Such recursive expression building results in what is called expression tree. The resulting code is essentially a one-liner obtained by recursively substituting combinations of simpler expressions into more complex ones. There is no way to properly imitate the line by line code so common for C#. - There is no way to define and initialize local variables for the resulting lambda expressions. You will see it in our sample below that we have to create lambdas with essentially unneeded parameters for non-block expressions.
- One cannot create an Expression wrapping a call to a method with some
ref or out arguments without using Block expression, precisely because only local variables can be used as ref or out arguments.
Simple (non-Block) Expressions
Most of the built-in expression functionality comes from the static methods of Expression class located within System.Linq.Expressions package.
Primitive or built-in Expressions are Expressions built by using static factory methods of the Expression class, for example:
Expression.Constant(5, typeof(int)) will create an expression for a constant 5 of type int.Expression.Parameter(typeof(double), "var1")) will create an expression for a variable named "var1" of type double.
By themselves, the primitive (built-in) expressions are largerly useless, but Expression class also provides static methods to combine them into composed Expressions and to compile the composed Expressions into useful lambdas.
Open a console application SimpleExpressionsExamples/NP.Samples.Expressions.SimpleExpressionsExamples.sln.
Look at the main file Program.cs. The content of the file consists of static methods; each of which corresponds to a single sample. At the end of the file, the commented out methods calls are written in the same order in which they are defined. When you work with a sample, uncomment the corresponding method and run it; when you are done, comment it out back for clarity sake.
Note that every expression has DebugView property visible only within the debugger that displays the expression's code not quite in C#, but very similar and understandable for a C# developer. This property can and should be used for checking and debugging the expressions.
Simplest Expression Samples
In this subsection, we present simple Expression samples without return statements and without loops.
WrongConstantAssignmentSample
The first sample method is called WrongConstantAssignmentSample(). Its name starts with 'Wrong' because it throws an exception.
The method demonstrates assigning a constant to a variable:
static void WrongConstantAssignmentSample()
{
var paramExpression = Expression.Parameter(typeof(int), "myVar");
var constExpression = Expression.Constant(5, typeof(int));
Expression<Action<int>> lambdaExpr =
Expression.Lambda<Action<int>>
(
assignExpression,
paramExpression
);
Expression<Action> lambdaExpr =
Expression.Lambda<Action>(assignExpression);
Action lambda = lambdaExpr.Compile();
lambda();
}
Once you uncomment and run the method call at the bottom of Program.cs file, you shall see that it throws an exception at lambdaExpr.Compile() line and the exception message is:
System.InvalidOperationException: 'variable 'myVar' of type 'System.Int32'
referenced from scope '', but it is not defined'
Get to the WrongConstantAssignmentSample() scope within the debuger, open up lambdaExpr variable in the debugger and take a look at the content of its DebugView property:
.Lambda #Lambda1<System.Action>() {
$myVar = 5
}
As I mentioned above, DebugView property (available only in the debugger) contains a detailed view of the code (not exactly in C# but very close) of the Expression and can be used for debugging and fixing it.
One can immediately see what is wrong with the expression above - the myVar variable was never defined.
There is no way to define a local variable within expressions except for a Block expression which we shall describe later.
Instead, we can define a parameter to the lambda and use it as a variable within the lambda expression as shown below.
CorrectedConstantAssignmentSample
Take a look at CorrectedConstantAssignmentSample() method:
static void CorrectedConstantAssignmentSample()
{
var paramExpression = Expression.Parameter(typeof(int), "myVar");
var constExpression = Expression.Constant(5, typeof(int));
var assignExpression = Expression.Assign(paramExpression, constExpression);
Expression<Action<int>> lambdaExpr =
Expression.Lambda<Action<int>>
(
assignExpression,
paramExpression
);
Action<int> lambda = lambdaExpr.Compile();
lambda(0);
}
Note that the resulting paramExpression is used twice:
- As the left part of the assignment:
Expression.Assign(paramExpression, constExpression);
In C# code, that would be myVar = 5; - As an input parameter for the lambda Expression:
Expression.Lambda<Action<int>>(assignExpression, <b>paramExpression</b>);
In C#, that would look as Lambda(myVar);
Take a look at how we obtain the lambda expression lambdaExpr:
Expression<Action<int>> lambdaExpr =
Expression.Lambda<Action<int>>
(assignExpression, paramExpression );
We use Action<...> as the Type parameter to Expression. The number of types within Action<...> should be equal to the number of input parameters to the lambda (in this sample - it is just one integer parameter, so we have Action<int>).
The arguments that we pass to Expression.Lambda<Action<int>>(...) are: the lambda body expression followed by the input parameters expressions in the same order in which the parameters are passed to the lambda. Of course, the number of input parameter expressions should be the same as the number of type arguments to Action<...> and their types should match.
In this sample - method body is represented by assignExpression while the single integer input parameter is represented by paramExpression.
Here is the DebugView code of lambda expression for this sample:
.Lambda #Lambda1<System.Action`1[System.Int32]>(System.Int32 $myVar) {
$myVar = 5
}
Important Note: As was explained above, we cannot define a local variable without using Block expressions and because of that, myVar variable is defined as an input argument (parameter) to the lambda expression, even though, from C# point of view - the input argument is otherwise completely useless: it cannot produce any change outside of the method since it is passed by value. Defining local variables is one of the reasons to use Block expression - this will be detailed below.
We no longer get an exception, but there is no indications that assignment had really taken place. The program runs without producing any console output.
ConstantAssignmentSampleWithPrintingResult
In the next sample, we demo calling Console.WriteLine(int i) method to print the resulting value of the variable.
static void ConstantAssignmentSampleWithPrintingResult()
{
var paramExpression = Expression.Parameter(typeof(int), "myVar");
var constExpression = Expression.Constant(5, typeof(int));
var assignExpression = Expression.Assign(paramExpression, constExpression);
MethodInfo writeLineMethodInfo =
typeof(Console).GetMethod(nameof(Console.WriteLine), new Type[] {typeof(int)})!;
var callExpression = Expression.Call(writeLineMethodInfo, assignExpression);
Expression<Action<int>> lambdaExpr =
Expression.Lambda<Action<int>>
(
callExpression,
paramExpression
);
Action<int> lambda = lambdaExpr.Compile();
lambda(0);
}
Here is how we add the call to Console.WriteLine(int i) that prints the result of the assignExpression:
MethodInfo writeLineMethodInfo =
typeof(Console).GetMethod(nameof(Console.WriteLine), new Type[] {typeof(int)})!;
var callExpression = Expression.Call(writeLineMethodInfo, assignExpression);
Running this ConstantAssignmentSampleWithPrintingResult() method will print number 5 to console.
The lambda expression's DebugView is:
.Lambda #Lambda1<System.Action`1[System.Int32]>(System.Int32 $myVar) {
.Call System.Console.WriteLine($myVar = 5)
}
The Principle of Building non-Block Expressions
The above samples illustrate the principle of building complex expressions from simpler ones - you pass the simpler expressions to some Expression static methods as arguments.
In the last example, we build an expression callExpression by using Expression.Call(...) method and passing to it assignExpression which is in turn obtained by Expression.Assign(...) method working on two other expressions - paramExpression and constExpression. In short, we have the following Expression tree for the call Expression (which is the body of our Lambda Expression):
Expression trees result in lambdas that recursively pass the results of the parent expressions to the transformation defined by the child expression.
Every expression aside from Block expressions is essentially a one liner - though such line (being a result of combining and expanding other expressions) can be very long.
Non-Block Expressions with Return Value
An Expression (both Simple and Block) can be made to return a value (same as a non-void method). Compiling such an expression would result in Func<...> and not Action<...> lambda.
The only thing you need to do differently is to create a lambda expression that returns a value, is to pass Func<...> instead of Action<...> to Expression.Lambda as the lambda type parameter. As always, Func<...> last type argument should specify the return type.
Of course, there is also a condition that the lambda body Expression returns some value, but most simple expressions do it anyways.
In this subsection, we shall provide examples of such methods.
SimpleReturnConstantSample
Take a look at SimpleReturnConstantSample() static method:
static void SimpleReturnConstantSample()
{
var lambdaExpr = Expression.Lambda<Func<int>>
(Expression.Constant(1234, typeof(int)));
var lambda = lambdaExpr.Compile();
int returnedNumber = lambda();
Console.WriteLine(returnedNumber);
}
Running it will print 1234 to console.
ReturnSumSample
ReturnSumSample() static method sums up two integer input arguments passed to it and returns the result:
static void ReturnSumSample()
{
var i1Expr = Expression.Parameter(typeof(int), "i1");
var i2Expr = Expression.Parameter(typeof(int), "i2");
var sumExpr = Expression.Add(i1Expr, i2Expr);
var sumLambdaExpr =
Expression.Lambda<Func<int, int, int>>
(
sumExpr,
i1Expr,
i2Expr
);
var sumLambda = sumLambdaExpr.Compile();
int i1 = 1, i2 = 2;
int result = sumLambda(i1, i2);
Console.WriteLine($"{i1} + {i2} = {result}");
}
The most interesting parts of the code above are:
- a new simple expression created by
Expression.Add(param1Expr, param2Expr) method - creating the lambda expression:
var sumLambdaExpr =
Expression.Lambda<Func<int, int, int>>
(
sumExpr,
i1Expr,
i2Expr
);
Note that we use Func<int, int, int> and not Action<int, int> as the generic type parameter to Expression.Lambda.
Non-Block Expressions with Loops
Loops allow to create expressions with loop control flow.
LoopSample
Take a look at LoopSample():
static void LoopSample()
{
var loopIdxExpr = Expression.Parameter(typeof(int), "i");
var loopIdxToBreakOnExpr = Expression.Parameter(typeof(int), "loopIdxToBreakOn");
LabelTarget breakLabel = Expression.Label(typeof(int), "breakLoop");
var loopExpression =
Expression.Loop
(
Expression.IfThenElse(
Expression.LessThan(loopIdxExpr,
loopIdxToBreakOnExpr),
Expression.PostIncrementAssign(loopIdxExpr),
Expression.Break(breakLabel, loopIdxExpr)
),
breakLabel
);
var lambdaExpr = Expression.Lambda<Func<int, int, int>>
(
loopExpression,
loopIdxExpr,
loopIdxToBreakOnExpr
);
var lambda = lambdaExpr.Compile();
int result = lambda(0, 5);
Console.WriteLine(result);
}
The simple loop iterates from 0 to 5 and then returns 5. The returned result is printed to console. The loop essentially imitates the following C# code:
while(true)
{
if (i < loopIdxToBreakOn)
i++;
else
break;
}
return i;
Note that variable i is not defined by the loop. This is a common problem - not being able to define a local variable within a non-block expression.
Since we cannot define the loop index i as a local variable, we have to define it as an input argument to the lambda (even though, there is no any other need to do it).
The most interesting part of the code is:
LabelTarget breakLabel = Expression.Label(typeof(int), "breakLoop");
var loopExpression =
Expression.Loop
(
Expression.IfThenElse(
Expression.LessThan(loopIdxExpr,
loopIdxToBreakOnExpr),
Expression.PostIncrementAssign(loopIdxExpr),
Expression.Break(breakLabel, loopIdxExpr)
),
breakLabel
);
Expression.Label allows to create a 'goto' label that will specify where the code execution resumes after breaking from the loop. It can be void or it can have a return type (int in our case).
Expression.IfThenElse(...) combines three other expressions - specifying the loop condition, the loop action and what happens when the loop condition is no longer satisfied. In our case, it results in approximately the following code:
if (i < loopIdxToBreakOn)
{
i++;
}
else
{
return i;
}
LoopForCopyingArrayValuesSample
Our next example LoopForCopyingArrayValuesSample() demostrates creating an expression to copy contents of one int[] array into another. The arrays (sourceArray and targetArraty) are passed to the lambda as input arguments. The arrays are assumed to be non-null and of the same size, but for the sake of simplicity, we do not check it within the expression lambda.
static void LoopForCopyingArrayValuesSample()
{
var sourceArrayExpr = Expression.Parameter(typeof(int[]), "sourceArray");
var targetArrayExpr = Expression.Parameter(typeof(int[]), "targetArray");
var arrayCellIdxExpr = Expression.Parameter(typeof(int), "i");
var arrayLengthExpr =
Expression.ArrayLength(sourceArrayExpr);
var loopLabel = Expression.Label("breakLabel");
var loopExpr =
Expression.Loop
(
Expression.IfThenElse
(
Expression.LessThan(arrayCellIdxExpr,
arrayLengthExpr),
Expression.Assign(
Expression.ArrayAccess(targetArrayExpr,
arrayCellIdxExpr),
Expression.ArrayAccess(sourceArrayExpr,
Expression.PostIncrementAssign
(arrayCellIdxExpr))
),
Expression.Break(loopLabel)
),
loopLabel
);
var arrayCopyLambdaExpr =
Expression.Lambda<Action<int[], int[], int>>
(loopExpr, sourceArrayExpr, targetArrayExpr, arrayCellIdxExpr);
var arrayCopyLambda = arrayCopyLambdaExpr.Compile();
int[] sourceArray = Enumerable.Range(1, 10).ToArray();
int[] targetArray = new int[10];
arrayCopyLambda(sourceArray, targetArray, 0);
Console.WriteLine(string.Join(", ", targetArray));
}
The method will print the contents of the targetArray (numbers from 1 to 10) to console.
Note that as always with non-block expressions, we pass the loop index variable i as a parameter since we cannot define it as a local variable.
The most interesting (new) part of the code is the Assign expression within the IfThenElse expression within the loop:
Expression.Assign(
Expression.ArrayAccess(targetArrayExpr, arrayCellIdxExpr),
Expression.ArrayAccess(sourceArrayExpr,
Expression.PostIncrementAssign(arrayCellIdxExpr))
),
It shows using the ArrayAccess expressions to access the array cells. Here is the resulting C# code
targetArray[i] = sourceArray[i++];
Note that we are forced to increment i within the array index operator [] since without Block expressions, we cannot have more than a single line if statement body.
LoopSumUpNumbersFromToSample
Last non-Block loop sample will show how to sum up consequitive integers from 0 to value specified by to integer variable:
static void LoopSumUpNumbersFromToSample()
{
var loopIdxExpr = Expression.Parameter(typeof(int), "i");
var toExpr = Expression.Parameter(typeof(int), "to");
var resultExpr = Expression.Parameter(typeof(int), "result");
var loopLabel = Expression.Label(typeof(int), "breakLabel");
var loopExpr =
Expression.Loop
(
Expression.IfThenElse
(
Expression.LessThanOrEqual(loopIdxExpr, toExpr),
Expression.AddAssign(resultExpr,
Expression.PostIncrementAssign(loopIdxExpr)),
Expression.Break(loopLabel, resultExpr)
),
loopLabel
);
var sumNumbersFromTooLambdaExpr = Expression.Lambda<Func<int, int, int, int>>
(loopExpr, loopIdxExpr, toExpr, resultExpr);
var sumNumbersFromTooLambda = sumNumbersFromTooLambdaExpr.Compile();
int from = 1;
int to = 10;
var sumResult = sumNumbersFromTooLambda(from, to, 0);
Console.WriteLine($"Sum of intergers from {from} to {to} is {sumResult}");
}
The 'interesting' part of the code is the body of the loop:
Expression.AddAssign(resultExpr,
Expression.PostIncrementAssign(loopIdxExpr)),
which results in the following C# code:
result += i++;
Note since we cannot define and initialize local variable, we have to pass the initial value 0 of result variable to the lambda as an input parameter.
Expressions using Expression.Block Method
As was mentioned in the beginning of this article, Expressions with Block Method (or Block Expressions) are more powerful than simple (non-Block) expressions. In particular, they
- allow line by line statements.
- allow defining local variable that do not need to be passed as input arguments.
- can wrap calls to methods with
ref or out arguments.
All code for Block expression samples is located within Program.cs file of BlockExpressionsExamples/NP.Samples.Expressions.BlockExpressionsExamples.sln solution.
The content of the Program.cs file consists of static methods each of which corresponding to a single sample. At the end of the file, the commented out methods calls are written in the same order in which they are defined. When you work with a sample, uncomment the corresponding method and run it; when you are done, comment it out back for the sake of clarity.
BlockSampleWithLocalVariableAndLineByLineCode
static void BlockSampleWithLocalVariableAndLineByLineCode()
{
var i1Expr = Expression.Parameter(typeof(int), "i1");
var i2Expr = Expression.Parameter(typeof(int), "i2");
var resultExpr = Expression.Parameter(typeof(int), "result");
var blockExpr =
Expression.Block
(
typeof(int),
new ParameterExpression[] { resultExpr },
Expression.Assign(resultExpr,
Expression.Add(i1Expr, i2Expr)),
Expression.MultiplyAssign
(resultExpr, resultExpr),
resultExpr
);
var lambdaExpr = Expression.Lambda<Func<int, int, int>>
(
blockExpr,
i1Expr,
i2Expr
);
var lambda = lambdaExpr.Compile();
int i1 = 1, i2 = 2;
var result = lambda(i1, i2);
Console.WriteLine($"({i1} + {i2}) * ({i1} + {i2}) = {result} ");
}
The resulting C# code of the lambda is approximately:
(int i1, int i2) =>
{
int result;
result = i1 + i2;
result *= result;
return result;
}
Of course, the same could have been achieved by a non-block expression without multiple lines and without local variable resulting as something like:
(int i1, int i2) => (i1 + i2) * (i1 + i2)
So, we only introduce this sample to demo the Block expression features.
The 'cool', new part of the method is:
var blockExpr =
Expression.Block
(
typeof(int),
new ParameterExpression[] { resultExpr },
Expression.Assign(resultExpr,
Expression.Add(i1Expr, i2Expr)),
Expression.MultiplyAssign
(resultExpr, resultExpr),
resultExpr
);
Let us take a closer look at Expression.Block(...) method used above.
The first argument to Block(...) specifies the return type of the block (int in our case).
The second argument is an array of ParameterExpression objects that specify the local variables of the block. In our case there is only one local variable result of type int.
After that, there are expressions corresponding to the lines of code.
Important Note: The last line expression will be returned (unless it is void) - in our case, result variable is returned since resultExpr is the last line of the code.
BlockLoopSumUpNumbersFromToSample
Our next sample is similar to LoopSumUpNumbersFromToSample() non-Block sample explained above. It results in Lambda with from and to integer input arguments which sums up all the intergers from from to to including both end points.
Unlike the non-Block expression:
- It does not require passing the result as an external input argument (the result is defined as a local variable)
- It allows initializing the result and incrementing the loop index on separate lines.
Here is the code:
static void BlockLoopSumUpNumbersFromToSample()
{
var loopIdxExpr = Expression.Parameter(typeof(int), "i");
var toExpr = Expression.Parameter(typeof(int), "to");
var resultExpr = Expression.Parameter(typeof(int), "result");
var loopLabel = Expression.Label(typeof(int), "breakLabel");
var blockExpr =
Expression.Block
(
typeof(int),
new ParameterExpression[] { resultExpr },
Expression.Assign
(resultExpr, Expression.Constant(0)),
Expression.Loop
(
Expression.IfThenElse
(
Expression.LessThanOrEqual(loopIdxExpr, toExpr),
Expression.Block
(
Expression.AddAssign
(resultExpr, loopIdxExpr),
Expression.PostIncrementAssign
(loopIdxExpr)
),
Expression.Break(loopLabel, resultExpr)
),
loopLabel
)
);
var sumNumbersFromTooLambdaExpr = Expression.Lambda<Func<int, int, int>>
(blockExpr, loopIdxExpr, toExpr);
var sumNumbersFromTooLambda = sumNumbersFromTooLambdaExpr.Compile();
int from = 1;
int to = 10;
var sumResult = sumNumbersFromTooLambda(from, to);
Console.WriteLine($"Sum of intergers from {from} to {to} is {sumResult}");
}
The interesting part is:
var blockExpr =
Expression.Block
(
typeof(int),
new ParameterExpression[] { resultExpr },
Expression.Assign(resultExpr, Expression.Constant(0)),
Expression.Loop
(
Expression.IfThenElse
(
Expression.LessThanOrEqual(loopIdxExpr, toExpr),
Expression.Block
(
Expression.AddAssign(resultExpr, loopIdxExpr),
Expression.PostIncrementAssign(loopIdxExpr)
),
Expression.Break(loopLabel, resultExpr)
),
loopLabel
)
);
Expression.Block(...) is used twice. The top level Expression.Block(...) defines resultExpr as a local variable, initializes it by Expression.Assign(resultExpr, Expression.Constant(0)) to zero calls the loop and returns the loop result.
The Block expression within the loop represents the loop body and is used to place i++ increment operator on a separate line after result += i; line. In C#, I prefer it for clarity, even though it will be compiled to the same code.
Running the sample method will result in:
Sum of integers from 1 to 10 is 55
printed to console.
BlockCallPlusRefSample
The last block sample will demonstrate how to call a method with a ref argument using Block expressions.
The method to be called is:
public static void PlusRef(ref int i1, int i2)
{
i1 += i2;
}
The first argument i1 is ref and gets updated with the sum of i1 and i2.
The purpose of this sample is to create a lambda that wraps such method something close to:
(int i1, int i2) =>
{
int result = i1;
PlusRef(ref result, i2);
return result;
}
Note that lambdas cannot have ref and out arguments, so we are forced to pass inputs as simple int arguments and return also int. Variable result has to be passed to PlusRef(...) method as a ref int argument so it has to be a local variable (since it cannot be passed to the lambda as a ref argument).
In order to be able to declare and initialize a local variable, we are forced to use a Block expression.
static void BlockCallPlusRefSample()
{
var i1Expr = Expression.Parameter(typeof(int), "i1");
var i2Expr = Expression.Parameter(typeof(int), "i2");
var resultExpr = Expression.Parameter(typeof(int), "result");
Type plusRefMethodContainer = typeof(Program);
MethodInfo plusRefMethodInfo =
plusRefMethodContainer.GetMethod(nameof(PlusRef))!;
var blockExpr = Expression.Block
(
typeof(int),
new ParameterExpression[] { resultExpr },
Expression.Assign(resultExpr, i1Expr),
Expression.Call(plusRefMethodInfo, resultExpr, i2Expr),
resultExpr
);
var lambdaExpr =
Expression.Lambda<Func<int, int, int>>
(
blockExpr,
i1Expr,
i2Expr
);
var lambda = lambdaExpr.Compile();
int i1 = 1, i2 = 2;
int result = lambda(i1, i2);
Console.WriteLine($"{i1} + {i2} = {result}");
}
Conclusion
In this article, I explained the basics and gave numerous hands-on examples of programming System.Linq.Expressions, went over Non-Block and Block expressions, explaining the concepts of Expressions and going over numerous static methods within Expression class used for building various expressions.
History
- 30th October, 2022: Initial version
