Entity Framework Dynamic Include Hierarchy

Introduction

Using system reflection to dynamically build EntityFrameWork Include like :

"x=> x.Connectors , x=> x.BlockEntryFieldsGroup.Select
(GhIm=> GhIm.BlockEntry_Field.Select(GhImcc=> GhImcc.BlockEntry_Value)) , 
x=> x.BlockEntryOffSet"

Background

When I got an assignment on building a generic Assembly for Entity Framework, I encountered a problem.

My class had many other objects. So I sometimes wanted to be able to include all other objects that are for the current object.

For example:

had User which it has Person, Role, and Address, Address had Country and so on.

Building an include for this is a pain in the ass.

Entity Framework provides something called lazy Loading. What it means is that it will always load all the trees at all-time which is really bad for performance.

So I built a tool that does just that when I wanted it.

Using the Code

Let’s build an example class here. Let’s call it Role:

public class Role{
public long Id { get; set;}
public string Name { get; set; }
public virtual List<User> Users { get; set; } // users in the Role
}
public class User {
public long Id { get; set; }
public string UserName { get; set; }
public virtual Person { get; set; } // reference to person
}
public class Person {
public long Id { get; set; }
public string PersonName { get; set; }
public virtual Address { get; set; } // reference to Address
}   
public class Address {
public long Id { get; set; }
public string AdressName { get; set; }
public virtual Country { get; set; } // reference to Country
}   
public class Country {
public long Id { get; set; }
public string Name { get; set; }
}   

Normally, you call this in entity framework like this:

dbContext.Role.Where(x=> x.Name == "Admin").Include
(x=> x.Users.Select(a=> a.Person.Address.Country ));

Imagine if you had a bigger object then this, it would have been a pain to do this each time.

Ok now imagine if you could do this with very simple quarry instead like this for example:

dbContext.Role.Where(x=> x.Name == "Admin").IncludeHierarchy();

Well, you got it. The whole point of this tip is to teach you how to do it.

So let’s begin building our dynamic include library.

public class DynamicLinqInclude {
private static Dictionary<string, MethodInfo> 
DynamicLinqIncludeSources = new Dictionary<string, MethodInfo>(); // for faster browsing
private static int DynamicLinqIncludeSourceLength = 0; // for faster Count
private List<string> _generatedKeys = new List<string>();
private const string valid = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";

// Create a random key like x, xcd and so on
private string RandomKey(int length = 2)
{
   var result = "";
   Random rnd = new Random();
   while (0 < length--)
   {
      result += valid[rnd.Next(valid.Length)];
   }
   if (_generatedKeys.Any(x => x == result))
   result = RandomKey(_generatedKeys.Last().Length + 1);
   else _generatedKeys.Add(result);
   return result;
}

// now ower method that build the quary by System.Type
//it should build a string that look like this.
"x=> x.Users.Select(a=> x.Person.Address.Country)"

private List<string> DynamicIncludeExpressionString
(Type entity, string parentName = "", bool isFirst = true)
{
            List<string> select = new List<string>();

            foreach (var prop in entity.GetProperties().Where
            (x => x.PropertyType.IsClass && x.PropertyType != typeof(string) && 
            x.GetCustomAttribute<IgnoreDataMemberAttribute>() == null))
            {
                var key = isFirst ? parentName : RandomKey();
                var subKey = RandomKey();
                if (prop.PropertyType.IsGenericList())
                {
                    var inclObjects = DynamicIncludeExpressionString
                    (prop.PropertyType.GetProperties().Last().PropertyType, "", false);
                    var incKey = string.Format("{0}.Select({1}=> {1}.{2}", parentName, key, prop.Name);
                    if (isFirst)
                    {

                        incKey = string.Format("{0}.{1}", key, prop.Name);

                    }
                    if (inclObjects.Any())
                    {
                        foreach (var txt in inclObjects)
                        {
                            if (isFirst)
                            {
                                bool startWithSelect = txt.StartsWith(".Select");
                                var res = incKey;
                                if (!startWithSelect)
                                    res = string.Format("{0}.{1}.Select({2}=> {3}", 
                                    key, prop.Name, subKey, subKey);
                                res += "." + txt.TrimStart('.');
                                res = res.TrimEnd('.');
                                if (!startWithSelect)
                                    res += ")";
                                select.Add(res);
                            }
                            else
                            {
                                var res = txt.TrimStart('.');
                                bool startWithSelect = txt.StartsWith(".Select");
                                if (!startWithSelect)
                                {
                                    incKey = incKey.TrimEnd('.');
                                    incKey += string.Format(".Select({0}=> {1}", 
                                    subKey, subKey) + "." + res.TrimStart('.') +")";
                                }
                                else
                                    incKey += "." + res.TrimStart('.');
                            }
                        }
                    }
                    if (!isFirst)
                    {
                        incKey = incKey.TrimEnd('.');
                        if (incKey.Contains("("))
                            incKey += ")";
                        select.Add(incKey);
                    }

                    if (isFirst && !inclObjects.Any())
                        select.Add(incKey);
                }
                else
                {
                    key = parentName;
                    var incKey = string.Format("{0}.{1}", key, prop.Name);
                    var inclObjects = DynamicIncludeExpressionString(prop.PropertyType, "", false);
                    if (inclObjects.Any())
                    {
                        foreach (var txt in inclObjects)
                        {
                            if (isFirst)
                            {
                                var res = incKey;
                                res += "." + txt.TrimStart('.');
                                res = res.TrimEnd('.');
                                select.Add(res);
                            }
                            else
                            {
                                incKey += "." + txt.TrimStart('.');
                            }
                        }
                    }
                    if (!isFirst)
                    {
                        incKey = incKey.TrimEnd('.');
                        if (incKey.Contains("("))
                            incKey += ")";
                        select.Add(incKey);
                    }

                    if (isFirst && !inclObjects.Any())
                        select.Add(incKey);
                }
            }

            return select;
}

//now we need to handle this quary and convert the string to expression.
// for that i used CSharpCodeProvider to do the job for me.
// CSharpCodeProvider is library that convert string to C# Code.

 public IQueryable<T> Parse<T>(IQueryable<T> source)
{
            var type = typeof(T);
            var key = type.FullName;
            // load it only once ans save it to a static variable.
            if (!DynamicLinqIncludeSources.ContainsKey(key)) 
            {
                var searchValue = DynamicIncludeExpressionString(typeof(T), "x");
                if (!searchValue.Any())
                    return source;
                // Build a class and a MethodInfo called Execute
                var func = "x=> " + string.Join(" , x=> ", searchValue);
                var builder = new StringBuilder();
                builder.Append("\r\nusing System;");
                builder.Append("\r\nusing System.Collections.Generic;");
                builder.Append("\r\nusing System.Linq;");
                builder.Append("\r\nusing System.Reflection;");
                builder.Append("\r\nusing Test.Core.Interface;");
                builder.Append("\r\nusing Test.Core.ObjectLibrary;");
                builder.Append("\r\nusing Test.Core.EntityFramwork_Extension;");
                builder.Append("\r\nnamespace DynamicLinqInclude");
                builder.Append("\r\n{");
                builder.Append("\r\npublic sealed class Dynamic_LinqInclude");
                builder.Append("\r\n{");
                builder.Append("\r\npublic static IQueryable<" + 
                type.FullName + "> Execute<T>(IQueryable<" + type.FullName + "> source)");
                builder.Append("\r\n{");
                builder.Append("\r\n return source.IncludeEntities(" + func + ");");
                builder.Append("\r\n}");
                builder.Append("\r\n}");
                builder.Append("\r\n}");
                var codeProvider = new CSharpCodeProvider();
                var compilerParameters = new CompilerParameters
                {
                    GenerateExecutable = false,
                    GenerateInMemory = false,
                    CompilerOptions = "/optimize"
                };
                compilerParameters.ReferencedAssemblies.Add("System.dll");
                compilerParameters.ReferencedAssemblies.Add
                           (typeof(System.Linq.IQueryable).Assembly.Location);
                compilerParameters.ReferencedAssemblies.Add
                           (typeof(System.Collections.IList).Assembly.Location);
                compilerParameters.ReferencedAssemblies.Add
                           (typeof(System.Linq.Enumerable).Assembly.Location);
                compilerParameters.ReferencedAssemblies.Add(typeof(T).Assembly.Location);
                string sourceCode = builder.ToString();
                CompilerResults compilerResults = 
                codeProvider.CompileAssemblyFromSource(compilerParameters, sourceCode);
                Assembly assembly = compilerResults.CompiledAssembly;
                Type types = assembly.GetType("DynamicLinqInclude.Dynamic_LinqInclude");
                MethodInfo methodInfo = types.GetMethod("Execute");
                methodInfo = methodInfo.MakeGenericMethod(type);
                if (!DynamicLinqIncludeSources.ContainsKey(key)) // it may have been added 
                                                                 // by another thread
                {
                    DynamicLinqIncludeSources.Add(key, methodInfo); // Save it in a temp 
                                  //instead of memory so we could choose to Clear it later.
                    DynamicLinqIncludeSourceLength++;
                }
            }
            if (DynamicLinqIncludeSources.ContainsKey(key)) // it may been deleted 
                                    //by another threads then force a control agen.
                return (IQueryable<T>)DynamicLinqIncludeSources[key].Invoke
                                             (null, new object[] { source });
            else return Parse<T>(source);
}
}

This will build the dynamic include and parse it to C# code.

So let’s build and see how to use it.

Let’s build a class that is called EntityFramework_Extension.

In this class, we need two methods, one that can handle the Params Include expressions and another that could build the dynamic include.

public static class EntityFramwork_Extension { 

/// <summary>
/// Choose the properties to include. this method will be used by the method that we dynamically create
/// in the previous class 
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="source"></param>
/// <param name="includes"></param>
/// <returns></returns>
public static IQueryable<T> IncludeEntities<T>(this IQueryable<T> source, 
params Expression<Func<T, object>>[] includes) where T : class, new()
{
  return includes.Aggregate(source, (current, include) => current.Include(include));
}

/// <summary>
/// this will include all level(Hierarchy)
/// really slow first time but expression is saved in memory 
/// so it will load much faster the secound time.
/// be caution of circle references exception, Add IgnoreDataMember Attributes 
/// so the plug in will recognize them(they will still be loaded)
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="source"></param>
/// <returns></returns>
public static IQueryable<T> IncludeHierarchy<T>(this IQueryable<T> source)
{
   return new DynamicLinqInclude().Parse(source);
}
}

And now, all we need is to call our Role object and see the magic happen.

dbContext.Role.Where(x=> x.Name== "Admin").IncludeHierarchy().First();

Hope you liked this. Please comment below about what you think.