Using Roslyn for Compiling Code into Separate Net Modules and Assembling them into a Dynamic Library

• Download source code - 200.2 KB
• Code on GITHUB

Introduction

MS Roslyn is a great tool, though pretty new and not well documented with few examples available on the internet. It is to fill this documentation/sample gap that I decided to write this article.

Over the past year, I worked on several projects that involved dynamic code generation, compilation and creating dynamic assemblies using MS Roslyn compiler as a service platform. To create a dynamic assembly, I would compile each of the individual components separately into Net Modules and then combine them into the assembly (in-memory DLL). This approach was adopted in order to avoid the costly recompilation of components that have not been modified.

Having recently spent several hours trying to answer a github question on why loading a module into a dynamic DLL produces an error github question on why trying to load a module produces an error, I decided to publish the solution on the CodeProject so that others did not have to go through the same pain in the future.

The Code

The RoslynAssembly solution was created as a simple "Console App" project using VS 2017. Then I added a single NuGet package Microsoft.CodeAnalysis.CSharp to it. Since there is a dependency on a NuGet package - it will show that it misses some references in your Visual Studio, but once you compile it (provided you have a working internet connection), the NuGet packages should be downloaded and installed and all the references should be filled.

The code consists of only one class Program within single file Program.cs.

Here is the code for the sample:

public static class Program
{
    public static void Main()
    {
        try
        {
            // code for class A
            var classAString = 
                @"public class A 
                    {
                        public static string Print() 
                        { 
                            return ""Hello "";
                        }
                    }";

            // code for class B (to spice it up, it is a 
            // subclass of A even though it is almost not needed
            // for the demonstration)
            var classBString = 
                @"public class B : A
                    {
                        public static string Print()
                        { 
                            return ""World!"";
                        }
                    }";

            // the main class Program contain static void Main() 
            // that calls A.Print() and B.Print() methods
            var mainProgramString = 
                @"public class Program
                    {
                        public static void Main()
                        {
                            System.Console.Write(A.Print()); 
                            System.Console.WriteLine(B.Print());
                        }
                    }";

            #region class A compilation into A.netmodule
            // create Roslyn compilation for class A
            var compilationA = 
                CreateCompilationWithMscorlib
                (
                    "A", 
                    classAString, 
                    compilerOptions: new CSharpCompilationOptions(OutputKind.NetModule)
                );

            // emit the compilation result to a byte array 
            // corresponding to A.netmodule byte code
            byte[] compilationAResult = compilationA.EmitToArray();

            // create a reference to A.netmodule
            MetadataReference referenceA = 
                ModuleMetadata
                    .CreateFromImage(compilationAResult)
                    .GetReference(display: "A.netmodule");
            #endregion class A compilation into A.netmodule

            #region class B compilation into B.netmodule
            // create Roslyn compilation for class A
            var compilationB = 
                CreateCompilationWithMscorlib
                (
                    "B", 
                    classBString, 
                    compilerOptions: new CSharpCompilationOptions(OutputKind.NetModule), 

                    // since class B extends A, we need to 
                    // add a reference to A.netmodule
                    references: new[] { referenceA }
                );

            // emit the compilation result to a byte array 
            // corresponding to B.netmodule byte code
            byte[] compilationBResult = compilationB.EmitToArray();

            // create a reference to B.netmodule
            MetadataReference referenceB =
                ModuleMetadata
                    .CreateFromImage(compilationBResult)
                    .GetReference(display: "B.netmodule");
            #endregion class B compilation into B.netmodule

            #region main program compilation into the assembly
            // create the Roslyn compilation for the main program with
            // ConsoleApplication compilation options
            // adding references to A.netmodule and B.netmodule
            var mainCompilation =
                CreateCompilationWithMscorlib
                (
                    "program", 
                    mainProgramString, 
                    compilerOptions: new CSharpCompilationOptions
                                     (OutputKind.ConsoleApplication), 
                    references: new[] { referenceA, referenceB }
                );

            // Emit the byte result of the compilation
            byte[] result = mainCompilation.EmitToArray();

            // Load the resulting assembly into the domain. 
            Assembly assembly = Assembly.Load(result);
            #endregion main program compilation into the assembly

            // load the A.netmodule and B.netmodule into the assembly.
            assembly.LoadModule("A.netmodule", compilationAResult);
            assembly.LoadModule("B.netmodule", compilationBResult);

            #region Test the program
            // here we get the Program type and 
            // call its static method Main()
            // to test the program. 
            // It should write "Hello world!"
            // to the console

            // get the type Program from the assembly
            Type programType = assembly.GetType("Program");

            // Get the static Main() method info from the type
            MethodInfo method = programType.GetMethod("Main");

            // invoke Program.Main() static method
            method.Invoke(null, null);
            #endregion Test the program
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex.ToString());
        }
    }

    // a utility method that creates Roslyn compilation
    // for the passed code. 
    // The compilation references the collection of 
    // passed "references" arguments plus
    // the mscore library (which is required for the basic
    // functionality).
    private static CSharpCompilation CreateCompilationWithMscorlib
    (
        string assemblyOrModuleName,
        string code,
        CSharpCompilationOptions compilerOptions = null,
        IEnumerable<MetadataReference> references = null)
    {
        // create the syntax tree
        SyntaxTree syntaxTree = SyntaxFactory.ParseSyntaxTree(code, null, "");

        // get the reference to mscore library
        MetadataReference mscoreLibReference = 
            AssemblyMetadata
                .CreateFromFile(typeof(string).Assembly.Location)
                .GetReference();

        // create the allReferences collection consisting of 
        // mscore reference and all the references passed to the method
        IEnumerable<MetadataReference> allReferences = 
            new MetadataReference[] { mscoreLibReference };
        if (references != null)
        {
            allReferences = allReferences.Concat(references);
        }

        // create and return the compilation
        CSharpCompilation compilation = CSharpCompilation.Create
        (
            assemblyOrModuleName,
            new[] { syntaxTree },
            options: compilerOptions,
            references: allReferences
        );

        return compilation;
    }

    // emit the compilation result into a byte array.
    // throw an exception with corresponding message
    // if there are errors
    private static byte[] EmitToArray
    (
        this Compilation compilation
    )
    {
        using (var stream = new MemoryStream())
        {
            // emit result into a stream
            var emitResult = compilation.Emit(stream);

            if (!emitResult.Success)
            {
                // if not successful, throw an exception
                Diagnostic firstError =
                    emitResult
                        .Diagnostics
                        .FirstOrDefault
                        (
                            diagnostic =>
                                diagnostic.Severity == DiagnosticSeverity.Error
                        );

                throw new Exception(firstError?.GetMessage());
            }

            // get the byte array from a stream
            return stream.ToArray();
        }
    }
}

Code Description

Main Program Explained

The code demonstrates how to compile and assemble three classes A, B and Program. A and B are compiled into net modules. Class Program is compiled into the runnable assembly. We load the A.netmodule and B.netmodule into the main assembly and then test it by running a static method Program.Main() which calls static methods both classes A and B.

Here is the code for A class:

// code for class A
var classAString = 
    @"public class A 
        {
            public static string Print() 
            { 
                return ""Hello "";
            }
        }";  

Its static method A.Print() prints "Hello " string.

Here is B class code:

var classBString = 
    @"public class B : A
        {
            public static string Print()
            { 
                return ""World!"";
            }
        }";

Its static method B.Print() prints string "World!" Note that in order to spice it up, I made class B extend class A. This will require passing a reference to A during B compilation (as shown below).

Here is the main Program class:

var mainProgramString = 
    @"public class Program
        {
            public static void Main()
            {
                System.Console.Write(A.Print()); 
                System.Console.WriteLine(B.Print());
            }
        }";

Here is how we create a A.netmodule and a reference to it.

1. Create Roslyn Compilation object for A.netmodule:

   C#

   var compilationA = 
       CreateCompilationWithMscorlib
       (
           "A", 
           classAString, 
           compilerOptions: new CSharpCompilationOptions(OutputKind.NetModule)
       );      

   Method CreateCompilationWithMscorlib is a utility method that creates the compilation and it will be discussed below.

2. Emit the compilation to a byte array:

   C#

   byte[] compilationAResult = compilationA.EmitToArray();

   This array is a binary representation of the module code. EmitToArray is another utility function that will be discussed below in detail.

3. Create a reference to A.netmodule to be used for creating B.netmodule (since class B depends on class A) and also for creating the main program code (since it also depends on A).

   C#

   MetadataReference referenceA = 
       ModuleMetadata
           .CreateFromImage(compilationAResult)
           .GetReference(display: "A.netmodule");

Very Important Note: Every time the compilation results are emitted (in our case, it happens within EmitToArray() method), the resulting byte code changes slightly, probably because of a time stamp. Because of that, it is important to have the reference and Module code produced from the same Emit result. Otherwise, if you have different Emit(...)s for the module code and for the reference, trying to load the module into the assembly will result in a hash mismatch exception, since the hash or the reference used to build the assembly and the hash of the module code will be different. This is what took me several hours to figure out and this is the primary reason I am writing this article.

Creating the B.netmodule and its reference is almost the same as that of A.netmodule, with the exception that we need to pass a reference to A.netmodule to the method CreateCompilationWithMscorlib(...) (since class B depends on class A).

Here is how we create the main assembly:

1. Create the Roslyn Compilation object for the main assembly:

   C#

   var mainCompilation =
       CreateCompilationWithMscorlib
       (
           "program", 
           mainProgramString, 
           // note that here we pass the OutputKind set to ConsoleApplication
           compilerOptions: new CSharpCompilationOptions(OutputKind.ConsoleApplication), 
           references: new[] { referenceA, referenceB }
       );  
   note that we pass <code>OutputKind.ConsoleApplication</code> option since it is an 
   assembly and not a net module. 

2. Emit the compilation result into a byte array:

   C#

   byte[] result = mainCompilation.EmitToArray();  

3. Load the assembly into the domain:

   C#

   Assembly assembly = Assembly.Load(result));  

4. Load the two modules into the assembly:

   C#

   assembly.LoadModule("A.netmodule", compilationAResult);
   assembly.LoadModule("B.netmodule", compilationBResult);  

   Note: It is at this stage that an exception will be thrown if the hashes of the reference and module code mismatch.

Finally, here is the code that tests the functionality of the assembly with the Net Modules:

1. Get the C# type Program from the assembly:

   C#

   Type programType = assembly.GetType("Program");      

2. Get the MethodInfo for the static method Program.Main() from the type:

   C#

   MethodInfo method = programType.GetMethod("Main");  

3. Invoke static method Program.Main():

   C#

   method.Invoke(null, null); 

The outcome of this program should be "Hello World!" printed on the console.

Utility Methods

There are two simple static utility methods:

• CreateCompilationWithMscorelib(...) - creates roslyn Compilation
• EmitToArray(...) - emits a byte array representing the .NET code of the compilation.

CreateCompilationWithMscorelib(...) Method

The purpose of the method is to create a Roslyn Compilation object adding to it the reference to mscore library containing the basic .NET functionality. On top of that, it can also add references to the modules passed as its last argument 'references' (if needed).

The method takes the following arguments:

1. string assemblyOrModuleName - name of the resulting assembly or the module
2. string code - a string containing the code to compile
3. CSharpCompilationOptions compilerOptions - should contain new CSharpCompilationOptions(OutputKind.NetModule) for modules or new CSharpCompilationOptions(OutputKind.ConsoleApplication) for the applications
4. IEnumerable<MetadataReference> references - the extra references to be added after the reference to mscore library

First, it parses the code into the syntax tree (Roslyn syntax tree converts the string code objects reflecting C# syntax in preparation for the compilation):

SyntaxTree syntaxTree = SyntaxFactory.ParseSyntaxTree(code, null, "");     

We build the allReferences collection by concatenation the reference to mscore library and the references passed to the method:

// get the reference to mscore library
MetadataReference mscoreLibReference = 
    AssemblyMetadata
        .CreateFromFile(typeof(string).Assembly.Location)
        .GetReference();

// create the allReferences collection consisting of 
// mscore reference and all the references passed to the method
IEnumerable allReferences = 
    new MetadataReference[] { mscoreLibReference };
if (references != null)
{
    allReferences = allReferences.Concat(references);
}  

Finally, we build and return the Roslyn Compilation object via CSharpCompilation.Create(...) method:

// create and return the compilation
CSharpCompilation compilation = CSharpCompilation.Create
(
    assemblyOrModuleName,
    new[] { syntaxTree },
    options: compilerOptions,
    references: allReferences
);

return compilation; 

EmitToArray(...) Method

The purpose of EmitToArray(...) method is to emit the byte code (the real compilation actually occurs at this stage), to check for errors (and throw an exception if emission is not successful) and to return the byte array of .NET code.

It takes only one argument - "compilation" of Roslyn Compilation type.

First, we create the MemoryStream to accommodate the byte array. Then, we emit the compilation result into the stream:

using (var stream = new MemoryStream())
{
    // emit result into a stream
    var emitResult = compilation.Emit(stream);  

Then, we test the compilation result for errors and throw an exception containing the first error message (if errors are found):

if (!emitResult.Success)
{
    // if not successful, throw an exception
    Diagnostic firstError =
        emitResult
            .Diagnostics
            .FirstOrDefault
            (
                diagnostic => 
                    diagnostic.Severity == DiagnosticSeverity.Error
            );

    throw new Exception(firstError?.GetMessage());
} 

Finally (if there are no errors), we return the byte array from the stream:

return stream.ToArray();  

Summary

Roslyn is a very powerful and underused framework whose full power is not quite realized by most of the companies due to the lack of documentation and samples.

In this article, I explain how to compile and assemble dynamically generated code into an executable dynamic assembly at run time using Roslyn.

I try to explain each stage of compilation and assembly also mentioning the possible traps in detail so that the readers of this article will not have to spend as much time on trying to make things work as I did:).

History

• 13^th November, 2017: Added link to GITHUB