ScriptEngine - User Defined Calculations in C#, VB, JScript and F#

• Download source - 61.75 KB

Introduction

While converting some software to .NET, I came across the need for a user defined calculation or scripting class. In the software I was working on, in places where there are formulas for performing engineering calculations, the user can add his/her own formula rather than just select from built-in methods. In the past, I accomplished this task using a system originally written in assembly and modeled on the Forth language[^]. Over the years, the system was converted to use algebraic notation and ported to Pascal, C, C++, VB and a few other languages. The threaded interpreted language[^] model provided for fast interpretation and execution speed rivaling compiled code, which is important when allowing user defined functions to process millions of data points.

In converting to .NET, however, it seems there is a better way to do things using reflection. Essentially it allows one to embed the standard .NET compilers directly in the program and compile user input "on-the-fly." As a result, rather than once again converting my calculation engine to a new language, I implemented a calculation engine that allows user functions to be written in any available .NET language.

Using the ScriptEngine Class

The project download contains the ScriptEngine C# class, as well as a simple program to allow user input and perform calculations. Note that if you have not installed the F# compiler[^], you may need to comment out all of the references to FSharp in the source code and F# will not be available as a scripting language.

Since the purpose of this engine is to perform engineering calculations, all variables are defined as double, although that could be changed. The important fields and methods are defined as follows:

• public enum Languages { VBasic, CSharp, JScript, FSharp }; - defines the available languages that are recognized
• public ScriptEngine(Languages language) - the constructor which takes a language as a parameter. The default constructor specifies VBasic, since most of my customers are most familiar with Basic syntax.
• public string Code - allows the user program code to be read or defined
• public void AddVariable(string VariableName) - allows variables to be defined
• public bool Compile() - compiles the code and returns true if successful
• public string[] Messages - a string array containing compiler messages
• public void SetVariable(string VariableName, double Value) - allows variable values to be initialized
• public double GetVariable(string VariableName) - retrieves variable values
• public double Evaluate() - runs the script and returns the value of the Result variable

Note that the user code should set a value for the Result variable to determine the value returned by the Evaluate() function. By default:

Result = Double.NaN

The general procedure for using ScriptEngine is to instantiate a new ScriptEngine object, specifying the language to be used, add any variables needed using AddVariable, define the calculation code, and Compile() the code and check for compiler errors. By design, Compile() will return false if there are any compiler errors or warnings and the calling application can access those in the Messages string array and display those to the user. To perform calculations, variable values are initialized using SetVariable, the calculation is performed using Evaluate(), and variable values are retrieved using GetVariable. The Evaluate() function returns a double which might be the only value needed from the calculation.

In my applications, the usual use of ScriptEngine in C# is within a loop that sets the needed variable values, calls the Evaluate() method, and then processes the result as illustrated in the following C# code, with variables X and Y:

ScriptEngine Engine = new ScriptEngine(ScriptEngine.Languages.VBasic);
Engine.Code = code;
Engine.AddVariable("X");
Engine.AddVariable("Y");
if (Engine.Compile())
{
    foreach (ValueType v in Values)
    {
        // Set the variable values for the script
        Engine.SetVariable("X", v.X);
        Engine.SetVariable("Y", v.Y);
        // Evaluate the script and return the Result
        double result = Engine.Evaluate();
        // Retrieve any variables that might have changed
        double x = Engine.GetVariable("X");
        // Do something with the result                        }
    }
}
else
{
    MessageBox.Show("Compiler message: " + Engine.Messages[0]);
}

Inside ScriptEngine

To do its job, the ScriptEngine class uses the .NET CodeDomProvider class to dynamically create an assembly in memory. Since one of the requirements of my applications is to be able to access various variables, the AddVariable method adds a class-level field using the variable name provided, as well as GetVariableName and SetVariableName methods, where VariableName is the name of the variable, for setting and reading the variable's values.

The rest of the generated code defines a namespace UserScript, a class RunScript and a Result field, as well as the Evaluate() method, then embeds the user defined code as the body of the method. Since various languages are supported, the generated code for each language is slightly different.

The C# code that actually compiles and evaluates the generated code is as follows:

public bool Compile()
{
    switch (Language)
    {
        case Languages.CSharp:
            source = "namespace UserScript\r\n{\r\nusing System;\r\n" +
                "public class RunScript\r\n{\r\n" + 
                variables + "\r\npublic double Eval()\r\n{\r\ndouble Result = 
		Double.NaN;\r\n" +
                code + "\r\nreturn Result;\r\n}\r\n}\r\n}";
            compiler = new CSharpCodeProvider();
            break;
        case Languages.JScript:
            source = "package UserScript\r\n{\r\n" + 
                "class RunScript\r\n{\r\n" + 
                variables + "\r\npublic function Eval() : 
	       String\r\n{\r\nvar Result;\r\n" +
                code + "\r\nreturn Result; \r\n}\r\n}\r\n}\r\n";
            compiler = new JScriptCodeProvider();
            break;
        case Languages.FSharp:
            source = "#light\r\nmodule UserScript\r\nopen System\r\n" +
                "type RunScript() =\r\n" +
                "    let mutable Result = Double.NaN\r\n" +
                variables + "\r\n" + variables1 +
                "    member this.Eval() =\r\n" +
                code + "\r\n        Result\r\n";
            compiler = new FSharpCodeProvider();
            break;
        default: // VBasic
            source = "Imports System\r\nNamespace 
	       UserScript\r\nPublic Class RunScript\r\n" +
                variables + "Public Function Eval() 
	       As Double\r\nDim Result As Double\r\n" +
                code + "\r\nReturn Result\r\nEnd Function\r\nEnd Class\
					r\nEnd Namespace\r\n";
            compiler = new VBCodeProvider();
            break;
    }
    parameters = new CompilerParameters();
    parameters.GenerateInMemory = true;
    results = compiler.CompileAssemblyFromSource(parameters, source);
    // Check for compile errors / warnings
    if (results.Errors.HasErrors || results.Errors.HasWarnings)
    {
        Messages = new string[results.Errors.Count];
        for (int i = 0; i < results.Errors.Count; i++)
            Messages[i] = results.Errors[i].ToString();
        return false;
    }
    else
    {
        Messages = null;
        assembly = results.CompiledAssembly;
        if (Language == Languages.FSharp)
            evaluatorType = assembly.GetType("UserScript+RunScript");
        else
            evaluatorType = assembly.GetType("UserScript.RunScript");
        evaluator = Activator.CreateInstance(evaluatorType);
        return true;
    }
}

public double Evaluate()
{
    object o = evaluatorType.InvokeMember(
               "Eval",
               BindingFlags.InvokeMethod,
               null,
               evaluator,
               new object[] { }
               );
    string s = o.ToString();
    return double.Parse(s.ToString());
}

As can be seen, a switch statement controls which source code is generated depending on the chosen language. Most of the code is similar, however the F# code needs to be handled slightly differently due to differences in the syntax and the .NET implementation.

Specifically, in F# the indentation is important when using the #light syntax, so special attention is paid to spacing at the beginning of lines. The code in the main form that reads the text specifically adds spaces when using F#. In addition, for some reason, the type generated by the F# compiler is "UserScript+RunScript" with a +, whereas the other languages generate the type "UserScript.RunScript" with a period (.). I'm not sure if that's an error in the F# compiler or whether it's by design, but it did cause quite a bit of frustration tracking it down!

For illustration, the generated code to return a vector magnitude given X and Y using the formula Result = Sqrt(X*X + Y*Y) in the various languages is as follows, with indentation added for readability:

In C#:

namespace UserScript
{
using System;
  public class RunScript
  {
    double X = 0;
    public void SetX(double x) { X = x; }
    public double GetX() { return X; }
    double Y = 0;
    public void SetY(double x) { Y = x; }
    public double GetY() { return Y; }

    public double Eval()
    {
      double Result = Double.NaN;
      Result = Math.Sqrt(X*X + Y*Y);  // This is the user code
      return Result;
    }
  }
}

In JScript:

package UserScript
{
  class RunScript
  {
    var X : double;
    public function SetX(x) { X = x; }
    public function GetX() : String { return X; }
    var Y : double;
    public function SetY(x) { Y = x; }
    public function GetY() : String { return Y; }

    public function Eval() : String
    {
      var Result;
      Result = Math.sqrt(X*X + Y*Y);  // This is the user code
      return Result; 
    }
  }
}

In Visual Basic:

Imports System
Namespace UserScript
  Public Class RunScript
    Dim X As Double
    Public Sub SetX(AVal As Double)
      X = AVal
    End Sub
    Public Function GetX As Double
      Return X
    End Function
    Dim Y As Double
    Public Sub SetY(AVal As Double)
      Y = AVal
    End Sub
    Public Function GetY As Double
      Return Y
    End Function
    Public Function Eval() As Double
      Dim Result As Double
      Result = (X*X + Y*Y)^0.5    ' This is the user code
      Return Result
    End Function
  End Class
End Namespace

In F#:

#light
module UserScript
open System
type RunScript() =
    let mutable Result = Double.NaN
    let mutable X = 0.0
    let mutable Y = 0.0

    member x.GetX = X
    member x.SetX v = X <- v
    member x.GetY = Y
    member x.SetY v = Y <- v
    member this.Eval() =
        Result <- Math.Sqrt(X*X + Y*Y)  // This is the user code
        Result

Purging the AppDomain

Per an excellent observation by Uwe Keim, I added a static AppDomain named "ScriptEngine" that is initialized when the ScriptEngine is created. All instances of ScriptEngine are placed in this AppDomain, so that when all calculations are completed, calling the Unload() method will unload the dynamic assembly from memory.

In most of my applications this does not seem to be a problem, but Uwe is correct in pointing out that using ScriptEngine with long running apps with multiple ScriptEngine instances would pollute the standard AppDomain with no means to unload the assemblies. Calling the Unload() method when the ScriptEngine is no longer needed easily circumvents that potential problem.

In addition, to avoid conflict with multiple instances of ScriptEngine, each subsequent class is named "RunScriptN," where N is an integer that increments every time an instance of ScriptEngine is created. This allows several ScriptEngine instances to run different code without conflict.

Conclusions

For the purpose of implementing user defined engineering calculations, ScriptEngine seems to work great. Besides allowing normal calculations, the use of standard .NET languages allows all of the features of the languages to be used for more complex iterations and other calculations. Essentially anything that can be placed in a function or method body can be used in the user defined code.

As far as performance, it's hard to tell the difference in execution between the user defined code and regular compiled code, other than a slight lag while compiling the code, since the user defined code is actually compiled. I didn't make a concerted effort to find out why, but it does seem that the F# code executes somewhat slower than the other languages. That may be due to the preliminary nature of the current F# compiler or due to the extra overhead involved in mapping F# syntax to .NET. Time will tell if future releases of F# perform better.

In addition, using the methods illustrated here, it is fairly easy to produce other special purpose code generated at run time. Although I'm not generally a fan of dynamic code due to the probability of introducing near impossible debugging problems, there certainly are cases where run time code generation is needed. I hope that by using ScriptEngine as an example, perhaps some headaches can be avoided for others who implement dynamic run time code.

And for me, a major advantage is that my Users Manuals can be simplified significantly, since I don't have to document all of the scripting language. Instead I can simply explain the use of the variables, that the result is returned in the Result variable, and refer the user to all of the language documentation available from Microsoft or on the internet. That certainly saves significant time and effort!

And as always, I can't claim the code presented here is optimal. If anyone has any suggestions or observations, I'd be pleased to hear about them.

History

• 15^th November, 2008 - Initial submission
• 17^th November, 2008 - Modified download to include Properties folder and update source code implementing a separate AppDomain .