Introduction
In this article, I describe how to extend an application functionality with F# scripting. Sometimes, there is a need to extend existing application functionality with some custom behavior. If you don't want or can't go through application developing and building cycle every time you need an additional feature, scripting can be a good solution. It allows to tailor your application for particular needs with minimal efforts.
Since F# is a fully supported language in .NET, it is easy and natural to incorporate F# scripting in .NET applications. I will show you how to add F# scripting to Windows Forms application.
The accompanying sample application has two major parts:
- Host program written in C#. This simple Windows Forms application is built in
FSCompileTestForm project. - Embedded extension written in F# and designed as a class library. It is built in
FSExecutor project.
The whole application is built against .NET Framework 4.0. Used F# compiler is taken from F# PowerPack package.
Example of running simple script with complex numbers arithmetic is shown in the following screenshot:
Preparations
Embedded script needs to exchange data with host application. It can be achieved by using common shared components or by data streams. There are standard input/output/error streams. It would be convenient to leverage them in scripting program using simple printfn functions. In the sample program, standard output and error streams are redirected to the bottom text area. This functionality is implemented in StdStreamRedirector module. This module has static method Init which accepts two actions and returns Redirector object. Redirector object implements IDisposable interface. When it is disposed, native stream handles are returned to its original state.
let Init(stdOutAction: Action<string>, stdErrAction: Action<string>) =
new Redirector(stdOutAction, stdErrAction)
Provided actions are used to handle data coming from standard output and standard error streams correspondingly. These actions can do anything with the provided data. It depends on their implementations. I chose to put incoming data into text area using black color for stdout data and red color for stderr data.
Standard streams in .NET applications work as if they are initialized during the first use and stay in this initialized state all the time. So standard stream initialization should be done as soon as possible before any use of Console.Out or Console.Error streams or any printf/eprintf function calls in script. I put this code into main form's Load event callback.
private void MainForm_Load(object sender, EventArgs e)
{
Action<string> aOut = (text => AppendStdStreamText(text, Color.Black));
Action<string> aErr = (text => AppendStdStreamText(text, Color.DarkRed));
m_r = StdStreamRedirector.Init(aOut, aErr);
}
Redirector object m_r should be disposed after .NET standard streams are finished to use. So its disposal is placed into main form's Dispose method:
protected override void Dispose(bool disposing)
{
if (disposing && (components != null))
{
((System.IDisposable)m_r).Dispose();
m_r = null;
components.Dispose();
}
base.Dispose(disposing);
}
Redirector object are implemented in F# and F# always implements interfaces explicitly, so to be able to call Dispose() method of Redirector object, it must be explicitly cast to IDisposable interface.
Internally Redirector object uses AnonymousPipeServerStream objects connected to standard streams. For this purpose, SetStdHandle and GetStdHandle functions from Kernel32.dll are used.
module private Win32Interop =
[<DllImport("Kernel32.dll")>]
extern [<marshalas(unmanagedtype.bool)>]
bool SetStdHandle(UInt32 nStdHandle, IntPtr hHandle)
[<DllImport("Kernel32.dll")>]
extern IntPtr GetStdHandle(UInt32 nStdHandle)
type Redirector(out: Action<string>, err: Action<string>) =
let stdOutHandleId = uint32(-11)
let stdErrHandleId = uint32(-12)
let pipeServerOut = new AnonymousPipeServerStream(PipeDirection.Out)
let pipeServerErr = new AnonymousPipeServerStream(PipeDirection.Out)
do if not(Win32Interop.SetStdHandle
(stdOutHandleId, pipeServerOut.SafePipeHandle.DangerousGetHandle()))
then failwith "Cannot set handle for stdout."
do if not(Win32Interop.SetStdHandle
(stdErrHandleId, pipeServerErr.SafePipeHandle.DangerousGetHandle()))
then failwith "Cannot set handle for stderr."
do if not(ThreadPool.QueueUserWorkItem
(fun o -> readPipe(pipeServerOut.ClientSafePipeHandle, out)))
then failwith "Cannot run listner thread."
do if not(ThreadPool.QueueUserWorkItem
(fun o -> readPipe(pipeServerErr.ClientSafePipeHandle, err)))
then failwith "Cannot run listner thread."
Two independent threads from application thread pool read data from pipes connected to standard streams and send data to provided actions. For simplicity, all exceptions are ignored inside these threads.
let private readPipe (h: SafePipeHandle, a: Action<string>) =
use clientPipeStream = new AnonymousPipeClientStream(PipeDirection.In, h)
use reader = new StreamReader(clientPipeStream)
try
while not(reader.EndOfStream) do
let s = reader.ReadLine()
a.Invoke(s)
with
| ex -> ()
This implementation makes output and error streams totally independent and, taking into account latency of thread pool threads, it is possible that output of printf/eprintf can be unordered. Data from thread pool threads are coming into Windows Forms text area which belongs to GUI thread. But actions called in thread pool cannot directly access GUI components, they must do it through Contorl.Invoke method in GUI thread. First, better to check if the Invoke call is actually required by checking InvokeRequired property. And if it is required, then the appending colored text is called using anonymous delegate through Invoke method. Invoke method and InvokeRequired property are thread safe, so it is OK to call them from thread pool threads. No additional synchronization is required.
private void AppendStdStreamText(string text, Color c)
{
if(this.InvokeRequired)
{
MethodInvoker del = delegate
{
AppendStdStreamText(text, c);
};
this.Invoke(del);
return;
}
AppendColoredText(text, c);
m_outputTB.AppendText("\n");
}
private void AppendColoredText(string text, Color c)
{
int l1 = m_outputTB.TextLength;
m_outputTB.AppendText(text);
int l2 = m_outputTB.TextLength;
m_outputTB.SelectionStart = l1;
m_outputTB.SelectionLength = l2 - l1;
m_outputTB.SelectionColor = c;
}
Method AppendColoredText adds provided text to the control and sets required color to it.
Here is the screenshot for different stream colored text:
Script Compilation and Execution
Now, when all preparations are finished, it's time to compile script text into executable assembly. For this purpose, F# compiler from F# PowerPack package is used. All compilation and execution functionality is implemented in FSExecutor module. For compilation, script code in a single string and list of referenced assemblies in seq<string>(F#) which corresponds to IEnumerable<string>(C#) are provided. Referenced assemblies can be set using absolute assembly paths or just DLL names for GAC registered assemblies.
First F# compiler object must be created. Then different compilation parameters are set to the CompilerParameters object. Sequence of referenced assemblies is also set to the compiler parameters.
Since script is usually not a big program, the resulting assembly is generated in memory to accelerate the whole process, so GenerateInMemory property is set to true. Script program is not a class library and should have single EntryPoint property of MethodInfo type, so GenerateExecutable property should be set to true as well.
let compile (code: string) references =
let compiler = new FSharpCodeProvider()
let cp = new System.CodeDom.Compiler.CompilerParameters()
for r in references do cp.ReferencedAssemblies.Add(r) |> ignore done
cp.GenerateInMemory <- true
cp.GenerateExecutable <- true
let cr = compiler.CompileAssemblyFromSource(cp, code)
(cr.CompiledAssembly, cr.Output, cr.Errors)
Function compile returns tuple with compiled assembly and compilation output and error messages. If there are error messages, they are print to standard error stream and script execution is finished.
If no error messages are present, then all output messages are print to standard output stream and the generated assembly is executed.
let CompileAndExecute(code: string, references: seq<string>) =
let sw = new Stopwatch()
sw.Start()
let (assembly, output, errors) = compile code references
if errors.Count > 0 then
for e in errors do eprintfn "%s" (e.ToString()) done
else
for o in output do printfn "%s" o done
executeAssembly assembly
sw.Stop()
printfn "%s %i milliseconds." "Compile and execute takes" sw.ElapsedMilliseconds
Stopwatch object is used to measure the total time of compilation and execution. After script is finished, the total execution time is printed as well.
Function executeAssembly is used for actual compiled assembly execution.
let executeAssembly (a: Assembly) =
try
a.EntryPoint.Invoke(null, null) |> ignore
printfn "Execution successfully completed."
with
| :? TargetInvocationException as tex -> eprintfn
"Execution failed with: %s" (tex.InnerException.Message)
| ex -> eprintfn "Execution cannot start, reason: %s" (ex.ToString())
The EntryPoint property is used to run the script. This simplifies writing script much because it is not needed to wrap the code in a class or module.
All script's runtime errors are captured in a TargetInvacationException class and handled differently from all other errors to be able to distinguish scripting errors from the host program's errors.
Script Execution Optimization
As you can see, the first script execution takes lots of time even for tiny scripts. It takes 1713 milliseconds in the first screenshot. The reason of slowness is the compilation. F# compiler is quite intelligent and it requires some time to figure out all omitted types and perform required optimizations. However if the same script is running multiple times, it is better to compile it once and then execute as many times as needed. For this purpose, CompileAndExecute function uses CompiledAssemblies dictionary to map script code and compiled assembly. If for some provided code, the compiled assembly is found in the dictionary, then this assembly is executed and no additional compilation happens. But script can be quite long, so it is better to use not the text itself but its hash value. MD5 hash algorithm is used to compare script codes. It does not provide full proof comparison but for the most practical solutions, the probability of collision is negligibly small. Modified CompileAndExecute function with MD5 calculations is provided here:
let getMd5Hash (code: string) =
let md5 = MD5.Create()
let codeBytes = Encoding.UTF8.GetBytes(code)
let hash = md5.ComputeHash(codeBytes)
let sb = new StringBuilder()
for b in hash do sb.Append(b.ToString("x2")) |> ignore done
sb.ToString()
let CompileAndExecute(code: string, references: seq<string>) =
let sw = new Stopwatch()
sw.Start()
let hash = getMd5Hash code
if CompiledAssemblies.ContainsKey(hash) then
executeAssembly CompiledAssemblies.[hash]
else
let (assembly, output, errors) = compile code references
if errors.Count > 0 then
for e in errors do eprintfn "%s" (e.ToString()) done
else
for o in output do printfn "%s" o done
executeAssembly assembly
CompiledAssemblies.Add(hash, assembly)
sw.Stop()
printfn "%s %i milliseconds." "Compile and execute takes" sw.ElapsedMilliseconds
As you see in the screenshots, successive executions which use cached compiled assemblies are extremely fast. These executions take less than 1 millisecond and stopwatch reports 0 milliseconds.
Conclusions
F# scripting is extremely convenient because F# language is succinct, execution performance is pretty good and can be compared with other high performance solutions on C#, Java and sometimes C++. Also F# can be perfectly merged with any .NET environment utilizing broad variety of high quality .NET libraries.
However F# compilation itself is quite slow. If script code is changing much and execution of each code version isn't repeated, then pure dynamic languages (e.g. Python) with fast compilation stage are more suitable. Their execution performance and type safety is not so great as in F# but, considering compilation time, overall performance can be better.
