Dynamic Prototyping

• Download source code (VS10 solution)

Contents

• Introduction
• The Problem
• Basic Infrastructural Concerns
  • Speed
  • Roundtrip and Shipping
  • Architectural Change
• Implementation
  • Default Implementation
  • Showing the Editor
  • Rebuilding the Type
  • Final Touches
• Conclusion
• History

Introduction

In this article, I want to talk about something called dynamic prototyping. The idea for this sort of software development approach came into my head as I got frustrated with the never-ending cycle of run-recompile-run that all of us developers are plagued with. So in this post, I’m going to explain the dynamic prototyping concept, discuss its implementation, and demonstrate its use.

The Problem

Software development tasks come in various sizes – some huge, some tiny. Depending on the functionality you’re trying to implement, a task may take a minute or a month. The minute-length tasks are very annoying, because their verification (let’s forget about unit tests for a moment) takes longer than the correction itself. They are even more annoying because they break my concentration.

As a result of this dissatisfaction, I started probing around the idea of getting rid of the roundtrip associated with recompile cycles. There are, essentially, just two options on how to handle this.

The first option is to rely on the DLR for code segments that need to be modified. However, this approach results in code that’s slower, and also gets us to use strange languages that we might not be willing to use in the first place.

The second option, and the one I favor, is in-process compilation. Since every .NET program has a capacity to compile other .NET programs from within, it is realistically possible to be able to modify a running program and – what’s more important – to be able to immediately see the results.

Basic Infrastructural Concerns

Since a diligent developer might instantly find several issues with this approach, I’m going to address some obvious concerns beforehand.

Speed

The first issue to address is: will the program be fundamentally slower because of this? Convention wisdom suggests that it will, quite simply because many of my readers are already thinking of cross-domain marshaling and things like that. In fact, you may have noticed that at no point in time did I suggest unloading assemblies. In actual fact, this is not necessary in order to get things running.

Why not? Well, mainly because nothing prevents you from loading several copies of the same assembly. Sure, you pay the price for that, and some additional fiddling is necessary, but the net result is that:

• The assembly you change is compiled and loaded immediately after you’re done with it. The newly loaded assembly is immediately put in use in lieu of the previous version.

Of course, prudent planning and thought-out development practices are required to guarantee that things don’t go wrong when loading the new assembly and using it. We’ll talk about those in a while.

Roundtrip and Shipping

So the next concern is somewhat multi-faceted. Namely, the following questions come to mind:

1. How do you both package the source code and compile it into the program at the same time?
2. How do you ensure that in-memory changes actually ‘return’ back to the program?
3. How do you ensure that none of this hackery ever gets to the deployed program?

Let me address these concerns in turn:

First, the source code is both compiled and embedded in the application. As a result, you already have a binary representation to work with, plus a textual representation that you can also use.

Second, in-memory changes return to the program because, realistically, there’s nothing preventing you from serializing some text to ..\..\SomeClass.cs. Your breakpoints (hint!) will be broken, of course, but apart from that, everything will function normally.

Third, the proposed infrastructure is intended (not saying that’s what you should do) to be Debug-only. Realistically, though, it doesn’t have to be – you can persist the program’s source code somewhere. Of course, you can also encrypt it so that it’s not human-readable.

Just to expand on that last point, the idea is that source code is typically available in the solution folder, and is not packaged in any way. Rather, it is simply kept where it needs to be – in the project directory! Now it should (hopefully) be obvious as to why there’s no real roundtrip concern – because we’re using the original program files for implementation!

Architectural Change

Okay, so you might imagine that a new assembly loaded with the same types as before would require re-wiring every necessary type in the IoC container, and you’re certainly correct – that’s how it’s done. However, this requirement is in line with best practices anyway, and arguing against it is like saying we shouldn’t brush our teeth.

Implementation

So, time to write some code. For the purposes of this article, I’ll present a minimal implementation of a program which is used for HTML encoding. The idea is that the HTML-encoding part of the program is an editable service, for which we’ll create provisions.

Default Implementation

Okay, so let’s say I have some interface called IConverter:

public interface IConverter
{
  string Convert(string source);
}

And subsequently I create my (minimal) implementation for HTML encoding:

public class HtmlConverter : IConverter
{
  public string Convert(string source)
  {
    var sb = new StringBuilder(source);
    sb.Replace("<", "&lt;");
    return sb.ToString();
  }
}

Now, being a good boy, I inject an IConverter into my main window, and use it to perform the conversion:

public partial class MainFrame : Form
{
  private IConverter converter;
  public MainFrame(IConverter converter)
  {
    this.converter = converter;
    InitializeComponent();
  }
  private void btnConvert_Click(object sender, EventArgs e)
  {
    tbConverted.Text = converter.Convert(tbSource.Text);
  }
}

Finally, I configure the container…

static void Main()
{
  var uc = new UnityContainer();
  uc.RegisterType<IConverter, HtmlConverter>();
  
  Application.EnableVisualStyles();
  Application.SetCompatibleTextRenderingDefault(false);
  Application.Run(uc.Resolve<MainFrame>());
}

And the application is usable:

Showing the Editor

Okay, so for the purposes of this exercise, I’m going to assume (meaning I’m not going to implement this just yet) that everything related to dynamic prototyping is either wrapped in #if DEBUG or has a [Conditional("DEBUG")] in front of this. And with this in mind, we can show the editor.

If you’re wondering as to how we figured out which file to open, the answer is simple: typically, you would need to write debug-specific code to actually show this window, so you may as well hardcode the file name. If you want a more industrial-strength solution, feel free to create your own infrastructure with nice little menus letting you specify the type/file to edit.

Rebuilding the Type

Okay, so we got an editor that lets us edit a file right from the solution and, naturally, lets us persist it back to where it came from. What next? Oh, we still have to compile and load the thing. Let’s deal with compilation first.

Compilation is actually easy, the only caveat being that you need to have a reference to your original executable and any related DLLs. Here’s an example:

public class InProcessCompiler
{
  public object CompileAndInstantiate(string sourceCode)
  {
    var ps = new CompilerParameters {GenerateInMemory = true, GenerateExecutable = false};
    ps.ReferencedAssemblies.Add("System.Drawing.dll");
    ps.ReferencedAssemblies.Add("System.Core.dll");
    ps.ReferencedAssemblies.Add("DynamicPrototypingSample.exe");
    var po = new Dictionary<string, string> {{"CompilerVersion", "v4.0"} };
    var p = new CSharpCodeProvider(po);
    var results = p.CompileAssemblyFromSource(ps, new[] {sourceCode});
    if (results.Errors.HasErrors)
    {
      var sb = new StringBuilder();
      foreach (var e in results.Errors)
        sb.AppendLine(e.ToString());
      throw new Exception(sb.ToString());
    }
    var ass = results.CompiledAssembly;
    var mainType = ass.GetTypes()[0];
    return Activator.CreateInstance(mainType);
  }
}

Plenty of hackery up above – specifying the explicit DLLs and our EXE to add as references, compiling source code under 4.0 and, last but not least – creating the first of all available types. Actually, that last point is prudent because in this case I’m only interested in compiling one type, and that’s the type found first in the generated assembly.

Final Touches

Having created an instance of the generated type, we do one simple thing: assign it to the variable. Thus, the whole process becomes as follows:

• Throw the editing UI
• Try compile the type
• If successful, save the new instance
• Overwrite the .cs file with new source

Here’s the event handler for our edit button:

private void btnEditProgram_Click(object sender, EventArgs e)
{
  string path = @"..\..\HtmlConverter.cs";
  var ce = new CodeEditor(path);
  if (ce.ShowDialog() == DialogResult.OK)
  {
    try
    {
      converter = (IConverter) Compiler.CompileAndInstantiate(ce.SourceCode);
      File.WriteAllText(path, ce.SourceCode, Encoding.UTF8);
    } 
    catch (Exception ex)
    {
      MessageBox.Show(ex.ToString());
    }
  }
}

And that’s pretty much it!

Conclusion

Okay, so as you can see, the implementation is not too complex. Of course, it can be ‘pruned’ so that the instrumentation of this in-process editing is only available in Debug code. And, by giving more time to the way you handle the compilation and ‘uptake’ of the code, you can make your prototyping development a little easier.

One useful thing which I do (but cannot show in this article) is using a commercial control for C# editing instead of just editing it as plain text. Apart from that, the set-up I use is pretty much like I described here.

Thanks for reading! Comments welcome!

History

• 3^rd November, 2010: Initial post