Expression Script (inspired from MIT Scratch)

github.com/Mohamed-Ahmed-Abdullah/ProgramDesigner

Introduction

This is basically a new visual programming language that enables you to write code and run it while at runtime. I was thinking about why we don’t have something like scratch in the business world, anyone can use it, simple intuitive and easy to learn. Sure, what I did till now isn’t even close to MIT scratch, but I will figure out how to continue and how we can do it.

Problem

I was thinking about why MIT Scratch is intuitively easy to use and learn, and why we don't have something like that in the business world. Can you imagine the users changing their own business rules, or can you imagine them solving the system bugs by themselves, or changing the workflow, or getting different result from some equation. From here, I started to research about how to do scratch myself but target it toward the business world.

This CodeProject article has a similar subject to what I am discussing here, but it doesn't have the UI part, and has a different grammar and a new concept and use cases and a vision.

Use Cases

We can use it in two places, I need you to take this town places as a case study.

First: imagine you have an HR system and you have the basic salary and an unknown number of allowances. Every customer has his own way to calculate this, but no matter how his equation looks like, you have the parameters saved in your Db, you can use it to get the result. You can write every equation with this expression script to return a single value used in the basic salary or one of the allowances.

Second: Imagine that you have an overtime workflow in that company, there are only 3 statuses for that workflow, but the transition logic is different from one company to another, either the logic is permission check or data validation. You can write the transmission logic with this expression script. The script will decide to move to the next step or not. If not, an exception will be thrown and the application should handle it gracefully.

When Can We Use It

What we have now is something we can’t use, for many reasons but the way is clear now, how we going to adjust this to get where we want.

We can’t use it because:

• We are using the tool to write normal code not as an interactive and clear meanings in other words, it’s simpler to write the code than drag and drop it.
• The grammar is not reached, it contains the expression, if statements and variables and one data type, but you can’t use “while statement” for example.

Terminologies

• Code Blocks: It’s simply anything you can write inside the method try catch, for, if, define variables, …
• Expression: It is a C# class Expression tree that represents code in a tree-like data structure, where each node is an expression, for example, a method call or a binary operation such as x < y.
• Grammar: A set of rules that define the language syntax.

How It Works

1. WPF UI: A canvas and a set of rectangles you can write your code with and it converted to simple string code
2. Irony: Irony is a development kit for implementing languages on .NET platform
3. Grammar
4. Irony Tree: Irony is giving a tree of your grammar
5. Expression Tree: We use the irony tree and convert it to expression tree

The idea here is simple. First, we need to create a grammar for Irony to use it for generating the code tree, then use the UI getting the code, pass it irony, in the result we are going to have the Code tree, taking this tree and convert it to C# Expression tree and getting the final result from it.

Using the Code

The program designer gives you the functionality of drag and drop, the output is the code string that will be passed to the expression compiler, if there is any syntax error, irony will point it out and return it, if there isn’t, we continue to generate the Code tree.

You will find ExpressionCompiler>Nodes set of classes. Every class represents a rule in the grammar and he is the responsible for generating part of Expression Tree from it. For example, VariableDeclarationNode.cs Class, its job is to create Expression.Parameter if a variable declaration is found, it will be converted to Expression Parameter.

After the conversion is completed, a single value will be returned.

 

UI

This is one canvas that has a toolkit in the left and a working area on the right, you can drag and drop the box from the toolkit to the working area (which is on the same canvas). After you finish the app, read all these boxes and convert them to code.

Drag And Drop

There is a control called DragGrip: Border inherited from Border and implementing new features:

• IsDragable
• IsSelected
• IsToolBarItem
• ContextMenue
• Clone()

The DragGrip representing anything you can drag inside the canvas, in other words, if it's not DragGrip, you can't drag it.

By changing the RenderTransform of this control to TranslateTransform, then changing the X and Y depending on the mouse move, yes it's that simple.

Snap Feature

snap is when a draggable control comes near other control, the draggable control will stick to that control, the snapping area defined by Point _snapOffset = new Point {X = 5, Y = 15}; which means if you come from left or right, the two controls (the stand still one and the draggable one) will not stick to each other until you reach 5 points near but if you come from the top or bottom, the snapping point is 15.

Selection And Deselection

There are two ways you can select or deselect with:

• Click the Box
• Drag on empty space in the Canvas and make sure the controls are inside the selection area

If you clicked the box (DragGrip), the control will handle it (changing the IsSelected to true) and this will change the UI and set the Selection Border, but if you drag to select many items...

...you have to calculate which control is inside the selection area and which is not:

//if(dragging in empty space in the canvas) then {draw rectangle}
if (isCanvasDragging && e.OriginalSource is Canvas)
{
    //get drag dimentions 
    var currentPosition = e.GetPosition(MainCanvas);
    var x1 = currentPosition.X;
    var y1 = currentPosition.Y;
    var x2 = clickPosition.X.ZeroBased();
    var y2 = clickPosition.Y.ZeroBased();
    Extentions.Order(ref x1, ref x2);
    Extentions.Order(ref y1, ref y2);

    //if(drag inside empty space in the canvas exceeded the limit that defined by _canvasDragOffset)
    if (x2 - x1 > _canvasDragOffset.X && y2 - y1 > _canvasDragOffset.Y)
    {
        _rectangleDrawn = true;
        //draw rectangle (change his shape depending on the mouse movment)
        SelectionRectangle.Visibility = Visibility.Visible;
        Canvas.SetLeft(SelectionRectangle, (x1 < x2) ? x1 : x2);
        Canvas.SetTop(SelectionRectangle, (y1 < y2) ? y1 : y2);
        SelectionRectangle.Width = Math.Abs(x1 - x2);
        SelectionRectangle.Height = Math.Abs(y1 - y2);

        //foreach(draggable child in the canvas)
        foreach (var child in MainCanvas.Children.OfType<DragGrip>())
        {
            //get his dimensions
            var translate = ((TranslateTransform) child.RenderTransform);
            var cx1 = translate.X;
            var cy1 = translate.Y;
            var cx2 = cx1 + child.ActualWidth;
            var cy2 = cy1 + child.ActualHeight;

            //if(control inside the selection area)
            if (x1 < cx1 && x2 > cx2 && y1 < cy1 && y2 > cy2)
            {
                //select it
                child.IsSelected = true;
            }
        }
    }
}

ToolBox

If you clicked over the toolbox button, the items that are related to that button will disappear from the toolbox below, this is done in a straight forwardmanner by:

Control.MouseDown += (a, b) =>
{
    if (b.LeftButton != MouseButtonState.Pressed)
        return;

    _isControlsVisible = !_isControlsVisible;

    //get the X dimension of the line that separate the toolbox from working area
    var x = Canvas.GetLeft(VerticalBarrier);

    MainCanvas.Children.OfType<DragGrip>()
        .Where( w =>
                ((TranslateTransform) w.RenderTransform).X <= x &&
                ((Border) w.Child).Background == Brushes.Orange)
        .ToList()
        .ForEach(f =>
        {
            f.Visibility = _isControlsVisible ? Visibility.Visible : Visibility.Collapsed;
        });

    NotifyPropertyChanged("");
};

There is a vertical line inside the canvas separating the toolbox from the working area.

Rename And Writing Value

This straightforward, it's just a matter of changing the textblock that held in DragGrip control by the new text from the TextBox.

Add New Draggable Item

We have types of the elements that you can drag right now (Control, Var, Token) but let's say you want to add reserved words like public, this,... what should you do?

1. Add the button in the toolBox Buttons and name it "Reserved Words"
2. And the reserved words in the toolbox, basically you should create a DragGrip controls and put them behind the Separation line, and specify the IsDragable to false and IsToolBarItem to true.
3. Give both a unique color

Code Extracting the Code from UI Elements

Every box (draggable element) has text inside, if you just take that text, you will have the code:

string code = "";
dragGrips.Select(s => ((TextBlock)((Border)s.Child).Child).Text).ToList().ForEach(f =>
{
    code += f;
});

try
{
    //pass it to Irony and getting the result
    Result.Text = Compiler.Compile(code).ToString();
}
catch (Exception ex)
{
    Result.Text = ex.Message;
}

Irony

Irony is a library that simplifies writing a compiler to the maximum because it will detect the syntax errors and many aspects of compilation process. You just to hand it your grammar, your conversion classes, and finally your code. Conversion classes will be discussed later in this article.

Using Irony

var grammar = new MyGrammar();
var compiler = new LanguageCompiler(grammar);
var program = (IExpressionGenerator)compiler.Parse(sourceCode);
var expression = program.GenerateExpression(null);

Any compiler in the world does one thing, convert your syntax (code) from shape to another, this case is nothing different, I want to convert the C# syntax to Expression Tree that will be described later on in this article.

Error Handling

Irony will give you this functionality out of the box. You just need to take these errors and show it in the UI.

var grammar = new MyGrammar();
var compiler = new LanguageCompiler(grammar);
var program = (IExpressionGenerator)compiler.Parse(sourceCode);

if (program == null || compiler.Context.Errors.Count > 0)
{
    // Didn't compile.  Generate an error message.
    var errors = "";
    foreach (var error in compiler.Context.Errors)
    {
        var location = string.Empty;
        if (error.Location.Line > 0 && error.Location.Column > 0)
        {
            location = "Line " + (error.Location.Line + 1) + ", column " + (error.Location.Column + 1);
        }
        errors = location + ": " + error.Message + ":" + Environment.NewLine;
        errors += sourceCode.Split('\n')[error.Location.Line];
    }

    throw new CompilationException(errors);
}

var expression = program.GenerateExpression(null);

Getting the Result

We are expecting one type of result - a decimal number at least for now but to implement the workflow case (see the use cases section) we are expecting a different type of results - an Exception.

return ((Expression<Func<decimal?>>)expression).Compile()();

Code Tree Conversion Classes

By writing your grammar, you are saying to Irony that this is the format (syntax) of my language, anything different, don't accept it.

First, you need to define the nodes.

Terminal: These are the leaves, the final thing in the tree like the numbers, reserved words, symbols + -,...

None Terminal: This is the variable, every none terminal has a rule - you change it with, like if you have x is non terminal and you have the rules x= 3 and x = 4, you can place x as 3 or 4 as you wish or more complex example if you have s = x +y and you have x = 0 |1| 2 | 3 |...|9 and y = ; your statement could be 1; or it could be 4; you get the picture.

var program = new NonTerminal("program", typeof(ProgramNode));
var statementList = new NonTerminal("statementList", typeof(StatementNode));
var statement = new NonTerminal("statement", typeof(SkipNode));
var expression = new NonTerminal("expression", typeof(ExpressionNode));
var binaryOperator = new NonTerminal("binaryOperator", typeof(SkipNode));

var variableDeclaration = new NonTerminal("variableDeclaration", typeof(VariableDeclarationNode));
var variableAssignment = new NonTerminal("variableAssignment", typeof(VariableAssignmentNode));

var ifStatement = new NonTerminal("ifStatement", typeof(IfStatementNode));
var elseStatement = new NonTerminal("elseStatement", typeof(ElseStatementNode));

// define all the terminals
var variable = new IdentifierTerminal("variable");
variable.AddKeywords("set", "var" , "to", 
"if", "freight", "cost", "is", "loop", "through", "order");
var number = new NumberLiteral("number");
var stringLiteral = new StringLiteral("string", "\"", ScanFlags.None);

RegisterPunctuation(";", "[", "]", "(", ")");

The rules:

Root = program;
binaryOperator.Rule = Symbol("+") | "-" | "*" | 
"/" | "<" | "==" | "!=" | ">" | "<=" | ">=" | "is";

program.Rule = statementList;
statementList.Rule = MakeStarRule(statementList, null, statement);
statement.Rule = variableDeclaration + ";" | 
variableAssignment + ";" | expression + ";" | ifStatement;

variableAssignment.Rule = variable + "=" + expression;
variableDeclaration.Rule = Symbol("var") + variable;

ifStatement.Rule = "if" + Symbol("(") + expression + Symbol(")")
                    + Symbol("{") + statementList + Symbol("}")
                    + elseStatement;
elseStatement.Rule = Empty | "else" + Symbol("{") + statementList + Symbol("}");

expression.Rule = number | variable | stringLiteral
    | expression + binaryOperator + expression
    | "(" + expression + ")";

Expression Tree

As we missioned before, we are trying to convert the code to expression, but what is Expression.

Any Linq expression you write, let's say numbersList.Where(w => w > 10); - this statement will be converted to Expression, if you try to reverse engineer code that has linq statement, you will not find the real where(w=>w>10) but you will find the expressions that mapped to that linq statement .

Microsoft definition of Expression: Expression trees represent code in a tree-like data structure, where each node is an expression, for example, a method call or a binary operation such as x < y. You can compile and run code represented by expression trees. This enables dynamic modification of executable code, the execution of LINQ queries in various databases, and the creation of dynamic queries.

So, for us to generate the expression tree, we have to tell irony how to do that, because this functionality is not provided out of the box, we have to write some code for it.

Let us see an example:

public class VariableAssignmentNode : AstNode, IExpressionGenerator
    {
        public VariableAssignmentNode(AstNodeArgs args): base(args)
        {}

        public Expression GenerateExpression(object tree)
        {
            var twoExpressionsDto = (TwoExpressionsDto)tree;
            return Expression.Assign(twoExpressionsDto.Expression2, twoExpressionsDto.Expression1);
        }
    }

When you have node like x =3, this is an assignment node and you want to convert it to assignment expression - the better thing to do is to create a class and create the functionality there, so, basically, you will have a class mapped to each node in your grammar, but this is not all.

We have the root node of our grammar:

var program = new NonTerminal("program", typeof(ProgramNode));

This is mapped to ProgramNode class so this is the first class that been called in the way of converting the code to expressions and irony will call it for you. From here, you take the control, you are the best one who knows the grammar so you need to make the ProgramNode class recursively call any class, else you created depending on the node.

ProgramNode>CompileStatementList()

First, we loop through the top nodes that were created by Irony.

foreach (var statement in childNode)

Then we determine the type of the node:

if (statement.ChildNodes[0] is VariableDeclarationNode) {}
else if (statement.ChildNodes[0] is VariableAssignmentNode) {}
else if (statement.ChildNodes[0] is ExpressionNode) {}
else if (statement.ChildNodes[0] is IfStatementNode) {}              

Irony will generate instances from your Node class for you, but you have to call GenerateExpression() to get the Expression from that node and to use it.

Finally, don't forget to use the lambda, as we said before, we are expecting one type of result here, so:

var lambda = Expression.Lambda<Func<decimal?>>(Expression.Block(
VariableList.Select(q => (ParameterExpression) q.FirstExpression).ToArray(),
lastExpressionNode));

Getting the Result

var grammar = new MyGrammar();
var compiler = new LanguageCompiler(grammar);
var program = (IExpressionGenerator)compiler.Parse(sourceCode);
var expression = program.GenerateExpression(null);
var result = ((Expression<Func<decimal?>>)expression).Compile()();

By calling program.GenerateExpression, Irony will call the GenerateExpression() that belongs to the root Node, in this case, it is ProgramNode method which is returning our precious value.

Future Enhancements

• Human-centric not code-centric: now you are just writing a code - you are not using a visual fun easy way to generate your business rules or equations
• Drop inside: You should able to see the full skeleton (is statement for example) and drop some code inside the condition and inside the “if” and in the “else” part.
• Better and more grammar

References

• irony.codeplex.com
• blogs.msdn.com/b/kirillosenkov/archive/2009/10/31/irony.aspx
• msdn.microsoft.com/en-us/library/bb397951.aspx
• msdn.microsoft.com/en-us/library/bb397951.aspx
• www.codeproject.com/Articles/29058/Writing-your-first-Domain-Specific-Language-Part
• www.codeproject.com/Articles/272494/Implementing-Programming-Languages-using-Csharp