Background
Embedded scripting engines have applications in several areas, including the ability to extend or modify the core functionality of a software application. In the game development world, scripting provides a means for the game engine developer to hand over control to the designer, allowing him or her to implement the game's plotline events, NPC behaviour and so on, without the intervention of the game engine developer who may otherwise need to hard-wire game play logic into the game engine.
Introduction
Following the interpreted Eezeescript embeddable scripting language presented in a previous article, this article describes Conscript, a more sophisticated scripting engine for .NET applications. Conscript compiles script sources into byte code for the Conscript VM, resulting in faster scripts than otherwise possible using an interpretive engine. Source code and binaries for the scripting engine and an illustrative demo are also included with the article.
The demo application presented in this article is a Windows application offering a simple testing environment for Conscript. The user may type scripts in the left panel and click the "Run Script" button to compile and run the script. The application exposes one host function, Print
, to allow the user to output data from the script onto an output panel at the bottom right. The top right panel displays the compiled script in Conscript assembly form.
Architectural Overview
The scripting system compiles scripts written in a high-level, C-like language down to a virtual instruction set designed for Conscript. The virtual instruction set is reminiscent of Intel instruction sets with additional instructions provided to simplify the design of the compiler. The Virtual Machine is a hybrid system featuring separate stacks for function frames and parameter passing; a memory system based on hierarchical dictionaries; a concurrency management system; and an interface allowing the invocation of native functions registered with the scripting system. In addition, the instruction operands and the variables stored in memory are of variant type with support for integers, floating point values, boolean values, strings and associative arrays. A number of instructions, such as ADD, operate on different types. In some cases, even mixed type operands are supported. A typical example is concatenation of a string with an integer.
Using the Scripting Engine API
This section provides a quick introduction fpr integrating Conscript into a .NET solution and using its API.
Integrating Conscript into a .NET solution
The Conscript host API can be integrated into an existing .NET solution by adding a reference to the Conscript.dll assembly. All API classes are defined within an Conscript
namespace that must be either specified with the using
clause or prefixed to the API classes.
Preparing the scripting environment
The scripting system is initialised by creating one or more instances of the ScriptManager
class. Each instance represents a scripting environment where scripts can be loaded and executed. Each instance also provides a global variable scope that the scripts can use to share data.
ScriptManager scriptManager = new ScriptManager();
Loading Scripts
Once a script manager is available, scripts can be loaded by creating instances of the Script
class. The script object's constructor requires a reference to the script manager and a name to identify the script. By default, this name corresponds to a disk filename.
Script script = new Script(scriptManager, "scripts\Wizard.cns");
The script object's constructor automatically compiles the script source into Conscript byte code. By default, the generated byte code is optimised using a "peephole" assembly optimiser. Debug instructions are added to facilitate mapping of the generated code with the original source. These settings may be controlled by setting the ScriptManager
's properties DebugMode
and OptimiseCode
.
scriptManager.DebugMode = false;
scriptManager.OptimiseCode = false;
Preparing Scripts for Execution
A script object represents only the programming instructions contained within and not its execution state. To execute the script, an instance of the ScriptContext
class must be created. The class' constructor requires a reference to the script to be executed or a reference to one of the script's functions. A script reference implies that the main
function will be executed. The script context provides execution control, as well as access to the execution state in terms of the variables defined during execution, the next statement to be executed and so on. The ScriptContext class represents a running instance of a script. Hence, multiple instances of the same script object can be executed within the same script manager by creating multiple script contexts referencing the same script.
ScriptContext scriptContext = new ScriptContext(script);
ScriptContext scriptContext = new ScriptContext(script.MainFunction);
ScriptFunction scriptFunction = script.Functions["WanderAround"];
ScriptContext scriptContext = new ScriptContext(scriptFunction);
Executing Scripts
The script context object allows execution of the referenced script via the three variants of its Execute
method. These are: execution of scripts for an indefinite amount of time; execution of scripts for a given time interval or execution of scripts for up to a maximum number of executed statements.
The first method variant allows the referenced script function to execute indefinitely or until the end of the function is reached. If the script contains an infinite loop, this method will block indefinitely unless an interrupt is generated. The Execute
method returns the total number of statements executed since its invocation.
scriptContext.Execute();
The second variant of the Execute
method allows the script context to execute up to a given maximum number of statements. The script context may break out of execution before the maximum is reached if there are no more statements to process or if an interrupt is generated.
scriptContext.Execute(10);
The third variant of the Execute
method accepts a TimeSpan
defining the maximum time interval allowed for script execution. The method may break out of execution earlier than the given interval if there are no more statements to process or an interrupt is generated. Given a script with a good balance of different statements, a possible use of this method is to determine the speed of the scripting system on the target environment in terms of statements executed per second.
TimeSpan tsInterval = new TimeSpan(0, 0, 0, 0, 10);
scriptContext.Execute(tsInterval);
The second and third variants of Execute
may be used to implement a virtual multi-threaded scripting environment. Global variables may be used as semaphores to synchronise concurrently running scripts.
Interrupting and Resetting Scripts
A script context will normally execute its referenced script function indefinitely, for a given time interval, until a given maximum number of statements are executed or until there are no more statements to process. In some cases it is desirable to break execution prematurely, such as to return control to the host when specific statements are executed, or because a script is too computationally intensive to execute in one go.
Conscript provides two ways for generating script interrupts:
- Using a
yield
statement to explicitly break execution at a specific point in the script
- Enabling the script context object's
InterruptOnHostfunctionCall
property to generate an interrupt automatically whenever a registered host function is executed
The "Dungeon of Despair" demo game, illustrated in a subsequent article, uses the second approach to allow each NPC script to be written as if it is meant to run in isolation from other scripts. Control is relinquished to the associated character object whenever a host function is executed. This in turn allows the character object to queue and process the corresponding actions. While the actions are in progress, the script context is kept in a suspended state. When the actions are completed, execution of the script is resumed.
An executing script may be reset via the script context's Reset
method. The net effect of invoking this method is that the local variables defined in the context are lost, the execution frame stack is cleared and the statement pointer is reset to point to the first statement in the script block referenced by the script context. In addition, any active locks requested by the context are cleared. Global variables are not affected and persist after a script context is reset.
Custom Script Loaders
To allow for loading of scripts from other sources -- such as an archive file, network or database -- a custom script loader class can be developed and bound to the script manager to be used for loading the script. The script loader class may be any class that implements the ScriptLoader
interface. The loader is used to retrieve the script specified in the Script class constructor and also any additional include scripts defined within the original script.
public class MyScriptLoader
: ScriptLoader
{
public List<string /> LoadScript(String strResourceName)
{
}
}
ScriptLoader scriptLoader = new MyScriptLoader();
scriptManager.Loader = scriptLoader;
Accessing the Local and Global Variable Scope
One approach for allowing a script to communicate with or control the host application entails the application polling the local variables of the associated script context and global variables of the associated script manager. It does this by querying the LocalDictionary
property of the ScriptContext
object, the ScriptDictionary
property of the Script
object or the GlobalDicitonary
property of the ScriptManager
object.
int iIndex = (int) scriptContext.LocalDictionary["index"];
int iScore = (int) script.ScriptDictionary["s_playerScore"];
bool bNewQuest =
(bool) scriptManager.GlobalDictionary["g_wizardQuestAvailable"];
Registering Host Functions
A more powerful alternative to allow a script to interface with the host application is to register host functions with the script manager and assign a script handler at script context or manager level. The script handler in turn provides an implementation for the host functions. These functions are first defined by creating an instance of the HostFunctionPrototype
class to define the functions' names, parameters and return values. Unlike script-defined functions, host functions can enforce type-checking on parameters and return values that are performed at compile time. Thus, scripts that use host functions must be loaded within a script manager that has the required host functions registered beforehand.
HostFunctionPrototype hostFunctionPrototype = new HostFunctionPrototype(
null, "Player_MoveTo", typeof(int), typeof(int));
A number of constructors are provided for quick construction of function prototypes with up to three parameters. For functions requiring more parameters, a constructor that accepts a type list, List<Type>
, is also provided:
List<Type> listParameterTypes = new List<Type>();
listParameterTypes.Add(typeof(String));
listParameterTypes.Add(typeof(int));
listParameterTypes.Add(typeof(int));
listParameterTypes.Add(typeof(bool));
listParameterTypes.Add(typeof(AssociativeArray));
HostFunctionPrototype hostFunctionPrototype = new HostFunctionPrototype(
typeof(float), "FuncWithManyParams", listParameterTypes);
Once the host function prototype is defined, it can be registered with the script manager. The function prototype ensures that a corresponding host function is recognised during compilation and runtime and that the parameters passed alongside correspond in number, type and order to those defined in the function prototype. The following statement registers a host function prototype without a handler:
m_scriptManager.RegisterHostFunction(hostFunctionPrototype);
The registration method illustrated above requires handler objects to be attached to every ScriptContext
created from a script that uses the host function. A handler may be any class that implements the HostFunctionHandler
interface consisting of the method OnHostFunctionCall(...)
. A good approach is to extend the class that owns the relevant script context. This is illustrated by the following example, which implements host functions for enemy A.I. scripts in a game:
public class Enemy
: HostFunctionHandler
{
private ScriptContext m_scriptContext
public Enemy(..., Script script, ...)
{
m_scriptContext = new ScriptContext(script);
m_scriptContext.Handler = this;
}
public object OnHostFunctionCall(String strFunctionName,
List<object> listParameters)
{
if (strFunctionName == "GetRandom")
{
int iMin = (int)listParameters[0];
int iMax = (int)listParameters[1];
return s_random.Next(iMin, iMax);
}
else if (strFunctionName == "Move")
{
int iDeltaX = (int)listParameters[0];
int iDeltaY = (int)listParameters[1];
Move(iDeltaX, iDeltaY);
}
else if (strFunctionName == "SetDirection")
{
Direction = (EntityDirection)(int)listParameters[0];
}
else if (strFunctionName == "SetAutomaticDirection")
{
AutomaticDirection = (bool)listParameters[0];
}
else if (strFunctionName == "GetPosition")
{
AssociativeArray associativeArrayPosition =
new AssociativeArray();
associativeArrayPosition["X"] = this.X;
associativeArrayPosition["Y"] = this.Y;
return associativeArrayPosition;
}
else if (strFunctionName == "GetPlayerPosition")
{
AssociativeArray associativeArrayPosition =
new AssociativeArray();
associativeArrayPosition["X"] = m_player.X;
associativeArrayPosition["Y"] = m_player.Y;
return associativeArrayPosition;
}
else if (strFunctionName == "CastSpell")
{
Image imageSpell = Properties.Resources.EnemySpell;
int iDamage = m_enemyType == EnemyType.Wizard ? 40 : 20;
Spell spell =
new Spell(m_enemies.Room.Spells,
imageSpell, false, iDamage, Direction);
spell.X = X;
spell.Y = Y - 30;
m_enemies.Room.Spells.Add(spell);
return null;
}
return null;
}
Worth noting in the above example is that no parameter validation is performed, nor is it needed because the validation is performed at compile time.
The ability to define a different handler for every script context allows a host function to have different implementations and / or contexts depending on the script contexts to which respective script handlers are bound. For example, the implementation for a Move(x, y)
function within a script might affect the movement of a player character, a non-player character or a projectile.
Alternatively, a host function prototype may be registered with an associated handler directly:
HostFunctionPrototype hostFunctionPrototype = new HostFunctionPrototype(
typeof(float), "Sin", typeof(float));
m_scriptManager.RegisterHostFunction(hostFunctionPrototype, trigHandler);
HostFunctionPrototype hostFunctionPrototype = new HostFunctionPrototype(
typeof(float), "Cos", typeof(float));
m_scriptManager.RegisterHostFunction(hostFunctionPrototype, trigHandler);
HostFunctionPrototype hostFunctionPrototype = new HostFunctionPrototype(
typeof(float), "Tan", typeof(float));
m_scriptManager.RegisterHostFunction(hostFunctionPrototype, trigHandler);
This approach allows the association of a common handler that is shared amongst all script contexts created within the same script manager. A typical example is the registration of trigonometric and other common math functions required in many scripts.
Registering Host Modules
Registering shared host functions individually required a substantial amount of setup code. To alleviate this problem, a HostModule
interface is provided to allow the registration of multiple host functions in bulk. The implementation of a host module entails defining a method that returns a list of function prototypes implemented by the module, together with a handler method as per the HostFunctionHandler
interface. This provides an implementation for the functions. The following illustrates an alternative implementation of the earlier trigonometry functions example:
public class TrigonometryModule
: HostModule
{
private static ReadOnlyCollection<hostfunctionprototype /> s_listHostFunctionPrototypes;
public TrigonometryModule()
{
if (s_listHostFunctionPrototypes != null) return;
List<hostfunctionprototype /> listHostFunctionPrototypes =
new List<hostfunctionprototype />();
HostFunctionPrototype hostFunctionPrototype = null;
hostFunctionPrototype =
new HostFunctionPrototype(typeof(float), "Sin", typeof(float));
listHostFunctionPrototypes.Add(hostFunctionPrototype);
hostFunctionPrototype =
new HostFunctionPrototype(typeof(float), "Cos", typeof(float));
listHostFunctionPrototypes.Add(hostFunctionPrototype);
hostFunctionPrototype =
new HostFunctionPrototype(typeof(float), "Tan", typeof(float));
listHostFunctionPrototypes.Add(hostFunctionPrototype);
s_listHostFunctionPrototypes =
listHostFunctionPrototypes.AsReadOnly();
}
public ReadOnlyCollection<hostfunctionprototype /> HostFunctionPrototypes
{
get { return s_listHostFunctionPrototypes; }
}
public object OnHostFunctionCall(String strFunctionName,
List<object> listParameters)
{
if (strFunctionName == "Sin")
return (float)Math.Sin((float)listParameters[0]);
else if (strFunctionName == "Cos")
return (float)Math.Cos((float)listParameters[0]);
else if (strFunctionName == "Tan")
return (float)Math.Tan((float)listParameters[0]);
throw new ExecutionException(
"Unimplemented function '" + strFunctionName + "'.");
}
}
Host module registration is very similar to individual function registration:
TrigonometryModule trigonometryModule = new TrigonometryModule();
m_scriptManager.RegisterHostModule(trigonometryModule)
A future article will present a more advanced Conscript demo containing a fully-featured IDE application for the scripting language. The IDE extends the notion of host modules via the implementation of a plug-in system that allows the user to load any .NET assembly into the IDE. The assembly is scanned for all classes implementing the HostModule
interface and their host functions are registered automatically.
Error Handling
API usage errors, compilation errors and script runtime errors trigger exception instances of ConscriptException
, ParserException
and ExecutionException
respectively. Each exception may contain inner exceptions at an arbitrary number of levels. The compiler exception ParserException
provides additional properties describing the source location and possibly a token where a lexing or parsing error occurred.
Scripting Language Guide
This scripting engine is a natural progression from the much simpler Eezeescript and features a fully fledged C-like language. Each script may consist of a number of global or script variable declarations, followed by a number of function declarations.
Comments
Comments may be interspersed throughout a script either in the form of single-line comments denoted by
or by using a multi-line block comment of the form
.
Global and Script Variable Declarations
Global declarations may be specified multiple times across a number of scripts sharing the same global context. Whenever a global declaration is encountered, the associated variable is initialised in the global scope unless a script variable with the same name is encountered. Script variables, on the other hand, are only accessible to the functions declared in the same script. Variables of either scope are initialised with a null value.
global g_world;
var s_name, s_surname;
Function Declarations
Function declarations require a function identifier followed by a number of untyped parameter identifiers. A function must return a null reference, a native value or an array value. Functions may invoke other functions, including recursive self-references or registered host functions.
function abs(x)
{
if (x >= 0)
return x;
else
return -x;
}
Statement Blocks
Statement blocks enclose multiple semicolon-delimited statements in an opening and closing brace-pair. Statement blocks define function bodies and are used in control statements.
function main()
{
{
var x = 10;
x = x * 2;
}
if (x < 20)
{
var y = x + 5;
}
}
When used in a control statement, a statement block consisting of only one statement may be written without braces:
if (x >= 0)
return x;
else
return -x;
Statements
Statements form the core of the Conscript scripting language and can consist either of an empty statement, denoted by a semicolon, a local variable declaration, an expression, a control construct, a concurrency control construct or a thread invocation. In addition, a statement may itself be a statement block enclosed by braces, allowing for arbitrary nesting of statement blocks.
Expressions
Because of the C-like nature of the scripting language, expressions encompass a variety of constructs. These include arithmetic, logical and relational expressions, variable assignments and function calls. Expressions may consist of an arbitrary mix of operands and operators and follow operator priority rules similar to C. The default operator order may be overridden using parentheses. Expressions can appear as standalone statements, as an array element reference or as an argument to a function call. Assignments are also treated as expressions whose value is equal to the right-hand side of the assignment. Conversely, the return value of a standalone expression is lost.
count = rows * columns + 2;
x = 4.5
name = "joe";
found = true;
list = {};
fullname = name + " " + surname;
list[index + 1] = "Desk";
dictionary["orange"] = 15;
matrix[row][column] = 1.0;
dictionary.orange = 15;
address.fullname.surname = "Smith";
count += 5;
list += {"Chair", "Sofa"};
x -= delta;
(1 + 2) * (3 + 4);
true;
Print("Hello World!");
root = sqrt(x);
sortedList = sort(list);
x1 = - (b + sqrt(b * b - 4 * a * c)) / (2 * a);
wet = raining && noUmbrella || plumbingProblem;
inRange = x >= lower && x <= upper;
A script may call either script-defined functions or natively defined .NET code registered as host functions. All host functions referenced by a script must be registered prior to attempting compilation.
Local Variable Declarations
Local variables are defined in a similar manner to script variables and their scope is limited to the function body in which they are defined. Local declarations may, however, include an initial assignment expression based on function calls, literals and previously defined variables. If an assignment is omitted, a null value is assigned by default.
var x, y, z;
var name = "Joe", surname = "Smith";
var count, found = true;
var list;
var list = null;
Conditional Statements
Conscript provides two conditional statements: the if-then-else block and the switch block. The if-then-else block is essentially identical to its C-language equivalent. Unlike C and other C-style languages, the switch block allows comparisons on any type of variable and requires the use of statement blocks instead of the "break" keyword to define the case conditions.
if (count > MAX)
count = MAX;
if (x >=a && x <= b)
{
x = (a + b) / 2;
inRange = true;
}
else
{
inRange = false;
}
switch (item.Type)
{
case "health":
{
player.health += 10;
if (player.health > MAX_HEALTH)
player.health = MAX_HEALTH;
}
case "weapon":
player.weapon += 1;
case "poison":
case "mine":
player.health = 0;
default:
return UNKNOWN_ITEM_TYPE;
}
Iterative Statements
The scripting language provides three looping constructs: "while" loops, "for" loops and "for-each" loops. The syntax of "while" and "for" loops is very similar to the C syntax. The "for-each" syntax borrows from managed languages that support a similar construct and may be used elements of an array or characters of a string variable.
while (!found)
{
found = true;
}
for (var index = 0; index < list.size; index++)
sum += list[index];
foreach (key, value in dictionary)
Print(key + ": " + value);
foreach (key in dictionary)
Print(key + ": " + dictionary[key]);
All looping constructs support the "break" and "continue" keywords to respectively exit the loop or skip to the next iteration.
for (var index = 0; index < 10; index++)
{
if (index < 4) continue;
if (index > 7) break;
}
Multi-threading
Depending on the execution mode in use by the scripting engine, a script may retain execution control for substantial periods of time, potentially impacting the host application. A "yield" statement may be used to trigger an interrupt, causing the scripting engine to relinquish control back to the host application.
for (var index = 0; index < 100000; index++)
{
if (index % 100 == 0)
yield;
}
Multi-threading is possible either by executing multiple script contexts through the scripting engine API or by asynchronous execution of script-defined functions using the "thread" keyword. This invocation method causes the invoked function to execute in parallel with the main execution thread of the script.
var finished;
function RunProcess(processId)
{
for (var index = 0; index < 100; index++)
Print("Process " + processId + ": " + index);
++finished;
}
function main()
{
finished = 0;
thread RunProcess(1);
thread RunProcess(2);
while (finished < 2)
yield;
}
Concurrency Control
Conscript supports a wait-notify concurrency mechanism and a critical-section mechanism via a locking construct.
var QUEUE_LOCK;
var queue;
function enqueue()
{
var value= 0;
while(true)
{
lock QUEUE_LOCK
{
queue += value;
Print("Queued: " + value);
++value;
}
}
}
function dequeue()
{
while(true)
{
lock QUEUE_LOCK
{
var index, value;
foreach (index, value in queue)
{
queue -= value;
Print("Dequeued: " + value);
break;
}
}
}
}
function main()
{
QUEUE_LOCK = "queue_lock";
queue = {};
thread enqueue();
thread dequeue();
while(true) yield;
}
Modular Coding
Conscript provides a simple code inclusion mechanism similar to the #include
pre-processor directive provided in C and C++.
global UP, DOWN, LEFT, RIGHT
function InitialiseConstants()
{
UP = 0;
DOWN = 1;
LEFT = 2;
RIGHT = 3;
}
include "common.cns";
function SetDirection(direction)
{
switch (direction)
{
case UP : { ... }
case DOWN : { ... }
case LEFT : { ... }
case RIGHT : { ... }
}
}
Points of Interest
Designing an instruction set for a virtual machine architecture that accommodates a higher level language, rather than the other way round, offers several advantages. These include the ability to include complex instructions, variant-type memory addressing and registers. This approach offers a further advantage: complex instructions reduce the overall size of the script executables, meaning that the VM can execute the instructions more quickly while the bulk of processing is performed "natively" within the VM.
Related Articles
- EezeeScript: A simple embeddable scripting language for .NET
- Dungeon of Despair: A scripted demo game using Conscript
- Conscript IDE: An Integrated Development Environment (IDE) implementation for the Conscript scripting language.
History
- 12/Jun/2007 - First version released