Introduction
This article in three parts looks at the development of a standard framework for a Sketch running on an Arduino board, and a .NET Class Library to encapsulate the interface to the board for a control programme in Windows.
The sketch framework provides a standard way of receiving commands from the control application and of reporting both status information (more or less static) and dynamic values (for example read from input pins on the board).
The Class Library allows the control application developer to be less concerned about the nuts and bolts of driving the serial comms, and to write an application that will work with devices having different capabilities.
Background
In my work (at a University Psychology School) we frequently require to measure various environmental and biometric parameters during research experiments as well as using a range of ‘unusual’ input and output devices to present a stimulus to a participant and detect a response.
This has previously involved a mixture of custom electronic design and interface to a computer controlling the experiment and/or hacking standard devices like keyboards and joysticks to be able to use special buttons or analogue inputs.
We are also typically concerned with reaction times, so minimising lag or jitter affecting timing accuracy can be critical.
When the Arduino was released it was immediately obvious that this could provide a standardised interface to a wide variety of sensors and be able to control all sorts of different output devices.
Various biometric and environmental sensors became available to connect directly to the Arduino, and their values could be presented to the experiment either over the USB serial port, or by configuring the Arduino to emulate a standard keyboard or game controller.
A number of issues quickly became apparent – the most basic being how to consistently identify what sketch and which version was running on any given board. In addition it was useful to be able to ask the firmware what capabilities it had and have the answer returned in a standard format.
Some commands are common across a wide range of applications – for example setting the length of the main sketch program loop, and this led to developing a standard command format so that all experiment applications were able to communicate with the boards in a common way.
This further allowed the development of a class library to encapsulate the communication with the board and remove some of the complications of having to deal with the nuts and bolts of serial communication from the programmer/researcher developing the main experiment software.
This article presents a particular solution to these issues which should be applicable in many different situations beyond our Psychology research programs. If you are using Arduinos with different sensors and controlling different hardware, then having a standard framework can make life a lot simpler.
There are three parts to this. In the first we will define a common command set and protocol and implement a standard sketch framework. At this stage we are not concerned with the control computer end of things, a simple terminal programme is all that is required – although the Arduino IDE own terminal is not capable of sending control characters to the board so you will need something like TeraTerm.
The second part develops a basic class library to encapsulate the interface to any board using the standard framework. A demo control programme is developed in the third part.
Using the code
One of the first decisions was to standardise on a common baud rate for serial communication between the PC and the Arduino. At 9600 baud the bit time on the wire is about 104 microseconds (us) so a single byte (or character) of 10 bits (including start+stop) takes 1.04 milliseconds (ms) to transmit. Since we are interested in timing accuracy down in the millisecond range and sample intervals in the 10’s of milliseconds this was a little slow. We decided to standardise on 115200 baud and to as far as possible use single characters for commands and responses. At this rate a character takes about 87us to transmit on the wire – there is then the low level processing overhead at both ends to drive the serial port and present the received value to the application.
Returning as a string value from a pin will typically use 5 chars which will take nearly 0.5 ms to transmit, In practice that means we will not be able to poll faster than about 2ms loop delay with this protocol, but that is ok for almost all our application.
Having agreed a common baud rate the next issue was simply to be able to identify what firmware was running on a given board.
In order to minimise response times when issuing cmmands we decided that a command should consist of a single character, optionally followed by characters providing any parameters needed, followed by a linefeed. We decided to reserve ASCII control characters (values less than 32) for standard commands, and allow application to use all other characters for their specific needs.
In the standard framework we would store the name of the application, its version number, the date it was compiled, and the developer name. We also decided to automatically give each board a unique name at compile time.
At last some sketch code:
#include <avr/pgmspace.h> //enable use of flash memory
static String unitID = "Demo-" + String(__TIME__);
const char sketchName[] PROGMEM = "SoP Demo"; const char sketchVer[] PROGMEM = "1.0.0"; const char sketchDate[] PROGMEM = __TIMESTAMP__;
const char sketchAuthor[] PROGMEM = "RogerCO"; const char* const sketchArr[] PROGMEM = { sketchName, sketchVer, sketchDate, sketchAuthor };
char strbuffer[50];
What we have done here is define some standard information as string constants and are saving them in program (flash) memory PROGMEM to save SRAM space.
The UnitID we have stored in SRAM as a variable so that we can, if we wish, change it later.
We are going to use Ctrl S (ASCII 19) as a single character command to which the unit will respond by returning the sketch info.
The main program loop will generally do whatever polling is required on input pins and then check the serial port for any incoming commands and respond as appropriate.
static long loopTime = 20;
void loop()
{
char serialChar;
while (Serial.available() > 0) {
serialChar = Serial.read();
if (serialChar = 19) { reportSketch();
} else {
}
delay(loopTime);
}
When we return status values from the sketch we are going to use a standard format of a single character identifier followed by an equals sign followed by the value and a line termination. This will allow us to parse the responses in a standard way.
void reportSketch(void) {
strcpy_P(strbuffer, (char*)pgm_read_word(&(sketchArr[0]))); Serial.print("S="); Serial.println(strbuffer);
strcpy_P(strbuffer, (char*)pgm_read_word(&(sketchArr[1]))); Serial.print("V="); Serial.println(strbuffer);
strcpy_P(strbuffer, (char*)pgm_read_word(&(sketchArr[2]))); Serial.print("D="); Serial.println(strbuffer);
strcpy_P(strbuffer, (char*)pgm_read_word(&(sketchArr[3]))); Serial.print("A="); Serial.println(strbuffer);
Serial.print("N="); Serial.println(unitID);
}
Here we are reading the information from PROGMEM and writing the identifier and the string to the serial port.
Rather than hard coding the identifiers it would be better to start by defining them all as constants. In general there are two types of data that a sketch will be returning - status information which rarely changes, and dynamic values which are returned every time around the main loop or in response to an interrupt input on the Arduino.
For status information we will identify what is being returned by a single character followed by an equals sign followed by the data followed by a line termination. As well as sketch name, version, date, author and unit ID shown above there are various other standard commands and responses that we might need. We will be reserving a bunch of upper case identifier characters for standard responses. We will not use control characters for the response labels so that they remain human readable.
If we are polling at more than ten times a second (loop time less than 100ms) and sending values back over the serial port every time around the loop there can be quite a flood of data arriving at the computer. It might be convenient to turn this off or on from the control end. Also if there are pins that are configured as interrupts on the Arduino, or we are using timer interrupts which are generating output on the serial port we might also wish to enable or disable these remotely.
A common requirement turned out to be to adjust the length of the main program loop, and also to report back any errors or jitter in the loop time.
static unsigned long loopTime = 20; static boolean sendSerialValues = false; static boolean sendSerialInts = false;
We also identified that we would like the sketch to be able to tell us how its input and output pins were being used, what commands it responds to (in addition to the standard set), and what status or values it is returning.
Here is a set of identifiers for the standard control character commands we are using
static const byte cmdLoopOn = 1; static const byte cmdIntOn = 2; static const byte qCmdDefn = 3; static const byte qStatusDefn = 4; static const byte qValDefn = 5; static const byte qIns = 6; static const byte qOuts = 7; static const byte qSketch = 19; static const byte cmdLoopTime = 20; static const byte cmdUnitID = 21; static const byte cmdLoopOff = 24; static const byte cmdIntOff = 25; static const byte qErrors = 26;
and here is a set of corresponding status labels
static const String lblAuthor = "A="; static const String lblCmd = "C="; static const String lblDate = "D="; static const String lblStatus = "E="; static const String lblValue = "F="; static const String lblIn = "G="; static const String lblOut = "H="; static const String lblIntOn = "I="; static const String lblUnitName = "N="; static const String lblLoopOn = "O="; static const String lblSketch = "S="; static const String lblLoopTime = "T="; static const String lblVersion = "V="; static const String lblErrors = "Z=";
In addition there will be some sketch specific commands and responses to be defined. For example suppose our board is reading an analogue input and reporting the value back to the control program every time around the loop. It may also be flashing an LED to indicate that a threshold value has been reached. We might want commands to enable or disable the LED output and to set the minimum duration of the LED pulse.
static const byte cmdDoLED = 'L';
const char Lcstr[] PROGMEM = "L=Enable LED";
static const byte cmdNoLED = 'l';
const char lcstr[] PROGMEM = "l=Disable LED";
static const byte cmdPulseWidth = 'p';
const char pcstr[] PROGMEM = "p=Set/Get LED pulse duration (ms)";
const char* const cmdArr[] PROGMEM = { Lcstr, lcstr, pcstr };
static int cmdSize = 3;
Here we have defined three commands that this sketch is going to respond to, and their descriptions. Where we are turning a function on or off we will use the upper case command to turn it on and the lower case to turn it off. If a command requires some parameter, eg a value for the pulse width, then it will follow immediately after the command character and be terminated with a line feed.
We will also need to define corresponding status labels and descriptions for the sketch specific values and status info. Again we will store these in PROGMEM.
The definition responses all have a standard format. Each definition is on one line starting with the appropriate label, then the definition then a linefeed. So when the sketch receives the command, eg qCmdDefn it responds by iterating through the list of valid commands for this sketch and returning each one on a separate line preceded by lblCmd.
In response to the command CtrlS as previously seen we respond with the information about the sketch. The response to CrtlC is to return a list of additional commands that the sketch will recognise together with their descriptions. Each command and description will be listed on a separate line starting with "C="
int x;
void reportCommands(void) {
for (x = 0; x < cmdSize; x++) {
strcpy_P(strbuffer, (char*)pgm_read_word(&(cmdArr[x])));
Serial.print(lblCmd);
Serial.println(strbuffer);
}
}
This function will iterate through the array of command definitions in PROGMEM and output them to the serial port. The control programme will receive this:
C=L=Enable LED
C=l=Disable LED
C=p=Set/Get Output pulse width (ms)
Now the control programme can know that if we send to the Arduino an “L” at the beginning of a line it will enable the LED output, and if we send “p500” it will set the pulse width to 500ms.
Similar functions are defined for each of the other standard definition commands CtrlE, CtrlF, CtrlG, and CtrlH: reportStatusCodes(), reportValueCodes(), reportInputPins(), reportOutputPins()
We will return any dynamic values that are reported every time around the loop again using a single character identifier, then a colon ":", the the value as a string of chars terminated with a linefeed. If required we can reduce the output by only returning values when they change, or by holding them in local storage and reading them out in response to a specific command.
static int anaVal;
static const String lblAnaVal = "A:";
const char avstr[] PROGMEM = "A:,Analogue input value (arbitary units)";
const char* const valArr[] PROGMEM = { avstr };
const int valSize = 1;
If sendSerialValues
is true then we will be getting a stream of values from the board
A:126
A:132
A:129
...
Now we can write a generic function to handle control commands and specific functions for each command.
In main programme loop
while (Serial.available() > 0) {
serialChar = Serial.read();
String cmdstr;
if (serialByte < 28) { doStdCmd(serialChar);
} else {
}
The doStdCmd() function to handle the standard commands (ascii values less than 32) and an example function reading a parameter value after the command to set the loop time
void doStdCmd(char cmd) {
switch (cmd) {
case qSketch:
reportSketch();
break;
case cmdLoopOn:
sendSerialValues = true;
Serial.print(lblLoopOn); Serial.println(sendSerialValues);
break;
case cmdLoopTime: setLoopTime();
Serial.print(lblLoopTime); Serial.println(loopTime);
break;
}
}
void setLoopTime (void) {
boolean endOfDigits = false;
String cmdStr = "";
int inChar;
while (!endOfDigits && Serial.available()) {
inChar = Serial.read();
if (isDigit(inChar)) {
cmdStr += (char)inChar;
} else {
endOfDigits = true;
}
}
if (cmdStr.length() >= 1) { long tmp = cmdStr.toInt();
if (tmp > 0) loopTime = tmp;
}
}
OK, assuming we have defined status, value, input and output definitions as required in PROGMEM and written appropriate functions reportStatusCodes() etc. then we should have a basic framework and an idea of how to handle sketch specific commands. These are shown in the full sample framework code to download.
One final thing, we happen to be a bit concerned to ensure that the main programme loop always takes the same amount of time to execute whatever complex command processing or interrupt handling or value conversion might have been involved during a particular trip round the loop.
In simple terms we will note the time from the built in millisecond timer, millis(), at the start and end of the loop and adjust the delay we are using accordingly.
unsigned long loopStart;
unsigned long loopEnd;
static unsigned long loopTime = 20; int exeTime;
void loop()
{
loopStart = millis();
loopEnd = millis();
exeTime = loopEnd - loopStart;
if (exeTime > 0) { exeTime = 0; }
if (exeTime > loopTime)
delay(loopTime - exeTime);
} else {
}
}
That’s it. We’ve now got a framework for any sketch that will respond to some standard commands and enable us to identify what is running and what its capabilities are.
A final word on tools. Obviously you can write sketch code in any development environment you like, including the Arduino native IDE, which you also need to compile and upload the code to the board.
Since the bulk of the rest of our work is in Windows we make extensive use of Visual Studio and I would recommend the Visual Micro addon to Visual Studio. It encapsulates the Ardunio development tools inside Visual Studio and gives you access to Intellisense and all the other tools you are used to. You do have to have C++ installed in Visual Studio.
In part 2 we will use facilities provided by this framework to build a class library that encapsulates interfacing to the Arduino for a Windows Forms Application, including identifying which virtual serial port the Arduino is connected to and presenting its status information and values in a consistent manner.
Points of Interest
We have defined a standard communication protocol for commands to the Arduino board and returning both status information and dynamic values.
We have used PROGMEM to store string constants out of the way. In a future article we will look at using the EEPROM to enable us to preserve status variables when the power is cycled on the board.
We can interogate the Sketch to ask what pins it is using, what commands it responds to, what status information and values it returns.
We have tried to ensure that the execution time for the main programme loop is consistent whatever processing happens during each trip around the loop. So if we are polling a sensor every 20ms we will get a result returned from the Arduino every 20ms - of course the overhead in the operating system of the control computer may be variable, that is a whole different topic. Neither Windows nor Linux/MacOS are real-time operating systems for the application developer.
If you are producing multiple different Arduino projects with different functionality then using a standard framework for all your sketches can make life a lot easier in the long run. Of course there is an overhead in terms of needing to include the standard code every time, but if you are a competent C programmer you will find how to put all of the standard functions into a library that you can simply #include at the top of each project.
History
First version submitted 26th July 2016, typos corrected and minor adds 27th July
Updated code 1st August to tidy some errors and add comments