DevCom Framework for Hardware Device Communications

• Download source - 107.63 KB

Introduction

This is a generic extendible framework that covers most of the communication aspects of hardware device control. It abstracts away the formatting parsing and transmitting of data to and from the device into a simple to use object model. There is no more need to rewrite all protocol and technology related stuff.

Background

Many hardware devices (mainly the controllers used) still use serial communications for their connections to the outside world. Serial communications are cheap and reliable. With the bridging ICs readily available to bridge to USB, Ethernet and the likes, there is also little need to change (at least for low bandwidth applications). Using the ICs it's easy to create multiple versions of the same device with different connections. When building applications to control this hardware remotely, you need to implement its hardware control protocol. But you also need to support each of different connection methods.

I have written some drivers for Girder (a homeautomation application) in the Lua scripting language. Girder had an abstraction layer that took out all the connection specific details. As I'm now moving over to xPL for my home, I'm developing in Visual Basic these days. After having reinvented the same thing over and over again, building a similar extensible abstraction layer seemed like a good idea. It should be able to handle communications over several technologies (RS232, TCP/IP, etc.) and some basic message protocols. The device specific control protocols are usually based upon the same type of message protocols, fixed length data, data with a prefixed length byte or data followed by a terminator sequence, are the most common. These should also be supported and easily extendible.

The Model Explained

The main construct is the Transport object. It uses 4 main interfaces:

1. IFormatter
2. ITransportLayer (and its accompanying ISettings)
3. IParser
4. IMessage

Below is a picture of the model and how it relates to the host application and the device being handled and a short description of each component.

IFormatter

Formats an object into an IMessage that can be transmitted to the device by the ITransportLayer. The available formatters are:

• none: just passes bytes without formatting
• terminated: adds a terminator sequence to the data
• fixed length: fills the remainder with a fill-byte when appropriate
• length byte: prefixes the command with a length byte

ITransportLayer

Handles the actual communications. It transmits the command over the wire (or wireless) to the device and listens for incoming data to be delivered to the IParser. For each technology, there should be a specific transportlayer. Currently only TCP/IP is available (and an Echo transportlayer for test purposes), RS232 is up next. The transportlayer must always be accompanied by an ISettings object that is capable of configuring the transportlayer and contains a user interface.

The transport uses its own queues for sending and receiving which are handled on separate threads. This simpifies the transportlayer development as it can use blocking calls (sync) instead of more complex threaded calls (async), without blocking the execution of the parent application.

IParser

The parser does exactly the opposite of the IFormatter, and has the same standard parsers available. So any bytes received from the remote device (through the transportlayer) will be parsed into a valid response. The response can be recombined with the original command send, so the resulting data the Transport delivers to the host application includes the original command and its response (of course, if data is received without a previous command, only response data will be available).

IMessage

Contains the data to be sent and will hold the response received. It also includes handlers to accomodate for additional functionalities. By creating separate objects for different commands, responses can be parsed into command specific properties that can be easily handled by the host application.

Using the Code: An Example

The first working (still rough and buggy) version is available. Let's walkthrough an example with some code.

Suppose we have the following situation; an xPL HAL server using the xHCP protocol. It's a logic engine for an xPL based homeautomation setup. The protocol is text based and has some of the following characteristics:

• Communications is TCP on port 3865
• Messages are terminated by 2 bytes; hex 0D 0A
• Commands are a single commandword, followed by optional parameters
• Responses are preceded by a 3 digit return code and followed by a textual description
• Multiline messages are terminated by '.' on a line of its own

Let's implement a basic transport and have it execute a SETGLOBAL command on the xPL-HAL server. This will set a global variable to a specific value in its scripting engine.

Using a small Windows Forms application to set the variable and its value, I'll demonstrate the library.

Step 1: Build and Configure the Transport

Private WithEvents tsp As Transport

Private Sub MainForm_Load(ByVal sender As System.Object, _
	ByVal e As System.EventArgs) Handles MyBase.Load
    Dim ts As TCPIPSettings
    Dim es As EchoSettings

    ' create transport object
    tsp = New Transport

    ' Create and add the TCP/IP connection
    ts = New TCPIPSettings(tsp)
    ts.Host = "testsystem.local.lan"
    ts.PortNum = 3865
    tsp.SettingsAdd(ts)

    ' Make TCP/IP settings the active set
    tsp.ActiveSettings(ts)

    ' Create and add the Echo layer
    es = New EchoSettings(tsp)
    tsp.SettingsAdd(es)

    ' Ask user for connection preferences
    tsp.SettingsShow()

    ' Set message defaults, terminated by hex 0D 0A
    Dim arrLineTerminator As Byte() = {13, 10}
    tsp.SetFormatter(New FormatterTerminated(arrLineTerminator))
    tsp.SetParser(New ParserTerminated(arrLineTerminator))

    ' start the show
    tsp.Initialize()
    Try
        tsp.Open()
    Catch ex As Exception
        MsgBox("Error while opening the transport: " & ex.Message)
        Me.Close()
    End Try

End Sub

The code creates a Transport in the tsp variable. It then adds 2 settings objects; TCP/IP and Echo (Echo is only added for demonstration purposes, RS232 would have made more sense but isn't available yet). Both are added to the transport by calling its AddSettings method. The active settings to be used is set as well as the default settings for the TCP/IP (setting hostname and portnumber).

The next thing is showing the connection configuration dialog, which allows the user to configure the connection to be used.

The configuration dialog shows the available connections in a dropdown list where the user can pick the connection to be used. The protocol settings pane is delivered as a UserControl through the settingsobject. Once the user picks a connection method in this dialog, the host application has no need to know what connection is actually used by the transport.

The last step in the setup process is to add the Formatter and Parser to be used by the Transport. In the case of an xPL-HAL server, it's a Terminated protocol, by hex 0D 0A. So that's how they are set up.

After adding Parser and Formatter, the transport is initialized and then opened. When the transport opens the connection, it calls the ActiveSettings object and requests a TransportLayer (the settings object creates and configures the TransportLayer and hands it over to the Transport).

In short:

1. Add each layer/connection supported by the device to the transport settings
2. Show the configuration dialog to the user
3. Add Formatter and Parser
4. Open the connection

Step 2: Receive a Message

When the form shows, the connection has been opened by the transport and the xPL-HAL server responds with a status message listing its xPL address and version information. Here's the initial window:

The code handling the incoming data is a Transport eventhandler;

Private Sub DataReceived(ByVal sender As Transport, ByVal data As IMessage) _
	Handles tsp.DataReceived
    ' Eventhandler for received responses
    Dim d As SetResponseDelegate
    ' cast returned data to byte array and convert to a string
    Dim resp As String = Utility.encBytes2String(CType(data.ResponseData, Byte()))
    ' Invoke method to display the async returned response
    d = AddressOf SetResponse
    If Me.InvokeRequired Then
        Me.Invoke(d, resp)
    Else
        SetResponse(resp)
    End If
End Sub
Delegate Sub SetResponseDelegate(ByVal response As String)
Private Sub SetResponse(ByVal response As String)
    tbResponse.Text = response
End Sub

The data received through the IMessage.ResponseData property is formatted as a string and then displayed in the tbReponse textbox (through the delegate as the incoming data is on a different thread).

Step 3: Send a Command

Sending data is just as straightforward. The example command SETGLOBAL has two parameters, the name and the value, separated by a space. To handle the command when the user clicks send:

Private Sub btnSend_Click(ByVal sender As System.Object, _
	ByVal e As System.EventArgs) Handles btnSend.Click
    Dim t As String
    ' Setup command string
    t = "SETGLOBAL " & tbVarName.Text & " " & tbValue.Text
    ' Convert to byte array and send
    tsp.Send(Utility.encString2Bytes(t))
End Sub

Create the command string, convert it to a byte array and call Transport.Send. If you click the Send button in the example, the response will immediately be displayed:

(The 2xx response codes indicate success.)

That's how easy the transport implements communications. Remember that we haven't seen a socket so far nor any COM port or threadpools and queue-handlers. All taken care off by the Transport. But when looking at the handling of the command and the response, we send a command and just wait for a response and then we have to match it, besides that we're still getting a generic byte array to digest. This is something that can also be handled by the transport. Let's have a more advanced example...

Another Example: Full Cycle Command-response

Whenever transmitting a command, it would be most convenient if the response to the command was linked upon returning. To do this, we can create a specific IMessage implementation for our command and have it rework the responses received. For the example, I'll introduce 4 properties; globalvariable name, its value, the response code, the response message. Here's the command implementation (the base class BasicMessage is a simple IMessage implementation included in the lib);

Friend Class cmdSetGlobal
    Inherits BasicMessage

    Public GlobalName As String = ""            ' name of the global variable to be set
    Public GlobalValue As String = ""           ' value to be set
    Public ReturnCode As Integer                ' xPL-HAL response code
    Public ReturnMessage As String              ' xPL-HAL response message
    Public Const SuccesCode As Integer = 232    ' ReturnCode that indicates success

    Public Sub New(Optional ByVal strGlobalName As String = "", _
		Optional ByVal strGlobalValue As String = "")
        Me.Name = "SetGlobal"
        Me.Description = "Set a value in global variable of the xPL-HAL scripting engine"
        Me.GlobalName = strGlobalName
        Me.GlobalValue = strGlobalValue
        Me.RequiresResponse = True

        Dim arrLineTerminator As Byte() = {13, 10}
        _parser = New ParserTerminated(arrLineTerminator)
    End Sub

    Private _parser As ParserTerminated
    Public Overrides Function GetParser(ByVal sender As Tieske.DevCom.Transport) _
	As Tieske.DevCom.IParser
        Return _parser
    End Function

    Public Overrides Property MessageData() As Object
        Get
            ' return data constructed from the command and the properties values as set
            Return Utility.encString2Bytes("SETGLOBAL " & Me.GlobalName & _
		" " & Me.GlobalValue)
        End Get
        Set(ByVal value As Object)
            ' do nothing
        End Set
    End Property
    Public Overrides Sub OnResponseReceived_
	(ByVal sender As Tieske.DevCom.Transport, ByVal data As Object)
        MyBase.OnResponseReceived(sender, data)
        ' Method runs when a received response has been successfully parsed
        ' Extract the values from the response received
        Dim msg As String = Utility.encBytes2String(CType(data, Byte()))
        Me.ReturnCode = CInt(Val(Microsoft.VisualBasic.Left(msg, 4)))
        Me.ReturnMessage = Mid(msg, 5)
    End Sub
End Class

Let's skip the self explanatory part and focus on the interesting details.

The New constructor is straight forward, but it has two important lines:

1. 'Me.RequiresResponse = True' tells the Transport that after sending the command, it cannot let go of the IMessage object, but it must retain it in a queue awaiting a proper response.
2. '_parser = New ParserTerminated(arrLineTerminator)' sets the specific Parser to be used for this message. In this case, its the same as the main parser (set in the Transport object itself), but it doesn't need to be.

When receiving data, the receiving queue is (in turn) handed to all messages still waiting for a response, on a first-in-first-out bases (the one longest in the queue will get the data first). If all parsers fail, the incoming data is passed to the generic Parser of the Transport.

The IMessage.MessageData property is the place where the command data is to be stored, so in this property the actual command is constructed based upon the earlier introduced properties. In a similar manner, the IMessage.OnDataReceived method is called when data has been successfully parsed for this message, so here we extract the data of the response properties from the returned byte array.

Now that a specific command object has been created, sending the command is simplified to only creating a new object and sending it;

Private Sub btnSend_Click(ByVal sender As System.Object, _
	ByVal e As System.EventArgs) Handles btnSend.Click
    ' Create a command class and send it
    tsp.Send(New cmdSetGlobal(tbVarName.Text, tbValue.Text))
End Sub

In the receiving part, we can now simply check for the object type (remember that the object received is the same instance as the one send, in the previous example a new object would have been returned). Only if it's a response to the command send, it will be handled. The generic welcome message by the xPL-HAL server will not pass here and hence it will not be displayed like it was in the first example. Here's the eventhandler for receiving:

Private Sub DataReceived(ByVal sender As Transport, _
	ByVal data As IMessage) Handles tsp.DataReceived
    ' Eventhandler for received responses
    Dim d As SetResponseDelegate

    ' If received data is not a response to the cmdSetGlobal, then exit
    If Not TypeOf data Is cmdSetGlobal Then Exit Sub

    ' Invoke method to display the async returned response
    d = AddressOf SetResponse
    If Me.InvokeRequired Then
        Me.Invoke(d, CType(data, cmdSetGlobal))
    Else
        SetResponse(CType(data, cmdSetGlobal))
    End If
End Sub

And the display method (SetResponse) now takes a cmdSetGlobal as a parameter, and sets the display properties based on the specific contents of the custom properties. This is the code:

Private Sub SetResponse(ByVal msg As cmdSetGlobal)
    tbResponse.Text = msg.ReturnMessage
    tbResponseCode.Text = msg.ReturnCode.ToString
    If msg.ReturnCode = cmdSetGlobal.SuccesCode Then
        ' success
        tbResponseCode.BackColor = Color.Green
    Else
        ' failure
        tbResponseCode.BackColor = Color.Red
    End If
End Sub

Which makes it look like this:

Points of Interest

Even with these rather simple examples, the potential is clear. The first example already does a great job in wrapping all communications stuff, no matter what technology you're using. The second example goes beyond that by not just delivering data in some general form, but delivering it tailored to the specific application/device it's being used for.

Because the whole thing is set up from a number of interfaces, it's easy to extend. Add a transportlayer, parser or formatter and they can be reused over and over again.

From Here...

The project is still far from production ready. The sources are available on SourceForge, a project is in the making. Comments and contributions are most welcome.

On the To-do List

• Add RS232 (COM-port) transport layer
• Add keep-alive signals
• Add connection control (auto restart connection if it fails)
• Strengthen exception handling (currently very weak)

History

• 12-Nov-2010 Initial article for the hardware competition