Introduction
A message is a simple data structure, comprising typed fields. Fields can be primitive types like int, uint, float, double, byte, byte array, etc. Also a message can include arbitrarily nested messages. I wrote a simple message structure library for my needs. My problem was building message structures for a device simulator, what I tried to achieve is creating predefined message structs, filling in values of fields and sending them to the device as byte array. Then, receiving data from the device as byte array and casting it to a message to process values of fields.
Background
Marshalling is very useful for these kind of operations, it is not unsafe code and it is really fast. A sample conversion could be done like this:
byte[] GetBytes(Message msg)
{
int size = Marshal.SizeOf(msg);
byte[] arr = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(msg, ptr, true);
Marshal.Copy(ptr, arr, 0, size);
Marshal.FreeHGlobal(ptr);
return arr;
}
Message GetMessage(byte[] arr)
{
Message msg = new Message();
int size = Marshal.SizeOf(msg);
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.Copy(arr, 0, ptr, size);
msg = (Message)Marshal.PtrToStructure(ptr, msg.GetType());
Marshal.FreeHGlobal(ptr);
return msg;
}
Why can't I use this?
Because my fields are not always byte aligned. I mean, I can have a field with only 3 bits. I want to illustrate it with a figure:
As shown above, Field1 is 3 bits, then 2 bits reserved, Field2 is 3 bits, Field3 is 4 bits and Field4 is 4 bits.
My sample message is 16 bits long and I'm waiting a byte array with length two. I need to define all my fields as byte and it sums up total 4 bytes (without reserved field). And I can't use marshalling because my struct is 4 bytes and incoming data is 2 bytes. So, I need to parse data and allocate my fields manually.
Structure
The structure of a message is as follows:
Message Structure
Header message can consist of a "Message ID" and "Message Length" fields.
Header fields are used for identification, we can check its length and type to identify incoming data.
Footer message contains a "Checksum" field.
Generally, a message has a checksum field which is located in last byte(s) of message. It is an error-detecting code commonly used in digital networks and storage devices to detect accidental changes to raw data. Blocks of data entering these systems get a short check value attached, based on the remainder of a polynomial division of their contents; on retrieval the calculation is repeated, and corrective action can be taken against presumed data corruption if the check values do not match.
And here's the class diagram of the message structure library:
IMessage is an interface which must be implemented by Message class.
public interface IMessage
{
List<Field> Fields { get; }
byte[] GetMessage();
void SetMessage(byte[] data);
}
Field is a class which holds all required data of a message field.
public class Field
{
#region ctor
public Field(string name, int length)
{
this.Name = name;
this.Length = length;
this.Resolution = 1;
this.DataType = DataTypes.BYTE;
var byteCount = (int)Math.Ceiling(length / 8f);
this.Data = new byte[byteCount];
}
public Field(string name, int length, DataTypes type)
: this(name, length)
{
this.DataType = type;
}
public Field(string name, int length, DataTypes type, int resolution)
: this(name, length, type)
{
this.Resolution = resolution;
}
public Field(int length, DataTypes type, int resolution)
: this(string.Empty, length, type, resolution)
{
this.Resolution = resolution;
}
public Field(int length, DataTypes type)
: this(string.Empty, length, type)
{
}
#endregion
#region Properties
public Action ValueUpdated { get; set; }
public string Name { get; set; }
public int Length { get; set; }
public double Resolution { get; set; }
public byte[] Data { get; set; }
public DataTypes DataType { get; set; }
#endregion
#region Public Methods
public void SetValue(object value)
{
switch (this.DataType)
{
case DataTypes.INT:
this.SetValue((int)value);
break;
case DataTypes.UINT:
this.SetValue((uint)value);
break;
case DataTypes.SHORT:
this.SetValue((short)value);
break;
case DataTypes.USHORT:
this.SetValue((ushort)value);
break;
case DataTypes.FLOAT:
this.SetValue((float)value);
break;
case DataTypes.DOUBLE:
this.SetValue((double)value);
break;
case DataTypes.BYTE:
this.SetValue((byte)value);
break;
case DataTypes.BYTEARRAY:
this.SetValue((byte[])value);
break;
default:
throw new Exception("Hata");
}
}
public void SetValue<T>(T value)
{
var len = (int)Math.Ceiling(this.Length / 8f);
if (typeof(T) == typeof(int))
{
var intvalue = (int)(Convert.ToInt32(value) / this.Resolution);
this.Data = BitConverter.GetBytes(intvalue).Reverse().ToArray().PadTrimArray(len);
}
else if (typeof(T) == typeof(uint))
{
var uintvalue = (uint)(Convert.ToUInt32(value) / this.Resolution);
this.Data = BitConverter.GetBytes(uintvalue).Reverse().ToArray().PadTrimArray(len);
}
else if (typeof(T) == typeof(short))
{
var shortvalue = (short)(Convert.ToInt16(value) / this.Resolution);
this.Data = BitConverter.GetBytes(shortvalue).Reverse().ToArray();
}
else if (typeof(T) == typeof(ushort))
{
var ushortvalue = (ushort)(Convert.ToUInt16(value) / this.Resolution);
this.Data = BitConverter.GetBytes(ushortvalue).Reverse().ToArray().PadTrimArray(len);
}
else if (typeof(T) == typeof(float))
{
var floatvalue = (float)(Convert.ToSingle(value) / this.Resolution);
this.Data = BitConverter.GetBytes(floatvalue).PadTrimArray(len).Reverse().ToArray();
}
else if (typeof(T) == typeof(double))
{
double doublevalue = Convert.ToDouble(value) / this.Resolution;
this.Data = BitConverter.GetBytes(doublevalue).Reverse().ToArray().PadTrimArray(len);
}
else if (typeof(T) == typeof(byte))
{
var bytevalue = (byte)(Convert.ToByte(value) / this.Resolution);
this.Data = BitConverter.GetBytes(bytevalue).Reverse().ToArray().PadTrimArray(len);
}
else if (typeof(T) == typeof(char))
{
var charvalue = (char)Convert.ToChar(value);
this.Data = BitConverter.GetBytes(charvalue).Reverse().ToArray().PadTrimArray(len);
}
else if (typeof(T) == typeof(byte[]))
{
this.Data = (byte[])(object)value;
}
else
{
throw new ArgumentException("value", "Invalid type.");
}
if (this.ValueUpdated != null)
{
this.ValueUpdated();
}
}
public string GetValue()
{
return this.GetValue(this.DataType);
}
public T GetValue<T>()
{
if (typeof(T) == typeof(int))
{
var arr = this.Data.PadTrimArray(4);
var value = (int)(BitConverter.ToInt32(arr.Reverse().ToArray(), 0) * this.Resolution);
return (T)Convert.ChangeType(value, typeof(T));
}
if (typeof(T) == typeof(uint))
{
var arr = this.Data.PadTrimArray(4);
var value = (uint)(BitConverter.ToUInt32(arr.Reverse().ToArray(), 0) * this.Resolution);
return (T)Convert.ChangeType(value, typeof(T));
}
if (typeof(T) == typeof(short))
{
var arr = this.Data.PadTrimArray(2);
var value = (short)(BitConverter.ToInt16(arr.Reverse().ToArray(), 0) * this.Resolution);
return (T)Convert.ChangeType(value, typeof(T));
}
if (typeof(T) == typeof(ushort))
{
var arr = this.Data.PadTrimArray(2);
var value = (ushort)(BitConverter.ToUInt16(arr.Reverse().ToArray(), 0) * this.Resolution);
return (T)Convert.ChangeType(value, typeof(T));
}
if (typeof(T) == typeof(float))
{
var arr = this.Data.PadTrimArray(4);
var value = (float)(BitConverter.ToSingle(arr.Reverse().ToArray(), 0) * this.Resolution);
return (T)Convert.ChangeType(value, typeof(T));
}
if (typeof(T) == typeof(double))
{
var arr = this.Data.PadTrimArray(4);
var value = BitConverter.ToDouble(arr.Reverse().ToArray(), 0) * this.Resolution;
return (T)Convert.ChangeType(value, typeof(T));
}
if (typeof(T) == typeof(byte))
{
var value = (byte)(this.Data[0] * this.Resolution);
return (T)Convert.ChangeType(value, typeof(T));
}
if (typeof(T) == typeof(char))
{
var value = (char)this.Data[0];
return (T)Convert.ChangeType(value, typeof(T));
}
if (typeof(T) == typeof(byte[]))
{
return (T)Convert.ChangeType(this.Data, typeof(T));
}
return default(T);
}
public override string ToString()
{
return this.Name;
}
#endregion
#region Private Methods
private string GetValue(DataTypes dataType)
{
switch (dataType)
{
case DataTypes.INT:
return this.GetValue<int>().ToString();
case DataTypes.UINT:
return this.GetValue<uint>().ToString();
case DataTypes.SHORT:
return this.GetValue<short>().ToString();
case DataTypes.USHORT:
return this.GetValue<ushort>().ToString();
case DataTypes.FLOAT:
return this.GetValue<float>().ToString();
case DataTypes.DOUBLE:
return this.GetValue<double>().ToString();
case DataTypes.BYTE:
return this.GetValue<byte>().ToString();
case DataTypes.CHAR:
return this.GetValue<char>().ToString();
case DataTypes.BYTEARRAY:
return BitConverter.ToString(this.GetValue<byte[]>());
default:
return null;
}
}
#endregion
}
Message class holds all fields, header and footer messages.
public abstract class Message : IMessage
{
#region Fields
protected Field CheckSumField;
private List<Field> fields;
private List<Field> fieldsWoChecksum;
#endregion
#region Properties
public Message HeaderMessage { get; set; }
public Message FooterMessage { get; set; }
public bool IsChecksumExists { get; set; }
public virtual List<Field> Fields
{
get
{
if (this.fields == null)
{
this.GatherFields();
return this.fields;
}
return this.fields;
}
}
#endregion
#region Public Methods
public virtual void SetMessage(byte[] data)
{
this.Fill(data);
}
public virtual byte[] GetMessage()
{
this.ReCalculateCheckSum();
return this.ToByteArray();
}
public override string ToString()
{
if (this.fields == null)
{
this.GatherFields();
}
try
{
return string.Join(
Environment.NewLine,
this.fields.Select(p => p.Name +
":\t" + (p.GetValue() == "\0" ?
"Null Character" : p.GetValue())));
}
catch
{
return string.Empty;
}
}
protected void InitFields()
{
this.GatherFields();
var fieldProperties =
this.GetType()
.GetProperties()
.Where(p => p.PropertyType == typeof (Field))
.ToList();
foreach (var a in fieldProperties)
{
object o = a.GetValue(this, null);
if (o == null)
{
continue;
}
var f = o as Field;
if (f != null && f.Name == string.Empty)
{
f.Name = Utils.SplitCamelCase(a.Name);
}
}
}
protected virtual byte GetCheckSum()
{
return Utils.CalculateChecksum(this.Fields.ToByteArray());
}
protected void ReCalculateCheckSum()
{
if (this.fields == null)
{
this.GatherFields();
}
if (this.IsChecksumExists && this.CheckSumField != null)
{
this.CheckSumField.SetValue
(Utils.CalculateChecksum(this.fieldsWoChecksum.ToByteArray()));
}
}
#endregion
#region Private Methods
private void GatherFields()
{
var fieldProperties = this.GetType().GetProperties().ToList();
this.fields = new List<Field>();
foreach (var a in fieldProperties)
{
object o = a.GetValue(this, null);
if (o == null)
{
continue;
}
if (o is Field)
{
fields.Add(o as Field);
}
else if (o is IEnumerable<Message>)
{
var y = o as IEnumerable<Message>;
fields.AddRange(y.SelectMany(p => p.Fields));
}
}
if (this.HeaderMessage != null)
{
this.fields.InsertRange(0, this.HeaderMessage.Fields);
}
if (this.FooterMessage != null)
{
this.fields.AddRange(this.FooterMessage.Fields);
}
if (this.IsChecksumExists)
{
this.CheckSumField = this.fields.Last();
if (this.fieldsWoChecksum == null)
{
this.fieldsWoChecksum = this.Fields.Except
(new List<Field> { this.CheckSumField }).ToList();
}
this.fieldsWoChecksum.ForEach
(p => p.ValueUpdated = this.ReCalculateCheckSum);
}
}
#endregion
}
Using the Code
I prepared a sample project to use this library, I defined some Rx (incoming) and Tx (outgoing) messages with arbitrary fields. And wrote some methods to parse incoming data for identification.
A sample Rx message, WrapAcknowledge:
public class WrapAcknowledge : Message
{
public Field Version { get; set; }
public Field OutgoingData { get; set; }
public WrapAcknowledge()
{
this.IsChecksumExists = true;
this.HeaderMessage = new MyHeader(0x01, 0x01, 0x0F);
this.FooterMessage = new MyFooter();
this.Version = new Field(16, DataTypes.USHORT);
this.OutgoingData = new Field(64, DataTypes.BYTEARRAY);
this.InitFields();
}
}
We need to tell class if we have checksum field and define our header and footer messages if exist. And we need to call InitFields base method to let message gather all own fields by reflection. All fields have a name property to print them out readable, and if we give declaration name to field properties clearly and camelcase, we don't need to worry about defining names for fields. We can parse them again with reflection.
A sample Tx message, WrapAround:
public enum WrapAroundType
{
Type1 = 0,
Type2
};
public class WrapAround : Message
{
public Field Version { get; set; }
public Field OutgoingData { get; set; }
public WrapAround()
{
this.IsChecksumExists = true;
this.HeaderMessage = new MyHeader(0x01, 0x01, 0x0F);
this.FooterMessage = new MyFooter();
this.Version = new Field(16, DataTypes.USHORT);
this.OutgoingData = new Field(64, DataTypes.BYTEARRAY);
this.InitFields();
}
public WrapAround(WrapAroundType type)
: this()
{
if (type == WrapAroundType.Type1)
{
this.OutgoingData.SetValue(new byte[]
{ 0x51, 0x45, 0xA0, 0x11, 0x00, 0x00, 0xFF, 0xFF });
}
else if (type == WrapAroundType.Type2)
{
this.OutgoingData.SetValue(new byte[]
{ 0x3A, 0xBA, 0x02, 0x0F, 0x34, 0xA5, 0xF0, 0xF0 });
}
}
}
A more complicated message, LWRThreats:
public class LWRThreats : Message
{
public Field Foo { get; set; }
public List<Threat> Treats{ get; set; }
public LWRThreats()
{
this.IsChecksumExists = true;
this.HeaderMessage = new MyHeader(0x01, 0x05, 0x36);
this.FooterMessage = new MyFooter();
this.Foo = new Field(8, DataTypes.BYTE);
this.Treats = new List<Threat>().SetCapacity(3);
this.InitFields();
}
}
This message has sequentially:
- A Header Message (3 Fields)
- Its own field Foo (1 Field)
- List of Threats of Message with length 3 (Each threat message has 8 fields, total 24 Fields)
- A Footer Message (1 Field)
So this message has a total of 29 fields with this sequence.
Rx and Tx messages are defined exactly the same, so we can use a message as Rx and Tx message at the same time.
And here's the method to identify and message cast incoming data:
public static Message DetermineMessage(byte[] data)
{
var header = new MyHeader();
header.SetMessage(data);
return ParseWithHeader(header, data);
}
private static Message ParseWithHeader(MyHeader header, byte[] data)
{
var len = header.MessageLength.GetValue<ushort>();
if (len != data.Length)
{
return null;
}
if (Utils.CalculateChecksum(data) != 0x00)
{
return null;
}
var tip = header.MessageType.GetValue<byte>();
switch (tip)
{
case 0x01:
Message mesaj = new WrapAcknowledge();
mesaj.SetMessage(data);
return mesaj;
case 0x03:
mesaj = new SystemStatus();
mesaj.SetMessage(data);
return mesaj;
case 0x05:
mesaj = new LWRThreats();
mesaj.SetMessage(data);
return mesaj;
default:
return null;
}
}
First, I'm casting incoming data to header message, by this way I can retrieve its identification values. If value of length field is not equal to incoming data length, we need to eleminate this data.
Second check is checksum, we need to calculate checksum of message and check if it is equal to zero or not. If it fails, we need to eleminate this data.
And last check is finding available message type with a switch case like jump flow by using message type field of header. If there is no defined message with this message type value, we need to eleminate this data. Else, we can cast data to message and return it to caller.
Screenshots from GUI
In the first situation, I'm sending WrapAround message and receiving WrapAcknowledge message.
In the second situation, I'm converting a predefined byte array data to message. Parser identifies it and casts it to message which is LWRThreats message.
By my simple benchmark results; with an incoming data 100 byte array long, it can parse and cast message in less than 1 ms.
Points of Interest
I know using reflection and generics in this kind of situations is not healthy, but I used them to ease work of coder by providing a flexible structure and let them come to grips with their own challenging algorithmic problems. I will be glad if you fix my code and guide me for better techniques.
Hope you like it, thank you for reading!
History
- 01.01.2015 - Initial version
