Fixed Size Formatter

• Download source code and samples - 519.25 KB
• Download binaries - 460.46 KB

Introduction

Nowadays, applications/systems transfer data in well known data formats such as XML/JSON, etc. However, in the past, systems were using proprietary data formats where a predefined fixed sized raw structure was agreed between the data producer and the consumer and written to a flat file or transferred on the wire.

Surprisingly, not so long ago, I decided to use this same "old fashion" practice when I needed HW devices to communicate with a server on a HTTP transport – with minimum messages size.

I've come up with this framework that helps developers to create and parse that type of data using declaration instead of hard coding achieving aesthetic, maintainable and proven working code within minimum work.

Even though it was planned to be used with small chunks of data - I've done some work to let it work with flat files as well.

Using the Code

It cannot get any easier, all you have to do is:

1. Reference the DLL and add the appropriate using statement.
2. Define a class that represents the raw data (POCO is recommended) and decorate it with FixedSizeContract attribute.
3. Decorate those class properties with the attributes that actually represents the data structure.
4. Call FixedSizeStreamReader or FixedSizeStreamWritter to do the work for you.

/// <summary>
/// This class will be represented with a fixed size string
/// </summary>
[FixedSizeContract]
public class Student
{
    [FixedSizeMember(Order = 0, Length = 20, 
	PaddingMode = PaddingMode.Right, PaddingChar = ' ')]
    public string Name { get; set; }

    [FixedSizeMember(Order = 1, Length = 10, 
	PaddingMode = PaddingMode.Left, PaddingChar = '0')]
    public string ID { get; set; }

    /// <summary>
    /// This property uses the default integer formatter
    /// </summary>
    [NumericFixedSizeMember(Order =2, Length = 3, NumberStyles = NumberStyles.Number)]
    public int Weight { get; set; }

    /// <summary>
    /// This property demonstrate the use of 
    /// a custom formatter. - therefore the use of formattername
    /// </summary>
    [NumericFixedSizeMember(Order = 3, Length = 3, 
	NumberStyles = NumberStyles.Number, FormatterName = "MyCustomContainerName")]
    public int Height { get; set; }

    [BooleanFixedSizeMember(Order = 4, Length = 1, TrueValue = "T", Falsevalue = "F")]
    public bool Good { get; set; }

    [FixedSizeMember(Order = 5, Length = 8, Format = "yyyyMMdd")]
    public DateTime BirthDate { get; set; }

    /// <summary>
    /// This property demonstrates the use of a CustomFixedSizeMemberAttribute
    /// </summary>
    [SpecialNumber(Order = 6, Length = 3,AddtionalStuff =123)]
    public int MyProperty { get; set; }
}

private void SaveFictiousData(string path)
{
    Student[] students = new Student[]
    {
        new Student{Name="Udi Peretz",Height=176,ID="12345678",
		BirthDate=new DateTime(1973,10,10),Weight=78},
        new Student{Name="Ziv Bar",Height=172,ID="78948152",
		BirthDate=new DateTime(1948,12,29),Weight=72},
        new Student{Name="Sion Cohen",Height=178,ID="15559875",
		BirthDate=new DateTime(1967,6,5),Weight=80}
    };

    using (FixedSizeStreamWriter<Student> writer = 
	new FixedSizeStreamWriter<Student>(path, false, Encoding.GetEncoding(1255)))
    {
        writer.Write(students);
        writer.Close();
    }
}

private List<Student> ReadFromFile(string path, out double avg)
{
    using (FixedSizeStreamReader<Student> reader = 
	new FixedSizeStreamReader<Student>(path, Encoding.UTF8))
    {
        var result = reader.ReadToEnd().ToList();
        avg = result.Average(x => x.Height);    
        return result;
    }
}

The above code demonstrates how to define the data structure (layout) and how to read and write the objects into a file.

Features

The instructions to the framework of how to format and position the data is provided with the DataMember attributes. Among these instructions are the following features:

Properties Layout

You should always declare the order of a property, otherwise the properties will be positioned in the order in which they appear in the code. payload property will always assumed to be the last.

Composition

In case where an object is composite to another object - the object will be layout entirely as a simple property and afterwards the remaining properties will be formatted.

Inheritance

Base class properties will always be positioned before the subclass.

Length & Padding

On each property, you will set its length. You can define the padding behavior, if required, and even let the padding char be inherited from the base FixedSizeContract class.

Payload data

You may allow the last data field (property) to use none-fixed length. The fw allows you to mark a single property as payload - and this property will not use padding nor any length restrictions.

Header

You are able to define a different type for the first raw in the file or even set the header raw in a separate file.

LINQ

Since that all data is represented as CLR objects, use any container you want and query it.

Performance Issues

None. Even though, heavily using reflection, I've made efforts to use it only once at the graph/scheme construction stage - so no performance issue has been introduced.

Extensibility

The framework enables several extensibility points.

First, you could add your own ICustomFormatter and register its types as well. In that case, whenever a specific type is called. However, you could use several formatters for the same type by naming.

You can also create your own DataMember attributes by inheriting from CustomFixedSizedMemberAttribute so that way you will be able to provide additional characteristics to your data structure.

<configuration>
    <configSections>
        <section name="FixedSizeFormatting" 
		type ="SC.Utilities.FixedSizeFormatting.Configuration.FSConfiguration,
		SC.Utilities.FixedSizeFormatting"/>
    </configSections>
    <FixedSizeFormatting>
    <Containers>
        <Container Name="MyCustomContainerName">
            <Formatters>
                <Formatter TypeFullName="System.Int32" 
		FormatWith="JerusalemSchool.SpecialInt32Formatter,JerusalemSchool" 
		DefaultFormatString="test"/>
            </Formatters>
        </Container>
    </Containers>
    </FixedSizeFormatting>
</configuration>

The framework's custom configuration section enables you to define a formatter of your own that will be called for each property marked with FormatterName:

[FixedSizeMember(Order = 0, Length = 5, 
	PaddingMode = PaddingMode.Right, PaddingChar = ' ' , 
	FormatterName="MyCustomContainerName")]
public int IntValue { get; set; }

When time enables, I hope that I will post an article regarding "Extending The Fixed Sized Formatting Framework" - but for now, trust me, it's all there!

Limitations & Known Issues

The framework does not support cyclic referencing and an object cannot hold reference to its own type. Use simple objects!

What's Next

• Add the ability to declare the framework's behavior outside the code based on configuration alone (config & XML map file).
• Create a Serializer based on the FW.
• Extend WCF with the appropriate channel dispatcher to use fixed size formatting instead of XML - I guess it will copy the way HTTP REST is implemented over WCF.

History

• 10^th May, 2010: Initial post