Cinchoo ETL - FixedLength Reader

Cinchoo

5.00/5 (3 votes)

27 Sep 2022CPOL29 min read

20.3K

Simple FixedLength file reader for .NET

Introduction
Requirement
"Hello World!" Sample
1. Quick load - Data First Approach
2. Code First Approach
3. Configuration First Approach
4. Code First with declarative configuration
Reading All Records
Read Records Manually
Customize FixedLength Record
Customize FixedLength Header
Customize FixedLength Fields
1. DefaultValue
2. ChoFallbackValue
3. Type Converters
  1. Declarative Approach
  2. Configuration Approach
4. Validations
Callback Mechanism
1. BeginLoad
2. EndLoad
3. BeforeRecordLoad
4. AfterRecordLoad
5. RecordLoadError
6. BeforeRecordFieldLoad
7. AfterRecordFieldLoad
8. RecordLoadFieldError
Customization
AsDataReader Helper Method
AsDataTable Helper Method
Using Dynamic Object
1. DefaultValue
2. ChoFallbackValue
3. FieldType
4. Type Converters
5. Validations
Working with sealed POCO object
Exceptions
Tips
1. Multiline FixedLength column value
2. FixedLength column value with single quotes
Using MetadataType Annotation
Configuration Choices
1. Manual Configuration
2. Auto Map Configuration
3. Attaching MetadataType class
LoadText Helper Method
Advanced Topics
1. Override Converters Format Specs
2. Currency Support
3. Enum Support
4. Boolean Support
5. DateTime Support
Fluent API
1. WithRecordLength
2. WithFirstLineHeader
3. WithField
4. QuoteAllFields
History

1. Introduction

ChoETL is an open source ETL (extract, transform and load) framework for .NET. It is a code based library for extracting data from multiple sources, transforming, and loading into your very own data warehouse in .NET environment. You can have data in your data warehouse in no time.

This article talks about using FixedLengthReader component offered by ChoETL framework. It is a simple utility class to extract FixedLength data from file / source.

Features

Follows FixedLength standard file rules. Gracefully handles data fields that contains commas and line breaks
Exposes IEnumarable list of objects - which is often used with LINQ query for projection, aggregation and filtration, etc.
Supports deferred reading
Supports processing files with culture specific date, currency and number formats.
Supports different character encoding
Recognizes a wide variety of date, currency, enum, boolean and number formats when reading files
Provides fine control of date, currency, enum, boolean, number formats when writing files
Detailed and robust error handling, allowing you to quickly find and fix problems
Shorten your development time

2. Requirement

This framework library is written in C# using .NET 4.5 Framework.

3. "Hello World!" Sample

Open VS.NET 2013 or higher.
Create a sample VS.NET (.NET Framework 4.5) Console Application project.
Install ChoETL via Package Manager Console using Nuget Command: Install-Package ChoETL
Use the ChoETL namespace.

Let's begin by looking into a simple example of reading FixedLength file with record length of 18 chars, having 2 columns (Id: 8 chars long, Name: 10 chars long)

Listing 3.1 Sample FixedLength Data File (RecordLength: 18 chars)

Id      Name      
1       Carl      
2       Mark

There are number of ways in which you can get the FixedLength file parsing started with minimal setup.

3.1. Quick Load - Data First Approach

It is the quick way to load a FixedLength file in no time. No POCO object is required. Sample code below shows how to load the file.

Listing 3.1.1 Load FixedLength File using Iterator

static void QuickDynamicLoadTestUsingIterator()
{
    using (var stream = new MemoryStream())
    using (var reader = new StreamReader(stream))
    using (var writer = new StreamWriter(stream))
    {
        writer.WriteLine("Id      Name      ");
        writer.WriteLine("1       Carl      ");
        writer.WriteLine("2       Mark      ");
        writer.Flush();
        stream.Position = 0;
 
        foreach (dynamic row in new ChoFixedLengthReader(reader).WithFirstLineHeader())
        {
            Console.WriteLine(String.Format("Id: {0}", row.Id));
            Console.WriteLine(String.Format("Name: {0}", row.Name));
        }
    }
}

Listing 3.1.2 Load FixedLength File using Loop

static void QuickDynamicLoadTest()
{
    dynamic row = null;
    using (var stream = new MemoryStream())
    using (var reader = new StreamReader(stream))
    using (var writer = new StreamWriter(stream))
    using (var parser = new ChoFixedLengthReader(reader).WithFirstLineHeader())
    {
        writer.WriteLine("Id      Name      ");
        writer.WriteLine("1       Carl      ");
        writer.WriteLine("2       Mark      ");
        writer.Flush();
        stream.Position = 0;
 
        while ((row = parser.Read()) != null)
        {
            Console.WriteLine(String.Format("Id: {0}", row.Id));
            Console.WriteLine(String.Format("Name: {0}", row.Name));
        }
    }
}

This model uses generic FixedLengthReader object for parsing the file. FixedLengthReader auto discovers the file layout schema from the first row automatically, ONLY if column values of first row are spaced between them. In case the auto discovery leads to yield incorrect file layout schema, you can spell out the schema by yourself via Configuration. Will be discussed down below.

If the input feed does not have header, FixedLengthReader auto name the columns as Column1, Column2 ... in the dynamic object model.

3.2. Code First Approach

This is another way to parse and load FixedLength file using POCO class. First, define a simple data class to match the underlying FixedLength file layout.

Listing 3.2.1 Simple POCO Entity Class

public partial class EmployeeRecSimple
{
    public int Id { get; set; }
    public string Name { get; set; }
}

In the above, the class defines properties matching the sample FixedLength file template.

Listing 3.2.2 Load FixedLength File

static void CodeFirstApproach()
{
    EmployeeRecSimple row = null;
    using (var stream = new MemoryStream())
    using (var reader = new StreamReader(stream))
    using (var writer = new StreamWriter(stream))
    using (var parser = new ChoFixedLengthReader<EmployeeRecSimple>
          (reader).WithFirstLineHeader())
    {
        writer.WriteLine("Id      Name      ");
        writer.WriteLine("1       Carl      ");
        writer.WriteLine("2       Mark      ");
        writer.Flush();
        stream.Position = 0;
 
        while ((row = parser.Read()) != null)
        {
            Console.WriteLine(String.Format("Id: {0}", row.Id));
            Console.WriteLine(String.Format("Name: {0}", row.Name));
        }
    }
}

FixedLengthReader auto discovers the file layout schema from the first row automatically, if column values of header row are spaced between them. In case, if the auto discovery leads to incorrect file layout schema, you can spell out the schema via Configuration.

3.3. Configuration First Approach

In this model, we define the FixedLength configuration with all the necessary parsing parameters along with FixedLength columns matching with the underlying FixedLength file.

Listing 3.3.1 Define FixedLength Configuration

ChoFixedLengthRecordConfiguration config = new ChoFixedLengthRecordConfiguration();
config.FixedLengthRecordFieldConfigurations.Add
       (new ChoFixedLengthRecordFieldConfiguration("Id", 0, 8) 
{ FieldType = typeof(int) });
config.FixedLengthRecordFieldConfigurations.Add
       (new ChoFixedLengthRecordFieldConfiguration("Name", 8, 10) 
{ FieldType = typeof(string) });

In the above, we define the configuration matching the sample FixedLength file template.

Listing 3.3.2 Load FixedLength file without POCO Object

static void ConfigFirstApproachReadAsDynamicRecords()
{
    ChoFixedLengthRecordConfiguration config = new ChoFixedLengthRecordConfiguration();
    config.FixedLengthRecordFieldConfigurations.Add
    (new ChoFixedLengthRecordFieldConfiguration("Id", 0, 8) 
    { FieldType = typeof(int) });
    config.FixedLengthRecordFieldConfigurations.Add
    (new ChoFixedLengthRecordFieldConfiguration("Name", 8, 10) 
    { FieldType = typeof(string) });
 
    dynamic row = null;
    using (var stream = new MemoryStream())
    using (var reader = new StreamReader(stream))
    using (var writer = new StreamWriter(stream))
    using (var parser = new ChoFixedLengthReader(reader, config).WithFirstLineHeader())
    {
        writer.WriteLine("Id      Name      ");
        writer.WriteLine("1       Carl      ");
        writer.WriteLine("2       Mark      ");
        writer.Flush();
        stream.Position = 0;
 
        while ((row = parser.Read()) != null)
        {
            Console.WriteLine(String.Format("Id: {0}", row.Id));
            Console.WriteLine(String.Format("Name: {0}", row.Name));
        }
    }
}

Listing 3.3.3 Load FixedLength file with POCO object

static void ConfigFirstApproachReadAsTypedRecords()
{
    ChoFixedLengthRecordConfiguration config = new ChoFixedLengthRecordConfiguration();
    config.FixedLengthRecordFieldConfigurations.Add
    (new ChoFixedLengthRecordFieldConfiguration("Id", 0, 8) 
                       { FieldType = typeof(int) });
    config.FixedLengthRecordFieldConfigurations.Add
    (new ChoFixedLengthRecordFieldConfiguration("Name", 8, 10) 
                       { FieldType = typeof(string) });
 
    EmployeeRecSimple row = null;
    using (var stream = new MemoryStream())
    using (var reader = new StreamReader(stream))
    using (var writer = new StreamWriter(stream))
    using (var parser = 
    new ChoFixedLengthReader<EmployeeRecSimple>(reader, config).WithFirstLineHeader())
    {
        writer.WriteLine("Id      Name      ");
        writer.WriteLine("1       Carl      ");
        writer.WriteLine("2       Mark      ");
        writer.Flush();
        stream.Position = 0;
 
        while ((row = parser.Read()) != null)
        {
            Console.WriteLine(String.Format("Id: {0}", row.Id));
            Console.WriteLine(String.Format("Name: {0}", row.Name));
        }
    }
}

3.4. Code First with Declarative Configuration

This is the combined approach to define POCO entity class along with FixedLength configuration parameters decorated declaratively.

Listing 3.4.1 Define POCO Object

public partial class EmployeeRec
{
    [ChoFixedLengthRecordField(0, 8)]
    public int Id { get; set; }
    [ChoFixedLengthRecordField(8, 10)]
    public string Name { get; set; }
}

The code above illustrates about defining POCO object to carry the values of each record line in the input file. First thing defines property for each record field with ChoFixedLengthRecordFieldAttribute to qualify for FixedLength record mapping. Each property must specify StartIndex, Size in order to be mapped to FixedLength column. StartIndex is 0 based.

It is very simple and ready to extract FixedLength data in no time.

Listing 3.4.2 Main Method

static void CodeFirstWithDeclarativeApproachRead()
{
    EmployeeRec row = null;
    using (var stream = new MemoryStream())
    using (var reader = new StreamReader(stream))
    using (var writer = new StreamWriter(stream))
    using (var parser = 
           new ChoFixedLengthReader<EmployeeRec>(reader).WithFirstLineHeader())
    {
        writer.WriteLine("Id      Name      ");
        writer.WriteLine("1       Carl      ");
        writer.WriteLine("2       Mark      ");
        writer.Flush();
        stream.Position = 0;
 
        while ((row = parser.Read()) != null)
        {
            Console.WriteLine(String.Format("Id: {0}", row.Id));
            Console.WriteLine(String.Format("Name: {0}", row.Name));
        }
    }
}

We start by creating a new instance of ChoFixedLengthReader object. That's all. All the heavy lifting of parsing and loading FixedLength data stream into the objects is done by the parser under the hood.

By default, FixedLengthReader discovers and uses default configuration parameters while loading FixedLength file. These can be overridable according to your needs. The following sections will give details about each configuration attributes.

4. Reading All Records

It is as easy as setting up POCO object match up with FixedLength file structure, you can read the whole file as enumerable pattern. It is a deferred execution mode, but be extra careful while making any aggregate operation on them. This will load the entire file records into memory.

Listing 4.1 Read FixedLength File

foreach (var e in new ChoFixedLengthReader<EmployeeRec>("Emp.txt"))
{
    Console.WriteLine(String.Format("Id: {0}", e.Id));
    Console.WriteLine(String.Format("Name: {0}", e.Name));
}

or:

Listing 4.2 Read FixedLength File Stream

foreach (var e in new ChoFixedLengthReader<EmployeeRec>(textReader))
{
    Console.WriteLine(String.Format("Id: {0}", e.Id));
    Console.WriteLine(String.Format("Name: {0}", e.Name));
}

This model keeps your code elegant, clean, easy to read and maintain. Also leverages LINQ extension methods to perform grouping, joining, projection, aggregation, etc.

Listing 4.3 Using LINQ

var list = (from o in new ChoFixedLengthReader<EmployeeRec>("Emp.txt")
           where o.Name != null && o.Name.StartsWith("R")
           select o).ToArray();
 
foreach (var e in list)
    Console.WriteLine(e.ToStringEx());

5. Read Records Manually

It is as easy as setting up POCO object match up with FixedLength file structure, you can read the whole file as enumerable pattern

Listing 5.1 Read FixedLength File

var reader = new ChoFixedLengthReader<EmployeeRec>("Emp.txt");
var rec = (object)null;
 
while ((rec = reader.Read()) != null)
{
    Console.WriteLine(String.Format("Id: {0}", rec.Id));
    Console.WriteLine(String.Format("Name: {0}", rec.Name));
}

6. Customize FixedLength Record

Using ChoFixedLengthRecordObjectAttribute, you can customize the POCO entity object declaratively.

Listing 6.1 Customizing POCO Object for Each Record

[ChoFixedLengthRecordObject(18)]
public class EmployeeRec
{
    [ChoFixedLengthRecordField(0, 8, FieldName = "id")]
    public int Id { get; set; }
    [ChoFixedLengthRecordField(8, 10, FieldName ="Name", QuoteField = true)]
    [Required]
    [DefaultValue("ZZZ")]
    public string Name { get; set; }
}

Here are the available attributes to carry out customization of FixedLength load operation on a file.

RecordLength - Optional. If not specified, it will be calculated from all the properties defined in the POCO class.
EOLDelimiter - The value used to separate FixedLength rows. Default is \r\n (NewLine).
CultureName - The culture name (e.g., en-US, en-GB) used to read and write FixedLength data.
IgnoreEmptyLine - A flag to let the reader know if a record should be skipped when reading if it's empty. A record is considered empty if all fields are empty.
Comments - The value used to denote a line that is commented out. Multiple comments can be specified. Must be separated by comma.
QuoteChar - The value used to escape fields that contain a delimiter, quote, or line ending.
QuoteAllFields - N/A for reader.
Encoding - The encoding of the FixedLength file.
HasExcelSeperator - N/A for reader. Reader seamlessly recognizes the Excel separator if specified in the FixedLength file and use them for parsing.
ColumnCountStrict - This flag indicates if an exception should be thrown if reading an expected field is missing.
ColumnOrderStrict - This flag indicates if an exception should be thrown if reading an expected field is in wrong position in the file. This check will be performed only when ColumnCountStrict is true.
BufferSize - The size of the internal buffer that is used when reader is from the StreamReader.
ErrorMode - This flag indicates if an exception should be thrown if reading and an expected field is failed to load. This can be overridden per property. Possible values are:
- IgnoreAndContinue - Ignore the error, record will be skipped and continue with next.
- ReportAndContinue - Report the error to POCO entity if it is of IChoNotifyRecordRead type
- ThrowAndStop - Throw the error and stop the execution
IgnoreFieldValueMode - A flag to let the reader know if a record should be skipped when reading if it's empty / null. This can be overridden per property. Possible values are:
- Null - N/A
- DBNull - N/A
- Empty - skipped if the record value is empty
- WhiteSpace - skipped if the record value contains only whitespaces
ObjectValidationMode - A flag to let the reader know about the type of validation to be performed with record object. Possible values are:
- Off - No object validation performed
- MemberLevel - Validation performed at the time of each FixedLength property gets loaded with value
- ObjectLevel - Validation performed after all the properties are loaded to the POCO object

7. Customize FixedLength Header

If the FixedLength file has header, you can instruct the POCO entity by using ChoFixedLengthFileHeaderAttribute.

Listing 6.1 Customizing POCO object for file header

[ChoFixedLengthFileHeader]
public class EmployeeRec
{
    [ChoFixedLengthRecordField(0, 8, FieldName = "id")]
    public int Id { get; set; }
    [ChoFixedLengthRecordField(8, 10, FieldName ="Name", QuoteField = true)]
    [Required]
    [DefaultValue("ZZZ")]
    public string Name { get; set; }
}

Here are the available members to add some customization to it according to your need.

FillChar - N/A for reader
Justification - N/A for reader
TrimOption - This flag tells the reader to trim whitespace from the beginning and ending of the FixedLength column header when reading. Possible values are Trim, TrimStart, TrimEnd.
Truncate - N/A for reader

8. Customize FixedLength Fields

For each FixedLength column, you can specify the mapping in POCO entity property using ChoFixedLengthRecordFieldAttribute.

Listing 6.1 Customizing POCO Object for FixedLength Columns

[ChoFixedLengthFileHeader]
public class EmployeeRec
{
    [ChoFixedLengthRecordField(0, 8, FieldName = "id")]
    public int Id { get; set; }
    [ChoFixedLengthRecordField(8, 10, FieldName ="Name", QuoteField = true)]
    [Required]
    [DefaultValue("ZZZ")]
    public string Name { get; set; }
}

Here are the available members to add some customization to it for each property:

StartIndex - The zero-based starting character position of a column in the fixed length row.
Size - Number of characters in the column.
FieldName - When mapping by name, you specify the name of the FixedLength column that you want to use for that property. For this to work, the FixedLength file must have a header record. The name you specify must match with the name of the header record.
FillChar - N/A for reader.
FieldValueJustification - N/A for reader.
FieldValueTrimOption - This flag tells the reader to trim whitespace from the beginning and ending of the field value when reading. Possible values are Trim, TrimStart, TrimEnd.
Truncate - N/A for reader.
Size - N/A for reader.
QuoteField - A flag that tells the reader that the FixedLength column value is surrounded by quotes.
ErrorMode - This flag indicates if an exception should be thrown if reading and an expected field failed to load. Possible values are:
- IgnoreAndContinue - Ignore the error and continue to load other properties of the record.
- ReportAndContinue - Report the error to POCO entity if it is of IChoRecord type.
- ThrowAndStop - Throw the error and stop the execution.
IgnoreFieldValueMode - A flag to let the reader know if a record should be skipped when reading if it's empty / null. Possible values are:
- Null - N/A
- DBNull - N/A
- Empty - skipped if the record value is empty
- WhiteSpace - skipped if the record value contains only whitespaces

8.1. DefaultValue

It is the value used and set to the property when the FixedLength value is empty or whitespace (controlled via IgnoreFieldValueMode).

Any POCO entity property can be specified with default value using System.ComponentModel.DefaultValueAttribute.

8.2. ChoFallbackValue

It is the value used and set to the property when the FixedLength value failed to set. Fallback value only set when ErrorMode is either IgnoreAndContinue or ReportAndContinue.

Any POCO entity property can be specified with fallback value using ChoETL.ChoFallbackValueAttribute.

8.3. Type Converters

Most of the primitive types are automatically converted and set them to the properties. If the value of the FixedLength field can't automatically be converted into the type of the property, you can specify a custom / built-in .NET converters to convert the value. These can be either IValueConverter or TypeConverter converters.

There are couple of ways in which you can specify the converters for each field:

Declarative Approach
Configuration Approach

8.3.1. Declarative Approach

This model is applicable to POCO entity object only. If you have POCO class, you can specify the converters to each property to carry out necessary conversion on them. The samples below show the way to do it.

Listing 8.3.1.1 Specifying Type Converters

[ChoFixedLengthFileHeader]
public class EmployeeRec
{
    [ChoFixedLengthRecordField(0, 8)]
    [ChoTypeConverter(typeof(IntConverter))]
    public int Id { get; set; }
    [ChoFixedLengthRecordField(8, 10, QuoteField = true)]
    [Required]
    [DefaultValue("ZZZ")]
    public string Name { get; set; }
}

Listing 8.3.1.2 IntConverter Implementation

public class IntConverter : IValueConverter
{
    public object Convert(object value, Type targetType, 
                          object parameter, CultureInfo culture)
    {
        return value;
    }
 
    public object ConvertBack(object value, Type targetType, 
                              object parameter, CultureInfo culture)
    {
        return value;
    }
}

In the example above, we defined custom IntConverter class. And showed how to use it with 'Id' FixedLength property.

8.3.2. Configuration Approach

This model is applicable to both dynamic and POCO entity object. This gives freedom to attach the converters to each property at runtime. This takes the precedence over the declarative converters on POCO classes.

Listing 8.3.2.2 Specifying TypeConverters

ChoFixedLengthRecordConfiguration config = new ChoFixedLengthRecordConfiguration();
config.FileHeaderConfiguration.HasHeaderRecord = true;
config.ThrowAndStopOnMissingField = false;

ChoFixedLengthRecordFieldConfiguration idConfig = 
        new ChoFixedLengthRecordFieldConfiguration("Id", 0, 8);
idConfig.AddConverter(new IntConverter());
config.FixedLengthRecordFieldConfigurations.Add(idConfig);

config.FixedLengthRecordFieldConfigurations.Add
       (new ChoFixedLengthRecordFieldConfiguration("Name", 8, 10));

In the above, we construct and attach the IntConverter to 'Id' field using AddConverter helper method in ChoFixedLengthRecordFieldConfiguration object.

Likewise, if you want to remove any converter from it, you can use RemoveConverter on ChoFixedLengthRecordFieldConfiguration object.

8.4. Validations

FixedLengthReader leverages both System.ComponentModel.DataAnnotations and Validation Block validation attributes to specify validation rules for individual fields of POCO entity. Refer to the MSDN site for a list of available DataAnnotations validation attributes.

Listing 8.4.1 Using Validation Attributes in POCO Entity

[ChoFixedLengthFileHeader]
[ChoFixedLengthRecordObject(18)]
public partial class EmployeeRec
{
    [ChoFixedLengthRecordField(0, 8, FieldName = "id")]
    [ChoTypeConverter(typeof(IntConverter))]
    [Range(1, int.MaxValue, ErrorMessage = "Id must be > 0.")]
    [ChoFallbackValue(1)]
    public int Id { get; set; }
 
    [ChoFixedLengthRecordField(8, 10, FieldName = "Name")]
    [Required]
    [DefaultValue("ZZZ")]
    [ChoFallbackValue("XXX")]
    public string Name { get; set; }
}

In the example above, used Range validation attribute for Id property. Required validation attribute to Name property. FixedLengthReader performs validation on them during load based on Configuration.ObjectValidationMode is set to ChoObjectValidationMode.MemberLevel or ChoObjectValidationMode.ObjectLevel.

Sometimes, you may want to override the defined declarative validation behaviors that come with POCO class, you can do with Cinchoo ETL via configuration approach. The sample below shows the way to override them.

static void ValidationOverridePOCOTest()
{
    ChoFixedLengthRecordConfiguration config = new ChoFixedLengthRecordConfiguration();
    var idConfig = new ChoFixedLengthRecordFieldConfiguration("Id", 0, 8);
    idConfig.Validators = new ValidationAttribute[] { new RequiredAttribute() };
    config.FixedLengthRecordFieldConfigurations.Add(idConfig);
    config.FixedLengthRecordFieldConfigurations.Add
    (new ChoFixedLengthRecordFieldConfiguration("Name", 8, 10));
    config.FixedLengthRecordFieldConfigurations.Add
    (new ChoFixedLengthRecordFieldConfiguration("Salary", 8, 10) 
                             { FieldType = typeof(ChoCurrency) });
 
    using (var stream = new MemoryStream())
    using (var reader = new StreamReader(stream))
    using (var writer = new StreamWriter(stream))
    using (var parser = 
           new ChoFixedLengthReader<EmployeeRecWithCurrency>(reader, config))
    {
        writer.WriteLine(",Carl,$100000");
        writer.WriteLine("2,Mark,$50000");
        writer.WriteLine("3,Tom,1000");
 
        writer.Flush();
        stream.Position = 0;
 
        object rec;
        while ((rec = parser.Read()) != null)
        {
            Console.WriteLine(String.Format("Id: {0}", rec.Id));
            Console.WriteLine(String.Format("Name: {0}", rec.Name));
            Console.WriteLine(String.Format("Salary: {0}", rec.Salary));
        }
    }
}
 
public class EmployeeRecWithCurrency
{
    public int? Id { get; set; }
    public string Name { get; set; }
    public ChoCurrency Salary { get; set; }
}

In some cases, you may want to take control and perform manual self validation within the POCO entity class. This can be achieved by inheriting POCO object from IChoValidatable interface.

Listing 8.4.2 Manual Validation on POCO Entity

[ChoFixedLengthFileHeader]
[ChoFixedLengthRecordObject(18)]
public partial class EmployeeRec : IChoValidatable
{
    [ChoFixedLengthRecordField(0, 8, FieldName = "id")]
    [ChoTypeConverter(typeof(IntConverter))]
    [Range(1, int.MaxValue, ErrorMessage = "Id must be > 0.")]
    [ChoFallbackValue(1)]
    public int Id { get; set; }
 
    [ChoFixedLengthRecordField(8, 10, FieldName = "Name")]
    [Required]
    [DefaultValue("ZZZ")]
    [ChoFallbackValue("XXX")]
    public string Name { get; set; }
 
    public bool TryValidate
    (object target, ICollection<ValidationResult> validationResults)
    {
        return true;
    }
 
    public bool TryValidateFor(object target, string memberName, 
                               ICollection<ValidationResult> validationResults)
    {
        return true;
    }
 
    public void Validate(object target)
    {
    }
 
    public void ValidateFor(object target, string memberName)
    {
    }
}

The sample above shows how to implement custom self-validation in POCO object.

IChoValidatable interface exposes the below methods:

TryValidate - Validate entire object, return true if all validation passed. Otherwise return false.
Validate - Validate entire object, throw exception if validation is not passed.
TryValidateFor - Validate specific property of the object, return true if all validation passed. Otherwise return false.
ValidateFor - Validate specific property of the object, throw exception if validation is not passed.

9. Callback Mechanism

FixedLengthReader offers industry standard FixedLength parsing out of the box to handle most of the parsing needs. If the parsing is not handling any of the needs, you can use the callback mechanism offered by FixedLengthReader to handle such situations. In order to participate in the callback mechanism, Either POCO entity object or DataAnnotation's MetadataType type object must be inherited by IChoNotifyRecordRead interface.

Tip: Any exceptions raised out of these interface methods will be ignored.

IChoNotifyRecordRead exposes the below methods:

BeginLoad - Invoked at the begin of the FixedLength file load
EndLoad - Invoked at the end of the FixedLength file load
BeforeRecordLoad - Raised before the FixedLength record load
AfterRecordLoad - Raised after FixedLength record load
RecordLoadError - Raised when FixedLength record load errors out
BeforeRecordFieldLoad - Raised before FixedLength column value load
AfterRecordFieldLoad - Raised after FixedLength column value load
RecordFieldLoadError - Raised when FixedLength column value errors out

Listing 9.1 Direct POCO Callback Mechanism Implementation

[ChoFixedLengthFileHeader]
[ChoFixedLengthRecordObject(18)]
public partial class EmployeeRec : IChoNotifyRecordRead
{
    [ChoFixedLengthRecordField(0, 8, FieldName = "id")]
    [ChoTypeConverter(typeof(IntConverter))]
    [Range(1, int.MaxValue, ErrorMessage = "Id must be > 0.")]
    [ChoFallbackValue(1)]
    public int Id { get; set; }
    
    [ChoFixedLengthRecordField(8, 10, FieldName = "Name", QuoteField = true)]
    [Required]
    [DefaultValue("ZZZ")]
    [ChoFallbackValue("XXX")]
    public string Name { get; set; }
 
    public bool AfterRecordFieldLoad(object target, int index, 
                                     string propName, object value)
    {
        throw new NotImplementedException();
    }
 
    public bool AfterRecordLoad(object target, int index, object source)
    {
        throw new NotImplementedException();
    }
 
    public bool BeforeRecordFieldLoad(object target, int index, 
                                      string propName, ref object value)
    {
        throw new NotImplementedException();
    }
 
    public bool BeforeRecordLoad(object target, int index, ref object source)
    {
        throw new NotImplementedException();
    }
 
    public bool BeginLoad(object source)
    {
        throw new NotImplementedException();
    }
 
    public void EndLoad(object source)
    {
        throw new NotImplementedException();
    }
 
    public bool RecordFieldLoadError(object target, int index, 
                string propName, object value, Exception ex)
    {
        throw new NotImplementedException();
    }
 
    public bool RecordLoadError
    (object target, int index, object source, Exception ex)
    {
        throw new NotImplementedException();
    }
}

Listing 9.2 MetaDataType Based Callback Mechanism Implementation

[ChoFixedLengthFileHeader]
[ChoFixedLengthRecordObject(18)]
public class EmployeeRecMeta : IChoNotifyRecordRead
{
    [ChoFixedLengthRecordField(0, 8, FieldName = "id")]
    [ChoTypeConverter(typeof(IntConverter))]
    [Range(1, int.MaxValue, ErrorMessage = "Id must be > 0.")]
    [ChoFallbackValue(1)]
    public int Id { get; set; }

    [ChoFixedLengthRecordField(8, 10, FieldName = "Name", QuoteField = true)]
    [Required]
    [DefaultValue("ZZZ")]
    [ChoFallbackValue("XXX")]
    public string Name { get; set; }
 
    public bool AfterRecordFieldLoad(object target, int index, 
                                     string propName, object value)
    {
        throw new NotImplementedException();
    }
 
    public bool AfterRecordLoad(object target, int index, object source)
    {
        throw new NotImplementedException();
    }
 
    public bool BeforeRecordFieldLoad(object target, int index, 
                                      string propName, ref object value)
    {
        throw new NotImplementedException();
    }
 
    public bool BeforeRecordLoad(object target, int index, ref object source)
    {
        throw new NotImplementedException();
    }
 
    public bool BeginLoad(object source)
    {
        throw new NotImplementedException();
    }
 
    public void EndLoad(object source)
    {
        throw new NotImplementedException();
    }
 
    public bool RecordFieldLoadError(object target, int index, 
                string propName, object value, Exception ex)
    {
        throw new NotImplementedException();
    }
 
    public bool RecordLoadError
    (object target, int index, object source, Exception ex)
    {
        throw new NotImplementedException();
    }
} 

[MetadataType(typeof(EmployeeRecMeta))]
public partial class EmployeeRec
{
    public int Id { get; set; }
    public string Name { get; set; }
}

9.1 BeginLoad

This callback invoked once at the beginning of the FixedLength file load. source is the FixedLength file stream object. In here, you have a chance to inspect the stream, return true to continue the FixedLength load. Return false to stop the parsing.

Listing 9.1.1 BeginLoad Callback Sample

public bool BeginLoad(object source)
{
    StreamReader sr = source as StreamReader;
    return true;
}

9.2 EndLoad

This callback invoked once at the end of the FixedLength file load. source is the FixedLength file stream object. In here, you have a chance to inspect the stream, do any post steps to be performed on the stream.

Listing 9.2.1 EndLoad Callback Sample

public void EndLoad(object source)
{
    StreamReader sr = source as StreamReader;
}

9.3 BeforeRecordLoad

This callback invoked before each record line in the FixedLength file is loaded. target is the instance of the POCO record object. index is the line index in the file. source is the FixedLength record line. In here, you have chance to inspect the line, and override it with new line if want to.

TIP: If you want to skip the line from loading, set the source to null.

TIP: If you want to take control of parsing and loading the record properties by yourself, set the source to String.Empty.