Introduction
It is often required to validate flat files before importing them to the database, or after exporting data from the database. Flat File Checker allows you to validate an inter-related set of flat files (*.csv, *.psv, *.txt, etc.) without having to parse them directly in your code.
The business rules against which you must validate the files must be made available to the FLaFi library in the form of an XML file, which is created through the graphical user interface of the FlaFi application. Here are examples of such rules:
- Value in the field is Required
- Field is a date with a given format and within a given range
- Field structure must match a Regular Expression
- Two files are linked [relational links]
- Field must be compared to another field
- Unique constraint is imposed on one or several fields
- Validation must be made against values stored in a database
The task of validation can be now addressed in a few easy steps:
- Create a Flat File Schema (XML file that contains business/data rules for files that you want to validate)
- Add the Flat File Checker library to your .NET application
- Run validation with minimum code within your application
Background
The idea behind Flat File Checker is to create an application that can validate structured data in a text file based on schema stored in XML, in a way similar to the way XML files are validated. This approach separates data/business rules from the validation itself, and thus considerably less coding is required.
Creating the schema
Here is a quick way to create a schema file to use for validation by FlaFi:
- Download the installer of the latest version of Flat File Checker from SourcForge.net
- Install the application and start the application
- Create a new schema using the application and save it locally in an XML file
When the schema is created, you can view it in browser as a readable data exchange specification:
Add the Flat File Library to your solution
Add the Flat File Library (see link at the top) project to your solution in Visual Studio.
Run the validation and process errors
To use the Flat File Library, you will need to use these main classes:
FlatFileSchema
- The class that contains a collection of filesFile
- The virtual class that gives access to several objects associated with a flat file: columns, errors, etc.DataError
- The class that contains details of the data error
You can also include system threading to your project as validation is done in asynchronous mode:
Imports System.Threading
The following code shows how to start validation (the RunValidation
sub) and process errors if needed (the FileSetValidated
delegate sub).
Private WithEvents _files As FlatFileSchema
Private do_checks As AutoResetEvent
Sub RunValidation()
_files = New FlatFileSchema(<Schema file path>)
do_checks=_files.RunChecks()
End Sub
Private Sub FileSetValidated(ByVal sender As Object, _
ByVal e As SchemaValidatedEventArgs) _
Handles _files.Validated
Dim file As FlatFile
Dim err As DataError
For Each file In _files.Files
For Each err In file.Errors
...
Next err
Next file
End Sub
Points of interest
The console version of Flat File Checker (code attached) makes it possible to run validation from command line with just one line:
c:/program files/flat file checker>flafi.exe <schema path>