A simple architecture to read arbitrarily formatted flat files into ADO DataSets

• Download class source files - 3 Kb
• Download VC# demo project - 6 Kb

Introduction

There are Ole Providers to read formatted flat files into DataSets, but ever wondered about importing data from files which are not simply comma delimited or tab delimited (not columnar) but some native format.

One of the simplest examples I could come up with are ini files. Although simple enough, there is no columnar or delimited data. So how do we use C# to import such files into a database ?

Since there are 2 distinct pieces of data (sections and key/value pairs) the a reader class uses a System.IO.StreamReader to read lines from the specified ini file and raise events when a section or a key/value pair is encountered. A data class, the IniFileData class has a System.Data.DataSet member holding all the sections, keys and values from the ini file. This class subscribes to the events (SectionRead and KeyValueRead) from IniFileReader creating the corresponding rows and adding them to the DataTables.

Once the data is read into the DataSet it is left to the user's interest to deal with it. I have given a small example by writing the GetValue() method on the IniFileData class to retrive a value by giving the section and key name.

Using the code

Create an instance of IniFileData class and give an ini file name. The class reads the file and prepares the DataSet.

The following code shows the declaration of delegates and events in the IniFileReader.

// delegate to receive a SectionRead event

public delegate void SectionHandler(string sectionName);
// event raised when a section is read

public event SectionHandler SectionRead;
// delegate to receive a KeyValueRead event

public delegate void KeyValueHandler(string section, 
    string key, string sValue);
// event raised when a key, value pair is read

public event KeyValueHandler KeyValueRead;

These events are kicked off as the StreamReader is reading the flat file. The most appropriate way to kick off these events is when a row of data is available in any table in the DataSet. Here you can see that these events are thrown when a section is read or a key/value pair is read.

    class IniFileReader
    {    .......
        public void ReadIniFile(string filename) 
        {
            StreamReader sr = new StreamReader(filename);
            ........
            while (not end of file)
            {
                if (oneLine.StartsWith("[") && oneLine.EndsWith("]")) 
                    SectionRead(currentSection); 
                        // raise the section read event

                }
                else {
                    .....
                    // raise an event if a key/value pair is read

                    KeyValueRead(currentSection, keyvalue[0], keyvalue[1]);
                }
            }
        }
    }
        

The class which is preparing holding the DataSet, in this case the IniFileData class subscribes to these events and prepares the rows and inserts them into the corresponding tables. The relevant code is pointed below.

    class IniFileData
    {
        .....
        public void ReadIniFile(string PathName)
        {
            ......
            IniFileReader ir = new IniFileReader(); 
               // create instance of iniFileReader


            // subscribe to the event handlers

            ir.SectionRead += new IniFileReader.SectionHandler(
               this.OnSectionRead);
            ir.KeyValueRead += new IniFileReader.KeyValueHandler(
               this.OnKeyValueRead);
            ......
        }
        private void OnSectionRead(string Section)
        {
            // create the sections here

            .......
        }
        private void OnKeyValueRead(string Section, string Key, 
               string sValue)
        {
            // insert the key value pairs here

            .......
        }
    }
        

Points of Interest

In the IniFileData.GetValue() method I couldn't find a way to join different tables in a single query when retreiving the value. If I find one I will post it here.