Configuration Manager for the Enterprise

• Download source code - 643 KB

Introduction

SysConfiguration is a component that allows storing of configuration data for an enterprise application. It is intended to be an improved version of the System.Configuration class or the Enterprise Library (Microsoft Patterns and Practices).

Enterprise Configuration

I work for a group at T4G Limited, which creates complex custom applications for large enterprises. It's not uncommon that a single application has multiple user interfaces (consumer, call center, B2B, etc.). As a result, we often have multiple architectures (web, Web Services, WinForms, COM+, etc.) running in parallel. As a technology services company, there is a high degree of requirement volatility, which means we have frequent deployments to multiple environments (Development, Test, Staging, User Acceptance Testing, Production, etc.). In line with the types of projects we do, the goals of this component were:

• Automatically detect if a configuration setting was missing
• Problem: A big frustration with the App.Config is that it's loosely typed. This means that if there is a setting that is used by a component, you have to actually test that component before you can determine whether it's there or not.
• Solution: SysConfiguration uses a strongly typed class. The very first time it's loaded, it will validate the configuration data against a schema to ensure that all mandatory settings are there, while still allowing for optional settings. This is a huge benefit when moving code through the environments (e.g., Development to Testing to Production).
• Works for multiple architectures
• Problem: If you have an application that has multiple architectures, each one stores their configuration in different places: Web.config for web applications and Web Services, App.config for WinForms and console applications, Registry for COM+, etc. The Enterprise Library does have the FileConfigurationSource class, which allows you to call a centralized configuration file, which is an improvement over the default .NET Framework.
• Solution: SysConfiguration can use the same file for all architectures. The only difference is the location of two entries that point to the location of the configuration file and the schema that validates it. This provides a degree of flexibility, because systems can share the same configuration, or point to different locations if you want to keep them distinct.
• Strongly typed hierarchical data
• Problem: Although you can create hierarchies with System.Configuration, when you actually access a property, you still have to use AppSettings["LooselyCoupledKey"]. This does not allow typos to be detected at compile time, only at runtime.
• Solution: SysConfiguration generates a hierarchical object model that represents the configuration class. This means you get code that is easy to understand, caught at compile time, and can use IntelliSense (e.g.: SysConfiguration.Data.AppConfig.ConnectionStrings.Core).
• Standardized configuration entries
• Problem: At T4G, we tend to share components whenever we can. If you want to pass a component to another person, it would be desirable to reliably pass the configuration entries specific to that component. If you use something like App.Config, all you can do is hope that somebody has commented it well enough to indicate which settings belong to which components.
• Solution: SysConfiguration is designed in a way that the hierarchical structure of the configuration can be broken up into many files. For example, they could be broken up into: AppConfig, which stores settings that are specific to the current application, T4GToolbox, which stores all of the settings for our re-usable framework, and RightsConfig which stores the settings for a re-usable security component, etc. SysConfiguration is designed to merge all of these definitions into a single configuration file, so there is only one file to maintain per environment.

Flexible Singleton

An ideal Design Pattern for storing configuration data is the Singleton. You want the configuration data to be loaded from disk once and then kept somewhere fast so that it can be accessed quickly. The problem with the Singleton is that almost all the examples just keep it in memory. If this was used in a web context, the memory would be cleared every time the Application Pool or process recycles the memory. If it was used in COM+ components, it would release that memory as soon as it was idle for a period of time.

SysConfiguration detects the context it is running as, and then stores it in an appropriate storage:

• For web applications and Web Services, it uses the Web Cache.
• For WinForms and Console applications, it uses memory.
• For COM+, it uses the Shared Property Manager.

Test Driven Development

I also wanted to mention that this project was entirely built using Test Driven Development (TDD). Although I probably didn't follow the principles of "You Ain't Gonna Need It" (YAGNI) as closely as I could have, I feel it was the correct balance of optimization and future use. Although I've used TDD for components before, I used this project as a test bed for some of the claims made by proponents of TDD and see for myself whether they are true. See Conclusions about Test Driven Development.

In terms of the process I followed, I consistently wrote the test first, failed it (red), got the code to work according to the test (green), refactored the unit test, and started the cycle all over again.

I've included all the unit tests, which may be of interest because I feel I built out a fairly good process when testing a single component across all the architectures. The most important design decision is that there are a set of tests, called by NUnit, but in many cases, those tests call other tests residing in different processes. So, there is a central assembly that NUnit calls, but the actual unit testing logic is stored somewhere else. This ensures I can call all of the unit tests at once any time I need to.

Using the Code

The install includes optional steps depending on your goals. The first set of steps are for those people who are interested in the configuration component. The second set of steps are additional steps, if you want to learn more about Test Driven Development.

I should mention, there are some manual steps for setting this up, but once it has been integrated, it is very easy to maintain.

Pre-Requisites for SysConfiguration

1. The project was built using the .NET Framework 2.0 and Visual Studio 2005.
2. You will need to install XSDObjectGen (http://www.microsoft.com/downloads/details.aspx?familyid=89e6b1e5-f66c-4a4d-933b-46222bb01eb0&displaylang=en). This is used to generate the configuration class generated by the schema. It assumes the default install path is used, which will put the VS2005 version into C:\Program Files\XSDObjectGenerator\vs2005.

Additional Steps to Use Unit Tests (Optional)

If you are interested in looking at executing the Unit Tests, there are a few more additional steps.

1. Install the latest version of NUnit. At the time of this article, it was up to version 2.4 (http://www.nunit.org/).
2. Download NUnitASP (http://nunitasp.sourceforge.net/). At the time of this article, it was version 1.5.1. Extract the zip file to C:\Program Files\NUnitASP.
3. Install NUnit 2.2 from the bin directory of NUnitASP. You will be able to run both NUnits in parallel.
4. I had to add <supportedRuntime version="v2.0.50727" /> to nunit.exe.config and nunit-console.exe.config. You may have to do the same depending on the .NET runtimes installed on your machine.
5. You will need to be on an Operating System that supports Component Services or COM+. There is a class with tests that are installed using the regsvcs.exe command line tool.
6. There are some constants in UnitTestCommon\Constants.cs that will have to be changed. These are set to paths that it expects back, which may differ from your file system. You should be able to figure these out the first time you run the unit tests and it fails.
7. Open the properties for the UnitTests project. Go to Debug. Choose the Start Action to be "Start External Program". Select the latest version of nunit.exe.
8. Open up the properties for the WebTestHarness project. Go to Web. Select "Don't open a page. Wait for a request from an external application."
9. Open the solution properties. Go to Startup Project, select "Multiple startup projects", and choose "UnitTests" and "WebTestHarness" to both start.

Adding SysConfiguration

1. The project is not designed as a standalone component. There is a class that is automatically generated based on the schema file each time the project is built. This file is then compiled along with the rest of the project. Therefore, it needs to be added to the solution of your application.
1. If you don't care about unit tests, copy the configuration project into your solution.
2. If you want the unit tests, copy the configuration project to your main solution. Copy all of the other projects to wherever you are keeping your unit tests. In my case, I kept them in a folder in my solution so there is a clear separation. See SysConfigurationWithUnitTests.sln for an example.



2. You will have to configure your application to find the configuration file and the schemas that validate it.
1. If it is a COM+ component, SysConfiguration needs to use the Registry to find the paths. There is a sample .reg file SetupComPlus.reg provided. It will put two keys (ConfigurationPath and ConfigurationSchemaPath) into the HKEY_LOCAL_MACHINE\SOFTWARE\Name.Randar key.
2. For any web based application, add the following to the Web.config and point the paths to the correct location. For a .NET application, make the same changes, but to the App.config.
XML

<?xml version="1.0" ?>
<configuration>
    <!-- Add these items to your configuration file and point them 
              at the location that has the configuration and schema-->
    <appSettings>

        <!-- Development Configuration Path -->
        <add key="ConfigurationPath" 
          value="C:\Projects\SysConfiguration\Code\Configuration\
                     Development\Configuration.xml" />
        <!-- Production Configuration Path -->
        <!-- <add key="ConfigurationPath" 
          value="C:\Projects\SysConfiguration\Code\Configuration\
                   Development\Configuration.xml" /> -->
        <!-- Path to Schema used to validate configuration files-->

        <add key="ConfigurationSchemaPath" 
          value="C:\Projects\SysConfiguration\Code\
                   Configuration\Schemas\Settings.xsd" />
    </appSettings>
</configuration>

3. If there are dependent schemas, you will have to change the absolute paths of xs:include in Settings.xsd. There does not seem to be a way to make the path relative, which is unfortunate. I would recommend you try to keep the same file structure on disk across the various environments (development, staging, etc.), or make sure to not overwrite your Settings.xsd when you update the code.

SysConfiguration Tasks

The following is a "help file" of common tasks you would need to perform to use SysConfiguration.

To add a configuration element

The following describes the steps to add a new configuration setting to the application, once the pre-requisite steps have been followed. SysConfiguration is designed to allow developers to quickly add new configuration settings and allow them to define the validation required.

1. If you would like to add a configuration element, simply change the definition in one of the schemas (e.g., AppConfig.xsd, RightsConfig.xsd, etc.). Do not add configuration elements to Settings.xsd. This is a central hub of the various configuration definitions. It will generate a class that has all references to all the other configuration files. Similar in concept to a project file.
2. The following are some examples that are supported:
   1. Data types. If you set the type of an xs:element, it will create a valid .NET type.
   2. minOccurs and maxOccurs. These allow you to create collections if maxOccurs is unbounded, and optional settings if minOccurs is 0.

To add configuration data

Once the structure has been defined, the following are the steps to add configuration data:

1. Open up Configuration.xml in your favourite XML editor. If you are using Visual Studio, it should have the targetNamespace setup so that Intellisense will allow you to build the hierarchy and add the data.

To access a configuration element

Once the configuration definition has been defined, and the data added, the following describes how the application would access that data:

1. Add a reference to Name.Randar.Configuration.dll.
2. Add a using statement for the namespace (e.g.: using Name.Randar.Configuration).
3. Access the configuration hierarchy, starting with the Singleton called SysConfiguration. You can access the data either through the static Data or Instance.SettingsData. Both are identical. Some examples:
1. SysConfiguration.Data.AppConfig.ConnectionStrings.Core
2. SysConfiguration.Instance.SettingsData. AppConfig.ConnectionStrings.Core
3. SysConfiguration.Data.DebugSettings.LogToFile
4. SysConfiguration.Instance.SettingsData.AppConfig.NumberOfRetries
5. SysConfiguration.Data.AppConfig.CloseOutDate
6. SysConfiguration.Instance.SettingsData.AppConfig.MaxAllowed

To access a list of configuration elements

SysConfiguration supports single name\value pairs as well as lists. The following describes how to access a list of data:

1. It stores lists as a strongly typed sub-class of the ArrayList class. So you can access it by index or by iterating through the list:
1. SysConfiguration.Data.RightsConfig.RightCollection[0].Name
2. SysConfiguration.Data.RightsConfig.RightCollection.Count

// Search through the roles until you find the one we are looking
int findRole = -1;
for (int i = 0; i < settings.RightsConfig.RightCollection.Count; i++ )
{
    if (settings.RightsConfig.RightCollection[i].Name == Constants.RoleToSearchFor)
        findRole = i;
}

To add a configuration definition

If you would like to extend the configuration definition, but leave the main definition alone, you can use xs:include to include other schemas. This allows you to break up the configuration definition based on components or other requirements.

1. Create your schema that defines your configuration. Make sure it has a root element.
2. Modify Settings.xsd:
1. Add an xs:include and point it at the new file.
2. Create an element that references the base element of the other schema file.

<?xml version="1.0" encoding="utf-8"?>
<xs:schema id="Settings" 
         targetNamespace="http://Randar.Name/Settings.xsd" 
         elementFormDefault="qualified" 
         xmlns="http://Randar.Name/Settings.xsd" 
         xmlns:mstns="http://Randar.Name/Settings.xsd" 
         xmlns:xs="http://www.w3.org/2001/XMLSchema">
    <xs:include schemaLocation=
      "C:\Projects\SysConfiguration\Code\Configuration\Schemas\AppConfig.xsd">
    </xs:include>
    <xs:include schemaLocation=
      "C:\Projects\SysConfiguration\Code\Configuration\Schemas\RightsConfig.xsd">
    </xs:include>
    <xs:element name="Settings">

        <xs:complexType>
            <xs:sequence>
                <xs:element ref="AppConfig">
                </xs:element>
                <xs:element ref="RightsConfig">
                </xs:element>

                <!-- Add new elements here that reference other schema files-->
            </xs:sequence>
        </xs:complexType>
    </xs:element>
</xs:schema>

To reload the configuration data

You should setup your application so there is a way to clear the cache and force it to re-load the data next time it is used. Otherwise, you will have to restart the process somehow to clear the data. The following steps describe how this can be done:

1. You will have to create something that can be called manually. For example:
1. A web page in an admin area with a button called "Clear Cache".
2. A Web Service with a web method called "ClearCache".
3. A button on the Help->About screen that has a button "Reload Configuration".
2. In all of these cases, the method should call SysConfiguration.ClearCache().

Out of Scope

The following scenarios have not been tested or are not fully supported

• Backwards compatibility. I have purged my development machine of all older development environments, including VB6. Therefore, I haven't tested it on anything older than the .NET 2.0 Framework.
• I was able to register to the component using Regasm.exe, view it in OLE Viewer, but none of the classes had methods. I haven't needed to do COM\.NET integration since 2002, so I'm sure there is something small I have forgotten, but didn't feel this was a priority.
• If you need to port it to the 1.0 framework:
• You will have to use an older solution file, or manually edit the solution using a text editor.
• Collapse the SysConfiguration into a single class instead of using a partial class.
• Use the legacy classes instead of the 2.0 System.Configuration namespace.
• Inheritance. Unfortunately, it is hard to make to make the Singleton design pattern inheritable, without some sort of hack, such as the base class knowing of the super class. It is unfortunate, because I can think of many scenarios where SysConfiguration could be a good base class. My best compromise to this was to break up the Singleton and configuration functionality using partial classes.
• C# only. It was enough work building this once. I didn't have time to create it in VB.NET as well.
• Centralized configuration data. Some third party configuration managers have a central location for configuration data, that all local and remote processes access. I've never liked this architecture because you have so many cross process\network calls. If you really wanted this, you could load it into Component Services and make all the calls through there, but you will get a performance hit.
• No user interface for editing configuration. I don't find that much value in the user interface provided by the Enterprise Library, and I usually edit my configurations with Notepad. Keep in mind that SysConfiguration uses standard XML, validated against a schema, so you can use the XML editor in Visual Studio or any other third party tool. Since the XML is validated against the schema, you should get a warning if you put in an invalid value.
• Configuration structural changes require recompilation. As I mentioned before, you cannot change the structure of the configuration without recompiling. This is because of the strongly typed nature of this project. I don't feel this is a deficiency because you would have to change your code to access the new configuration element anyways, but wanted to bring it up. Obviously, you can change the configuration data without recompilation, just not the structure.
• Editing configuration. There is no built-in support for editing configuration data. There are probably reasons for doing this, but I've never run across it. If the data is volatile, I will always store it in the database so it can be shared across the server farm. At the end of the day, it's just XML, so you could edit it using the standard XML classes.
• No integrated encryption. .NET 2.0 has the ability to encrypt sections of config files. I have not built this functionality in, but can see myself adding it later. For now, use the standard cryptography libraries and do it yourself.
• Not CLS compliant. Although my code is CLS compliant, the code generated by XSDObjectGen is not. The generated code is a key item, that I felt it best to mark the entire class as non-CLS compliant.
• Automatic file change detection. A nice (or not nice) feature of App.config and Web.config is that it will restart the process if the config file changes, reloading the settings. I have not looked into replicating this feature (or defect, depending on your point of view). I personally think I prefer being able to force the reload manually, but would like to make this an optional feature.

Points of Interest

The following are some comments about the code and development process that may be of interest to some people.

Conclusions about Test Driven Development

The following section describes my opinion of Test Driven Development when compared to the traditional process of developing the code and (maybe) creating automated unit tests. It is not based on any empirical study, and should be treated as opinion, not fact.

I definitely feel that TDD helped me make a much more robust component than using traditional development. It was very good for testing the various architectures quickly and repeatedly. A few times, I made some major refactoring of the architecture, and having a full regression test was invaluable in helping me to get the component functional again.

In terms of whether I actually developed this faster using TDD than traditional development methodologies, I am not sure I agree. Creating and refactoring the unit tests do take time and can't be discounted. The argument is that creating the tests takes less time than debugging the problem. The only way to accurately test this would be to clone a person and have them develop the component under identical conditions but using both techniques, which is just not possible. I do think you would see a much greater return when you have a team larger than a single person, because the scenario of another developer breaking somebody else's code is much more likely in that case.

Some proponents of TDD will say that when tests fail unexpectedly, reverting the code to the last version that passed all tests is almost always more productive than debugging. I have to say, I did spend considerably less time debugging than usual, but don't feel I should have ever rolled back. I think debugging was always the better option, especially considering how good the debugger is in VS2005.

Even though this version was developed by a team of one, I tried to act like I was working in a team. I tried to make sure to only check in the test when the underlying code passed (i.e., green). In theory, this means that if we were using Continuous Integration, it would always pass. The only issue was to fight the urge to check in the test once it was written. I'm used to the mentality of keeping my changes small and check-ins frequent, but you have to change your mentality slightly.

One item of interest is the ratio of lines of code of the unit tests vs. application code.

• 4887 total lines of code
• 3401 lines of unit tests
• 1486 lines of application code

So we end up with an approximate ratio of 2:1 of unit tests to application code. That is higher than typical, although many of the tests are duplicates for different architectures, which would explain the high ratio.

To answer the question of whether I would suggest using TDD on a project, the answer is yes, with caveats. Although you can't tell from this project, I would suggest using TDD for any component that is called by other components. Writing effective unit tests for the web layer is still difficult unless you build in some way to bypass authentication and session data. I definitely feel it creates much more reliable code and makes changes easier due to the extensive regression test it creates.

Quality of Code

Since this is intended to be used and extended by a broad range of people, I tried to:

• Clearly comment the code as much as possible.
• Conform to the C# Coding Standards for .NET by Lance Hunt (http://weblogs.asp.net/lhunt/pages/CSharp-Coding-Standards-document.aspx), which is the standard used at T4G.
• Use FxCop to clean up some exception handling and other style issues. Many of the exclusions were from the code XSDObjectGen generated, or simply didn't apply to what I was trying to accomplish (e.g., globalization).
• I really wanted to use NCover to ensure I had 100% code coverage with my unit tests. This was actually harder than I expected. Many of the tests run in different processes (console, web, COM+, etc.). NCover can't figure out they are all running from the same set of tests, so it showed poor coverage, even though the superset of all the tests would probably be close to 100%.

XSDObjectGen.exe

I wanted to make a few comments about XSDObjectGen, a.k.a. Sample Code Generator (http://www.microsoft.com/downloads/details.aspx?familyid=89e6b1e5-f66c-4a4d-933b-46222bb01eb0&displaylang=en).

• At one point, instead of using XSDObjectGen.exe, I did try to use XSD.exe to generate my class files, but it suffers from one major deficiency. No support for the <xs:import> tag. This forces you to keep all of your definitions in one file, which is difficult to manage.
• I find XSDObjectGen so much better than Strongly Typed Datasets, because they are lighter and you can create deeper hierarchies than row and column. The only issue is that you have to load the data yourself.
• If you ever need to create Data Transfer Objects (http://en.wikipedia.org/wiki/Data_Transfer_Object), you should seriously look at XSDObjectGen. It creates great lightweight objects that serialize well.

Troubleshooting

Problem: System is raising a System.IO.FileNotFoundException.

It cannot find the configuration file or the schema used to validate it. See step 2 of "Adding SysConfiguration".

Problem: The compiler is raising: Could not find a part of the path 'C:\Projects\SysConfiguration\Code\Configuration\Schemas\RightsConfig.xsd'. C:\Projects\SysConfiguration\Code\Configuration\Schemas\Settings.xsd.

xs:import seems to need the absolute path. Edit Settings.xsd to point to the correct location.

Problem: The compiler is raising: Unable to copy file "obj\Debug (With Sample Configuration)\Name.Randar.txUnitTests.dll" to "bin\DebugWSC\Name.Randar.txUnitTests.dll". The process cannot access the file 'bin\DebugWSC\Name.Randar.txUnitTests.dll' because it is being used by another process.

The COM+ component is still running. Open up Component Services, expand COM+ Application, right click on SysConfiguration Unit Tests, and choose "Shut Down".

Conclusion

I wrote the first version of this component in 2002. Since then, it's been rebuilt and refactored many times. At the same time, the overall core design has changed very little and it's weathered the test of time. This latest version and the accompanying article took quite a bit of time to write. You are free to incorporate it into your code, and all I ask is that you don't take credit for my work and to please email me with feedback (both good and bad) if you do use it.

History

• July 12, 2007: First revision.
• August 8, 2007: Removed the password on the strong name key that is used to sign the assembly.