Building a framework - Part II (Utilities)

Linked articles

Building a framework - Part I (DAL)

Introduction

The first article handled database connections (one of the many possibilities) and how to make this reusable. This article builds further upon part I and will tackle other well used functionalities that often end up spread throughout the application (and between applications).

Again note there are many ways of doing things and this is just one of them. This article is more about structuring functionality than about the code.

Pick what best suits you.

Logging

Logging is in many cases still done with the use of ASCII files in very divergent ways. Log entries can be a single line or a block of lines, there can be a turn around file system (like log1.txt through log3.txt returning to log1.txt when log3.txt is full) and the number of file extensions can be vast ranging from *.txt to *.log to whatever else the developer came up with. To make things worse, some logs are done to binary files.

Often people ask what to log in an application. That's up to you really but here are some pointers and things to think about:

• ALWAYS log exceptions
• SQL queries (if it is parametrized, also provide the values)
• * each call to a function with the arguments passed.
• * each return of function (if possible)
• Key points in code, eg milestones in calculations, starting/ending of a certain workflow process, ...
• logon/logoff of users
• Important decisions of users
• Communications with other applications (send/receive)
• Always imagine you're in a position where you need to figure out a problem based on the logs alone.
• The advantage of logging to database is that you can develop a viewer or monitor for it and display it in any way you want (eg as a webpage on your phone, completely color coded) . You can also build reports from it for your managers and build easy-to-use filters to only see part of what you need at the time. So my advice would be to use a database (instead of files) if you can)
• Logs could originate from other systems like linux eg.
• If using files, keep the size of the file limited. I would recommend 2 MB to 5 MB, with an absolute maximum of 10 MB.
• Careful when logging inside a loop and keep in mind there is such a thing as too much !

* - not absolutely necessary. In fact, if you add logging here, it is probably more a debugging process then an operational process.

Settings

Settings are somewhat similar to logging in the sense that it often ends up in a file on a local computer and even in multiple files. Again multiple versions are in use: XML files, the .NET config files (web.config, app.config), *.ini files, binary files, etc...

Settings are mostly limited to key / value pairs and so here as well I opted to add them in the database. Even (linked) lists could be added (to fill combo boxes eg), but are not discussed in this article.

What should be in the settings? That's up to you really, but I usually add paths, urls, configuration parameters, etc...

Here you could write a configuration application allowing to change this easily. It has as an advantage that you don't have to start looking for the configuration file in the hope you'll quickly find it and if that one is really in use and if the settings are not overwritten by another file.

Background

I created this log module because we used to have a big system with many applications working together, sending information up and down, through the database, via udp, via tcp, via message queuing etc... When debugging, in many cases we ended up with two to four logfiles open attempting to navigate different formats and comparing datetimes of logs that where generated on different machine that have non-synchronized clocks.

When coming to another company I had some freedom in redesigning some stuff and I decided to create a module to log to database. The timestamp on the log is that of the database so all timings are in sync and you can use SQL to filter out what you need. In addition, you have one source to look in.

For both Settings and Logging you could use the Enterprise Libraries of .NET, which comes with a configuration program. The configuration program, however, is (IMHO) not very user friendly and logging requires some figuring out, though it is OK to use it with or ASCII files or SQL-Server. When you have another database (oracle eg) you're in for some fun and you're better of dropping this thing.

An important thing to note, and I guess this goes for both ways, is that when an application crashes you have the log that will explain this crash. In many cases the application crashed BEFORE the log could be written.

A quick word on design

This is a very basic design for a small to middle sized desktop or web application. In larger applications, layers can be split like a Business Object layer and a Business Logic layer or multiple DAL components where one part talks to database, the other to web services and yet another reads and writes files. Other applications like interfaces or services on the other hand don't have a GUI layer.

This article handles the Utilities layer. The goal of this layer is to make functionality available throughout the layers. Logging and Settings are discussed here, but other things are suited for this layers as well like user-access rights, converters you need (eg gregorian date to julian date , ...) etc.

In my case I put the DAL and Utilities framework in one assembly, which makes sense because the functionality is small and the utilities use the DAL component. Since it is one assembly they're not dependent of each other. If the framework does grow large you might want to decide to split the projects in multiple assemblies.

Using the code

The framework now contains two blocks:

1. Dal
2. Utiltities

For the Dal component I refer to my first article. The Utilities component consist of

• Log
• Settings

but this is merely an example. In our own framework I also added Julian/Gregorian data converters eg. Before LINQ existed I also had an XML module in there, etc...

Note: Although I called this utilities, I've seen this under many different names: utils, tools, bricks, ...

The log module consists mainly of 3 classes. One is the actual log class which takes some settings passed to it and "dispatches" to the Log2File and/Or Log2Database class.

Let's look at the logging classes first.

Log

To create a new log instance you need to use the static "GetInstance" method :

//create a log object that logs to file
Log log = Log.GetInstance(@"C:\temp\logtest.txt", "");

The log comes with four settings:

public string Path{}
public string ConnectionString{}
public Framework.Dal.Enums.provider DBProvider {}
public int MaximumFileSize{}

that can be used to override the default settings or to change logfile or logdatabase.

To actually log an entry you would call the WriteLog method:

public void WriteLog(Enums.LogType type, string text, string applicationormodulename, Enums.Actor actor, Enums.Medium medium){}

You could choose to create a class that holds these arguments as properties and just pass that as a class if you want.

The WriteLog method will check what it needs to log and pass the arguments to the Log2File and/or Log2Database classes, which you cannot call directly.

Note that we can pass Enums.Medium.File|Enums.Medium.Database as last parameter to log to both mediums. That's because the Enum for Medium has the Flags property:

[Flags]
public enum Medium{
    None = 0,
    File = 1,
    Database = 2
    /*next should be 4, 8, 16, ..; */
}; 

The Log2File class has some notable features.

• It will check filesize and recreate the file if necessary. (turnaround, but just one file)
• It uses the lock feature to make the process thread safe.
• It does not use database time (be careful when logging to both mediums, because the machines might not be synchronized!)
• It will add the datetime for you, so you do not need to provide it with the logtext.
• It uses one line per log (you can add newlines with the text), but you can format the file any way you want by changing the Write method.
• The class is internal, so you cannot use it from outside the assembly. (You need to use the Log class for that)

The Log2Database has some notable features.

• In this case the Write function uses OleDb to an oracle instance, but you can modify this to allow for multiple databases and access providers (MySql, SQL-Server, OleDb, Odp.Net, ODBC, ...)
• The datetimestamp for the log does not need to be provided, instead the current timestamp of the database is used. (This is different for each database, in case of Oracle this is systimestamp), make sure to set the "granularity" sufficiently high (also when defining the column!) meaning up to millisecond level (or basically as high as you can)
• Of course the table you log too needs to EXIST.
• The class is internal, so you cannot use it from outside the assembly. (You need to use the Log class for that)

To finalize the log module some explanation on the properties of a log:

1. LogType
   This indicates the type of log. Information/Error/Warning/ ...
2. text
   The logtext
3. application or module name
   The name of the application making the log. If you use multiple applications that log to the same place, this is important.
4. Actor
   User or System. This could be used to indicate the difference between an action or decision of a user (in workflow processes eg) or a machine log (automated systems, interfaces, ...)
5. Medium
   Medium to log to like file, database, system event logs, you're iPad, sms text, ...

In a java version I use right now I also added severity and debuglevel, the latter comes in handy if you are in a debugging or problem solving mode, but you want to keep the operational logging low.

Settings

The Settings class is very, very simple. The reason I added it, is because when I was still a junior developer I had trouble in getting information across levels of code. So you do what you probably saw before: Create a class that holds a bunch of global variables accessible for all. But that's not very clean, is it?

The simplest way I found looks like this:

 public class Settings {
    private static Dictionary<string, string> settings;
    private static void Load(){
        /*    Load settings from database and / or file.
         *    hardcoded for testing
         */
        settings = new Dictionary<string,string>();
        settings.Add("database", "PostgreSQL");
        settings.Add("username", "John Doe");
        settings.Add("path", @"C:\temp\");
    }

    /// <summary>
    /// Gets the value associated with a certain key.
    /// </summary>
    /// <param name="key">The key of the setting.</param>
    /// <returns>The value of the setting.</returns>
    public static string GetSetting(string key){
        string setting = "";
        if(settings == null){
            Load();
        }            //end if
        if(settings != null && settings.ContainsKey(key)){
            setting = settings[key];
        }            //end if
        return setting;
    }
} 

The Load method is launched when accessing the settings for the first time. In my case this involves reading the config for the database parameters and reading out the settings table. Here the key / value pairs are always strings and this class only handles key/value pairs. However you could extend this to your own needs. In addition you could also load (linked) lists from file or database to populate comboboxes, categorize settings, assign them to users, work with override categories etc ...

If you add settings to the database you can keep the following in mind

• settings are mostly key / value pairs so the least you have are these two columns.
• settings might need to have one or more categories assigned to them.
• There are basically two types of settings: system settings and user settings. The normal user should not access the system settings. User settings can be different between users.
• Different applications can have an identical key, but a different value.

Points of Interest

• If your settings are in the database, is the database connection hardcoded or something?
  The answer is "no", the database credentials are still in the (preferably) standard config files and that's also the only thing in there. (WCF is an exception)
  You could "hardcode" development / test / production database credentials in code and just set a parameter Dev / Test / Prod eg.
  So I agree, putting settings in a database does not solve the settings file completely, but it does simplify things.
• If I would build the log from scratch, I would probably redesign it in using an abstract base class.
  In addition, if you choose to only use files or only use a database, you can simplify this significantly. On the other hand you might need to log to the windows event logs eg. In that case you could add a module.

Final word

I'm sure some people who read this article would disagree with this way of working, but I want to stress that this is just one way of doing things. In fact, this article is more about getting structure into the code and avoiding some pitfalls than the actual code. Although the classes changed a bit throughout the years, the basic principle was kept.

If this is interesting for you, don't blindly copy/paste the code. Use whatever is useful to you and adapt to your own needs.

History

Version 1.0 (April 2014)