How to Write a Custom Logging Provider in ASP.NET Core

Introduction

Source code can be found on github.

There are no official instructions on how to write a custom logging provider in ASP.NET Core in the available documentation yet. So if someone is in need of writing a custom logging provider in ASP.NET Core, he has to study that documentation and the related source code of the framework.

The required parts are:

• a simple class for the log options, that is a POCO
• an implementation of the ILogger interface
• an implementation of the ILoggerProvider interface
• a few extension methods for registering the logger provider to the framework

Let us see the two interfaces:

namespace Microsoft.Extensions.Logging
{
    public interface ILogger
    {
        IDisposable BeginScope<TState>(TState state);
        bool IsEnabled(LogLevel logLevel);
        void Log<TState>(LogLevel logLevel, EventId eventId, 
             TState state, Exception exception, Func<TState, Exception, string> formatter);
    }
}
 
namespace Microsoft.Extensions.Logging
{
    public interface ILoggerProvider : IDisposable
    {
        ILogger CreateLogger(string categoryName);
    }
}

How to Implement the Required Interfaces

The sole purpose of ILoggerProvider is in creating ILogger instances when asked by the framework.

ILogger provides the Log() method. A call to Log() produces a unit of log information, a log entry.

Which of those two code elements should be responsible for displaying or persisting that log entry?

By studying the ASP.NET Core code, it becomes obvious that this responsibility goes to ILogger implementations, such as ConsoleLogger, DebugLogger and EventLogLogger classes. Should we do that too?

If the answer is yes, then we need an ILogger and an ILoggerProvider implementation for any medium, say text file, database or a message queue.

If the answer is no, then we need just a universal ILogger implementation and just an ILoggerProvider implementation for any different medium.

We are going to follow that second approach.

A single universal Logger class that produces a unit of log information, packs that information into an instance of a LogEntry class, and then passes that instance to its creator LoggerProvider for further processing. That LoggerProvider is going to be a base class, so any specialization regarding any different medium, text file, database, etc. goes to descendant LoggerProvider classes.

We are going to apply the above idea and create a FileLoggerProvider class.

Base Classes and Accessories

LogEntry represents the information of a log entry. The Logger creates an instance of this class when its Log() method is called, fills the properties and then passes that information to the provider calling WriteLog().

public class LogEntry
{ 
    public LogEntry()
    {
        TimeStampUtc = DateTime.UtcNow;
        UserName = Environment.UserName;
    }
 
    static public readonly string StaticHostName = System.Net.Dns.GetHostName();
 
    public string UserName { get; private set; }
    public string HostName { get { return StaticHostName; } }
    public DateTime TimeStampUtc { get; private set; }
    public string Category { get; set; }
    public LogLevel Level { get; set; }
    public string Text { get; set; }
    public Exception Exception { get; set; }
    public EventId EventId { get; set; }
    public object State { get; set; }
    public string StateText { get; set; }
    public Dictionary<string, object> StateProperties { get; set; }
    public List<LogScopeInfo> Scopes { get; set; }
}

LogScopeInfo represents Scope information regarding a LogEntry.

public class LogScopeInfo
{ 
    public LogScopeInfo()
    {
    }
 
    public string Text { get; set; }
    public Dictionary<string, object> Properties { get; set; }
}

Logger represents an object that handles log information. This class does not save log information in a medium. Its sole responsibility is to create a LogEntry. It then fills the properties of that instance, and then passes it to the associated logger provider for further processing.

internal class Logger : ILogger
{ 
    public Logger(LoggerProvider Provider, string Category)
    {
        this.Provider = Provider;
        this.Category = Category;
    }
 
    IDisposable ILogger.BeginScope<TState>(TState state)
    {
        return Provider.ScopeProvider.Push(state);
    }
 
    bool ILogger.IsEnabled(LogLevel logLevel)
    {
        return Provider.IsEnabled(logLevel);
    }
 
    void ILogger.Log<TState>(LogLevel logLevel, EventId eventId, 
        TState state, Exception exception, Func<TState, Exception, string> formatter)
    {
        if ((this as ILogger).IsEnabled(logLevel))
        { 
            LogEntry Info = new LogEntry();
            Info.Category = this.Category;
            Info.Level = logLevel;
            // well, the passed default formatter function 
            // does not take the exception into account
            // SEE: https://github.com/aspnet/Extensions/blob/master/src/
              Logging/Logging.Abstractions/src/LoggerExtensions.cs
            Info.Text = exception?.Message ?? state.ToString(); // formatter(state, exception)
            Info.Exception = exception;
            Info.EventId = eventId;
            Info.State = state;
 
            // well, you never know what it really is
            if (state is string)   
            {
                Info.StateText = state.ToString();
            }
            // in case we have to do with a message template, 
            // let's get the keys and values (for Structured Logging providers)
            // SEE: https://docs.microsoft.com/en-us/aspnet/core/
            // fundamentals/logging#log-message-template
            // SEE: https://softwareengineering.stackexchange.com/
            // questions/312197/benefits-of-structured-logging-vs-basic-logging
            else if (state is IEnumerable<KeyValuePair<string, object>> Properties)
            {
                Info.StateProperties = new Dictionary<string, object>();
 
                foreach (KeyValuePair<string, object> item in Properties)
                {
                    Info.StateProperties[item.Key] = item.Value;
                }
            }
 
            // gather info about scope(s), if any
            if (Provider.ScopeProvider != null)
            {
                Provider.ScopeProvider.ForEachScope((value, loggingProps) =>
                {
                    if (Info.Scopes == null)
                        Info.Scopes = new List<LogScopeInfo>();
 
                    LogScopeInfo Scope = new LogScopeInfo();
                    Info.Scopes.Add(Scope);
 
                    if (value is string)
                    {
                        Scope.Text = value.ToString();
                    }
                    else if (value is IEnumerable<KeyValuePair<string, object>> props)
                    {
                        if (Scope.Properties == null)
                            Scope.Properties = new Dictionary<string, object>();
 
                        foreach (var pair in props)
                        {
                            Scope.Properties[pair.Key] = pair.Value;
                        }
                    }
                },
                state); 
            }
 
            Provider.WriteLog(Info); 
        }
    }
 
    public LoggerProvider Provider { get; private set; }
    public string Category { get; private set; }
}

LoggerProvider is an abstract base logger provider class. A logger provider essentially represents the medium where log information is saved or displayed. This class may serve as base class in writing a file or database logger provider.

public abstract class LoggerProvider : IDisposable, ILoggerProvider, ISupportExternalScope
{
    ConcurrentDictionary<string, Logger> loggers = new ConcurrentDictionary<string, Logger>();
    IExternalScopeProvider fScopeProvider;
    protected IDisposable SettingsChangeToken;
 
    void ISupportExternalScope.SetScopeProvider(IExternalScopeProvider scopeProvider)
    {
        fScopeProvider = scopeProvider;
    }
 
    ILogger ILoggerProvider.CreateLogger(string Category)
    {
        return loggers.GetOrAdd(Category,
        (category) => {
            return new Logger(this, category);
        });
    }
 
    void IDisposable.Dispose()
    {
        if (!this.IsDisposed)
        {
            try
            {
                Dispose(true);
            }
            catch
            {
            }
 
            this.IsDisposed = true;
            GC.SuppressFinalize(this);  // instructs GC not bother to call the destructor   
        }
    }
 
    protected virtual void Dispose(bool disposing)
    {
        if (SettingsChangeToken != null)
        {
            SettingsChangeToken.Dispose();
            SettingsChangeToken = null;
        }
    } 
 
    public LoggerProvider()
    {
    }
 
    ~LoggerProvider()
    {
        if (!this.IsDisposed)
        {
            Dispose(false);
        }
    }
 
    public abstract bool IsEnabled(LogLevel logLevel);
 
    public abstract void WriteLog(LogEntry Info);
 
    internal IExternalScopeProvider ScopeProvider
    {
        get
        {
            if (fScopeProvider == null)
                fScopeProvider = new LoggerExternalScopeProvider();
            return fScopeProvider;
        }
    }
 
    public bool IsDisposed { get; protected set; }
}

A FileLoggerProvider Concrete Class and Its Accessories

FileLoggerOptions is the Options class for the file logger.

public class FileLoggerOptions
{
    string fFolder;
    int fMaxFileSizeInMB;
    int fRetainPolicyFileCount;
 
    public FileLoggerOptions()
    {
    }
 
    public LogLevel LogLevel { get; set; } = Microsoft.Extensions.Logging.LogLevel.Information;
 
    public string Folder
    {
        get { return !string.IsNullOrWhiteSpace(fFolder) ? 
              fFolder : System.IO.Path.GetDirectoryName(this.GetType().Assembly.Location); }
        set { fFolder = value; }
    }
 
    public int MaxFileSizeInMB
    {
        get { return fMaxFileSizeInMB > 0 ? fMaxFileSizeInMB : 2; }
        set { fMaxFileSizeInMB = value; }
    }
 
    public int RetainPolicyFileCount
    {
        get { return fRetainPolicyFileCount < 5 ? 5 : fRetainPolicyFileCount; }
        set { fRetainPolicyFileCount = value; }
    }
}

There are two ways to configure file logger options:

1. using the ConfigureLogging() in Program.cs and by calling the second version of the AddFileLogger(), the one with options delegate, or
2. using the appsettings.json file.

1. ConfigureLogging()

.ConfigureLogging(logging =>
{
    logging.ClearProviders();
    // logging.AddFileLogger(); 
    logging.AddFileLogger(options => {
        options.MaxFileSizeInMB = 5;
    });
}) 

2. appsettings.json File

"Logging": {
    "LogLevel": {
      "Default": "Warning"
    },
    "File": {
      "LogLevel": "Debug",
      "MaxFileSizeInMB": 5
    }
  }, 

FileLoggerOptionsSetup configures a FileLoggerOptions instance by using ConfigurationBinder.Bind() against an IConfiguration. FileLoggerOptionsSetup class essentially binds a FileLoggerOptions instance with a section in the appsettings.json file. This is a crucial connection especially if we want to be notified for changes to appsettings.json regarding our logger provider. Don't worry, it’s just plumbing.

internal class FileLoggerOptionsSetup : ConfigureFromConfigurationOptions<FileLoggerOptions>
{
    public FileLoggerOptionsSetup(ILoggerProviderConfiguration<FileLoggerProvider> 
                                  providerConfiguration)
        : base(providerConfiguration.Configuration)
    {
    }
}

FileLoggerProvider is a logger provider that writes log entries to a text file. “File” is the provider alias of this provider and can be used in the Logging section of the appsettings.json, see above.

FileLoggerProvider does some interesting things.

It writes each LogEntry passed to it by a Logger to a text file, with extension *.log, in a folder specified in FileLoggerOptions (or in the appsettings.json where FileLoggerOptions reads from).

Actually, a Logger calls the abstract LoggerProvider.WriteLog(LogEntry Info). The overridden FileLoggerProvider.WriteLog(LogEntry Info) does not block because it pushes the passed LogEntry to a thread safe queue. Later, a thread examines that queue, pops the LogEntry and writes it to a text file. It’s an asynchronous operation.

FileLoggerProvider also cares about the retention policy of the log files it creates. It does that respecting some retention policy related settings, of the FileLoggerOptions.

FileLoggerProvider, thanks to FileLoggerOptionsSetup shown above and the IOptionsMonitor passed to its constructor, is notified about changes happened to appsettings.json file and responds accordingly.

[Microsoft.Extensions.Logging.ProviderAlias("File")]
public class FileLoggerProvider : LoggerProvider
{ 
    bool Terminated;
    int Counter = 0;
    string FilePath;
    Dictionary<string, int> Lengths = new Dictionary<string, int>();
    
    ConcurrentQueue<LogEntry> InfoQueue = new ConcurrentQueue<LogEntry>();
 
    void ApplyRetainPolicy()
    {
        FileInfo FI;
        try
        {
            List<FileInfo> FileList = new DirectoryInfo(Settings.Folder)
            .GetFiles("*.log", SearchOption.TopDirectoryOnly)
            .OrderBy(fi => fi.CreationTime)
            .ToList();
 
            while (FileList.Count >= Settings.RetainPolicyFileCount)
            {
                FI = FileList.First();
                FI.Delete();
                FileList.Remove(FI);
            }
        }
        catch
        {
        } 
    }
 
    void WriteLine(string Text)
    {
        // check the file size after any 100 writes
        Counter++;
        if (Counter % 100 == 0)
        {
            FileInfo FI = new FileInfo(FilePath);
            if (FI.Length > (1024 * 1024 * Settings.MaxFileSizeInMB))
            {                   
                BeginFile();
            }
        }
 
        File.AppendAllText(FilePath, Text);
    }
 
    string Pad(string Text, int MaxLength)
    {
        if (string.IsNullOrWhiteSpace(Text))
            return "".PadRight(MaxLength);
 
        if (Text.Length > MaxLength)
            return Text.Substring(0, MaxLength);
 
        return Text.PadRight(MaxLength);
    }
 
    void PrepareLengths()
    {
        // prepare the lengs table
        Lengths["Time"] = 24;
        Lengths["Host"] = 16;
        Lengths["User"] = 16;
        Lengths["Level"] = 14;
        Lengths["EventId"] = 32;
        Lengths["Category"] = 92;
        Lengths["Scope"] = 64;
    }
 
    void BeginFile()
    {
        Directory.CreateDirectory(Settings.Folder);
        FilePath = Path.Combine(Settings.Folder, LogEntry.StaticHostName + 
                   "-" + DateTime.Now.ToString("yyyyMMdd-HHmm") + ".log");
 
        // titles
        StringBuilder SB = new StringBuilder();
        SB.Append(Pad("Time", Lengths["Time"]));
        SB.Append(Pad("Host", Lengths["Host"]));
        SB.Append(Pad("User", Lengths["User"]));
        SB.Append(Pad("Level", Lengths["Level"]));
        SB.Append(Pad("EventId", Lengths["EventId"]));
        SB.Append(Pad("Category", Lengths["Category"]));
        SB.Append(Pad("Scope", Lengths["Scope"]));
        SB.AppendLine("Text");
 
        File.WriteAllText(FilePath, SB.ToString());
 
        ApplyRetainPolicy();
    }
 
    void WriteLogLine()
    {
        LogEntry Info = null;
        if (InfoQueue.TryDequeue(out Info))
        {
            string S;
            StringBuilder SB = new StringBuilder();
            SB.Append(Pad(Info.TimeStampUtc.ToLocalTime().ToString("yyyy-MM-dd HH:mm:ss.ff"), 
                      Lengths["Time"]));
            SB.Append(Pad(Info.HostName, Lengths["Host"]));
            SB.Append(Pad(Info.UserName, Lengths["User"]));
            SB.Append(Pad(Info.Level.ToString(), Lengths["Level"]));
            SB.Append(Pad(Info.EventId != null ? Info.EventId.ToString() : "", 
                      Lengths["EventId"]));
            SB.Append(Pad(Info.Category, Lengths["Category"]));
 
            S = "";
            if (Info.Scopes != null && Info.Scopes.Count > 0)
            {
                LogScopeInfo SI = Info.Scopes.Last();
                if (!string.IsNullOrWhiteSpace(SI.Text))
                {
                    S = SI.Text;
                }
                else
                {
                }
            }
            SB.Append(Pad(S, Lengths["Scope"]));
 
            string Text = Info.Text;
 
            /* writing properties is too much for a text file logger
            if (Info.StateProperties != null && Info.StateProperties.Count > 0)
            {
                Text = Text + " Properties = " + 
                       Newtonsoft.Json.JsonConvert.SerializeObject(Info.StateProperties);
            }                
                */
 
            if (!string.IsNullOrWhiteSpace(Text))
            {
                SB.Append(Text.Replace("\r\n", " ").Replace("\r", " ").Replace("\n", " "));
            }
 
            SB.AppendLine();
            WriteLine(SB.ToString());
        } 
    }
    void ThreadProc()
    {
        Task.Run(() => {
 
            while (!Terminated)
            {
                try
                {
                    WriteLogLine();
                    System.Threading.Thread.Sleep(100);
                }
                catch // (Exception ex)
                {
                }
            } 
        });
    }
 
    protected override void Dispose(bool disposing)
    {
        Terminated = true;
        base.Dispose(disposing);
    }
 
 
    public FileLoggerProvider(IOptionsMonitor<FileLoggerOptions> Settings)
        : this(Settings.CurrentValue)
    {  
        // https://docs.microsoft.com/en-us/aspnet/core/fundamentals/change-tokens
        SettingsChangeToken = Settings.OnChange(settings => {      
            this.Settings = settings;                  
        });
    }
 
    public FileLoggerProvider(FileLoggerOptions Settings)
    {
        PrepareLengths();
        this.Settings = Settings;
 
        // create the first file
        BeginFile();
 
        ThreadProc();
    } 
 
    public override bool IsEnabled(LogLevel logLevel)
    {
        bool Result = logLevel != LogLevel.None
            && this.Settings.LogLevel != LogLevel.None
            && Convert.ToInt32(logLevel) >= Convert.ToInt32(this.Settings.LogLevel);
 
        return Result;
    }
 
    public override void WriteLog(LogEntry Info)
    {
        InfoQueue.Enqueue(Info);
    } 
 
    internal FileLoggerOptions Settings { get; private set; } 
}

FileLoggerExtensions contains methods used in adding the file logger provider, aliased as 'File', to the available services as singleton and binds the file logger options class to the 'File' section of the appsettings.json file.

As you can see, there are no ILoggerFactory extensions, only ILoggingBuilder ones. Which means that you should register the file logger provider in the Program.cs, as shown above, and not in the Startup class. Inspecting AspNet.Core code, regarding similar extension methods, it seems that registering a logger provider through ILoggerFactory is already obsolete.

static public class FileLoggerExtensions
{ 
    static public ILoggingBuilder AddFileLogger(this ILoggingBuilder builder)
    {
        builder.AddConfiguration();
 
        builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<ILoggerProvider, 
                                          FileLoggerProvider>());
        builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton
           <IConfigureOptions<FileLoggerOptions>, FileLoggerOptionsSetup>());
        builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton
           <IOptionsChangeTokenSource<FileLoggerOptions>, 
           LoggerProviderOptionsChangeTokenSource<FileLoggerOptions, FileLoggerProvider>>());
        return builder;
    }
 
    static public ILoggingBuilder AddFileLogger
           (this ILoggingBuilder builder, Action<FileLoggerOptions> configure)
    {
        if (configure == null)
        {
            throw new ArgumentNullException(nameof(configure));
        }
 
        builder.AddFileLogger();
        builder.Services.Configure(configure);
 
        return builder;
    }
}

Epilogue

It was a long post but I hope you enjoy it. Using the above, you may write your own logger provider, based on the abstract LoggerProvider for any medium you like. If you do it, just drop me a notice.

The above code is almost the full source code. I’ve just deleted the comments for clarity.

Tested on:

• Visual Studio 2019 Community
• ASP.NET Core SDK 2.2.2