.NET Core Microservices in the Enterprise: Logger as Microservice

• Download solution - 36 KB

Background

The Traditional Approach

In a monolithic application or in a world of independent application that comes from different vendors, the pattern to logger information in the system is mainly using a logger program that is incorporated directly in the application and it is configured individually in each application.

But in a Microservices ecosystem, we can have dozens of microservices that each need a complex configuration and installation to logger using the standard approach, that gives a bunch of problems to resolve to maintain the system. Let's take a look at some of them.

• You need to individually configure each service with the selected logger framework.
• Each microservice needs access to the DB where your log is stored.
• If you want to update your logger, you need to update each microservice individually.

On the other hand, microservices in the enterprise are tending more and more to be installed in the cloud, with some container, even in a different operating system.

The other concern is about security, you need to give access to your DB system to each service, in case you want to logger to the database.

Logger as Service Approach

Then if microservices are good, why not use the same concept to create our logger system? Using logger as a service gives you a lot of advantage, first the logger framework is only installed in one microservice, and the rest only needs to send the information to the logger using some network protocol minimizing the exposure of the DB resources.

This architecture also gives the following advantages:

First thing, the update of the service only needs to be done in one service: The logger microservice, if you don’t change the interface with the logger client, you can update the logger framework, or even change for other in only one service without touching the rest of your system.

The second big win is that you only need to configure the access to the DB in this sole service, not in all of them.

The other good advantage here is that you can hide your DB very deep in your intranet, the logger service needs to be open to the microservices inside the company intranet, given a level of security, but your database will be in a subnet completely isolated from the services that use the logger microservices.

And, at the end, but also a big value, because the logger is a service, you don’t need to take care about over what the client is running, for you, it is someone that accesses your service.

Implementing a Logger as Service

Alongside the general strategy of creating a logger as service, we can also introduce some design decision that allows our service to be more flexible than the conventional service framework.

Each Microservice Logger in Its Individual Table

Maybe you are asking how the logger classified the different entries from different services in the DB, in this implementation, we decided to logger each microservice in a different table, to manage this, the logger request has a field with a specific token per service.

The token is used to get the name of the table, the information to link this resides in the appsettings.json of our application, or in a web.config if you are working with a previous version of .NET). If the token exists, the table is selected and the logger writes the information in the table.

Control of Level of Error

In our approach, we maintain the service as simple as possible, the implementation that we use as example, does not manage the level of error, then we leverage it in a client .dll (that can be implemented as NuGet package) and you can create for your specific necessity the management of the level of error and when the information should be moved to the Logger or rejected depending of the level of the setting.

This introduces some level of configuration in the client service, but it is not a big deal to manage a setting string in the configuration of the application.

Using the Code

We create an implementation of a logger as service that can be download as open source in Github. We explain here the fundamental part of the code. At the end, the logger that we create is a simple service that is able to log in a SQL Server database. We create this as an example, but the code is capable of working in the production environment if it is required.

The code for the service basically has the following structure:

• Service to Log the information
• Internal Log to log problems with it
• Configuration Settings to add the pair Token - Table for each client to be used
• Access to the database that we don't discuss here because it is trivial

The service is very simple:

/// <summary>
/// Enter the info in the log.
/// if error returns a http 500 internal server error 
/// and an HttpError based in the exception information
/// </summary>
/// <param name="request">
/// The information to be logged
/// </param>
/// <returns>null is ok</returns>
/// <exception cref="HttpError">If error return a httpError Class</exception>
[HttpPost("LogEntryAsync")]
[Consumes(MediaTypeNames.Application.Json)]
[ProducesResponseType(typeof(ErrorManager), StatusCodes.Status500InternalServerError)]
[ProducesResponseType(typeof(ErrorManager), StatusCodes.Status400BadRequest)]
public async Task<ActionResult> PostLogEntryAsync([FromBody] LogRequest request)
{
      await bo.PostLogEntryAsync(request);
      return Ok();
}

And the object to Log is the following:

/// <summary>
/// The Log Request Class
/// </summary>
public class LogRequest
{
     /// <summary>
     /// Used to link the error to the user with the log 
     /// logged
     /// </summary>
     [DataType(DataType.DateTime)]
     public DateTime? CreatedDate { get; set; }
 
     /// <summary>
     /// The name of the machine where log is emitted
     /// </summary>
     [Required(AllowEmptyStrings = false)]
     [StringLength(maximumLength: 255, MinimumLength = 5)]
     public string MachineName { get; set; }
 
     /// <summary>
     /// The register Level
     /// </summary>
     [Required]
     [StringLength(50, MinimumLength = 5)]
     public string Level { get; set; }
 
     /// <summary>
     /// The name of the logger, 
     /// this is the token used to localized the table
     /// </summary>
     [Required(AllowEmptyStrings = false)]
     [StringLength(maximumLength: 255, MinimumLength = 5)]
     public string Logger { get; set; }
 
     /// <summary>
     /// The message to be show
     /// </summary>
     [Required(AllowEmptyStrings = true)]
     public string Message { get; set; }
 
     /// <summary>
     /// The exception Message to be logged
     /// </summary>
     public string Exception { get; set; }
}

Interesting things here in the request object are:

• CreatedTime: is optional. Why? We can leave to the log to enter the date and time that the log happens, but also you can set the date time from your service. This is specially useful when you want to link a friendly message to the user with the technical information stored in the logger.
• MachineName: This field allows you to enter the name of the machine that sent the log request. This is very useful in microservices when you can have different instances of the microservices, to know exactly what machine is sent the information.
• Level: This is to enter the level of the logged information. Typically, values are ERROR FATAL, INFO, etc. This is an open field. The idea is that you can use a client that configures the level of error. When the user of a NuGet package for example, you can set the level of the error in your particular client and make it different for each of them.
• Logger: This is the field that sent the token to identify the client that uses the service. This is used in conjunction with the appSettings.json to determine in what table you want to log the information.
• Message and Exception: Use these two fields to enter the information that you want to logger.

The Internal Log

This is used to log in its proper table error because false token bad formed request object, missing table in the DB, etc. The internal logger has its proper settings configuration.

The internal logger also is done in a form that prevents error looping in the application.

The parameter to config the internal logger is simple as you can see in the following code:

{
  "InternalLogSettings": {
    "Table": "ServeLog",
    "_Comment": "Log Level Values: DEBUG, ERROR",
    "Level": "DEBUG",
    "_Comment1": "Time Zone Examples: UTC, Eastern Standard Time",
    "TimeZone": "UTC"
  }
}

You can choose between different time zone to enter the log annotation.

And the other important thing here is how the program matches the token with the Table name. That is easily done using the appSettings.json file.

{
  "Tables": {
    "TESTTOKEN": "TestLog",
    "YOURTOKEN": "YourTableLog" 
  }
}

The first entry is for the internal logger, and you can add more entries to logger in different tables as many clients as you have. In the example, we add one extra entry.

Points of Interest

• Conventional logger that is installed in each service can be a pain when you have an eco system of several dozen of microservices. You need to configure each of then individually, also you got a lot of DB work to do in case you are logger to DB.
• On the other hand, if you use create a microservice whose function is logger, the logger is the only other microservice in your environment, that can be accessed through HTTP protocol like other service. Problems as configuration and access to the DB can be heavily simplified with this configuration.
• We explain here an example that allows you logger each service in a different Table, and also personalize the level of your logger using a software client.
• You can see a functional ready for production service in our example. You can download it from GIT, or contact us in the comments section below. The code is offered as an open source project. 
• You can see a video related to this article at https://youtu.be/3O1DymP8XOo.
• A copy of the example is provided here.

History

• 21^st August, 2021
  • First version
  • Added missing link and code