Introduction
This guide is about how to create a service monitor application, but what is it? In simple words: it's an application that allows to monitor services in a network and save results from monitoring in a database, SQL Server for this case.
I know there are a lot of tools that can provide this feature, also there are better tools that money can buy but my intention with this guide is to show how to use .NET Core power to build an application that developers can extend for custom requirements.
The basic idea is this: have a process to run in infinite ways to monitor hosts, databases and APIs; save monitoring results in SQL Server database, then we can build a fancy UI to end-user and show status for each service, we can have a lot of targets to monitoring but it's better to allow users to subscribe for specific services and not all; for example DBAs need to watch database servers not APIs, developers need to watch development databases and APIs, etc.
Also think about having big monitors in your development room and watching the status for your services and in the best of cases, have charts. :)
One special feature could be to have a notification service to send messages for all administrators in case one or more services fail, in this context, service means a target such as host, database, API.
In this guide, we'll work with monitoring the following services:
Name | Description |
Host | Ping an existing host |
Database | Open and close the connection for existing database |
RESTful API | Consume one action from existing API |
Background
As we said before, we'll create an application to monitoring existing targets (hosts, databases, APIs), so we need to have basic knowledge about these concepts.
Hosts will be monitoring with ping action, so we'll add networking related packages to perform this action.
Databases will be monitoring with open and close connections, don't use integrated security because you'll need to impersonate your service monitor process with your credentials, so in that case, it's better to have a specific user to connect with database and only that action to avoid hacking.
RESTful APIs will be monitoring with REST client to target an action that returns simple JSON.
Database
Inside of repository, there is a directory with name \Resources\Database and this directory contains related database files, please make sure to run the following files in this order:
File Name | Description |
00 - Database.sql | Database definition |
01 - Tables.sql | Tables definition |
02 - Constraints.sql | Constraints (primary keys, foreign keys and uniques) |
03 - Rows.sql | Initial data |
We can found scripts for database here.
Tables Description Table | Description |
EnvironmentCategory | Contains all categories for environments: development, qa and production |
ServiceCategory | Contains all categories for services: database, rest API, server, URL and web service |
Service | Contains all services definitions |
ServiceWatcher | Contains all components from C# side to perform watch operations |
ServiceEnvironment | Contains the relation for service and environment, for example we can define a service named FinanceService with different environments: development, qa and production |
ServiceEnvironmentStatus | Contains the status for each service per environment |
ServiceEnvironmentStatusLog | Contains the details for each service environment status |
Owner | Contains the user list for application that represents all owners |
ServiceOwner | Contains the relation between service and owner |
User | Contains all users to watching services |
ServiceUser | Contains the relation between service and user |
Please don't forget We are working with a solution that runs on local machine, there is a sample API in resources directory to performing tests, but you need change the connection string and add your services according to your context.
Also I don't recommend to expose real connection strings in ServiceEnvironment table, please request to your DBA a single user can only perform open connection for target database, in case the security of databases be a task from your side, create specific users to perform ONLY open connection to databases and prevent exposing sensitive information.
.NET Core Solution
Now we need to define the projects for this solution to get a clear concept about project's scope:
Project Name | Type | Description |
ServiceMonitor.Core | Class Library | Contains all definitions related to database storage |
ServiceMonitor.Common | Class Library | Contains common definitions for ServiceMonitor project such as watchers, serializer and clients (REST) |
ServiceMonitor.WebAPI | Web API | Contains Web API Controllers to read and write information about monitoring |
ServiceMonitor | Console Application | Contains process to monitoring all services |
ServiceMonitor.Core
This project contains all definitions for entities and database access, so we need to add the following packages for project:
Name | Version | Description |
Microsoft.EntityFrameworkCore.SqlServer | Latest version | Provides access for SQL Server through EF Core |
This project contains three layers: Business Logic, Database Access and Entities; please take a look at the article EF Core for Entreprise to get a better understanding about this project and layers.
DashboardService
class code:
using System;
using System.Threading.Tasks;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using ServiceMonitor.Core.BusinessLayer.Contracts;
using ServiceMonitor.Core.BusinessLayer.Responses;
using ServiceMonitor.Core.DataLayer;
using ServiceMonitor.Core.DataLayer.DataContracts;
using ServiceMonitor.Core.EntityLayer;
namespace ServiceMonitor.Core.BusinessLayer
{
public class DashboardService : Service, IDashboardService
{
public DashboardService(ILogger<DashboardService> logger, ServiceMonitorDbContext dbContext)
: base(logger, dbContext)
{
}
public async Task<IListResponse<ServiceWatcherItemDto>> GetActiveServiceWatcherItemsAsync()
{
Logger?.LogDebug("'{0}' has been invoked", nameof(GetActiveServiceWatcherItemsAsync));
var response = new ListResponse<ServiceWatcherItemDto>();
try
{
response.Model = await DbContext.GetActiveServiceWatcherItems().ToListAsync();
Logger?.LogInformation("The service watch items were loaded successfully");
}
catch (Exception ex)
{
response.SetError(Logger, nameof(GetActiveServiceWatcherItemsAsync), ex);
}
return response;
}
public async Task<IListResponse<ServiceStatusDetailDto>> GetServiceStatusesAsync(string userName)
{
Logger?.LogDebug("'{0}' has been invoked", nameof(GetServiceStatusesAsync));
var response = new ListResponse<ServiceStatusDetailDto>();
try
{
var user = await DbContext.GetUserAsync(userName);
if (user == null)
{
Logger?.LogInformation("There isn't data for user '{0}'", userName);
return new ListResponse<ServiceStatusDetailDto>();
}
else
{
response.Model = await DbContext.GetServiceStatuses(user).ToListAsync();
Logger?.LogInformation("The service status details for '{0}' user were loaded successfully", userName);
}
}
catch (Exception ex)
{
response.SetError(Logger, nameof(GetServiceStatusesAsync), ex);
}
return response;
}
public async Task<ISingleResponse<ServiceEnvironmentStatus>> GetServiceStatusAsync(ServiceEnvironmentStatus entity)
{
Logger?.LogDebug("'{0}' has been invoked", nameof(GetServiceStatusAsync));
var response = new SingleResponse<ServiceEnvironmentStatus>();
try
{
response.Model = await DbContext.GetServiceEnvironmentStatusAsync(entity);
}
catch (Exception ex)
{
response.SetError(Logger, nameof(GetServiceStatusAsync), ex);
}
return response;
}
}
}
ServiceMonitor.Common
Contracts
IWatcher
IWatchResponse
ISerializer
IWatcher
interface code:
using System.Threading.Tasks;
namespace ServiceMonitor.Common.Contracts
{
public interface IWatcher
{
string ActionName { get; }
Task<WatchResponse> WatchAsync(WatcherParameter parameter);
}
}
IWatchResponse
interface code:
namespace ServiceMonitor.Common.Contracts
{
public interface IWatchResponse
{
bool Success { get; set; }
string Message { get; set; }
string StackTrace { get; set; }
}
}
ISerializer
interface code:
namespace ServiceMonitor.Common.Contracts
{
public interface ISerializer
{
string Serialize<T>(T obj);
T Deserialze<T>(string source);
}
}
Watchers
These are the implementations:
DatabaseWatcher
HttpRequestWatcher
PingWatcher
DatabaseWatcher
class code:
using System;
using System.Data.SqlClient;
using System.Threading.Tasks;
using ServiceMonitor.Common.Contracts;
namespace ServiceMonitor.Common
{
public class DatabaseWatcher : IWatcher
{
public string ActionName
=> "OpenDatabaseConnection";
public async Task<WatchResponse> WatchAsync(WatcherParameter parameter)
{
var response = new WatchResponse();
using (var connection = new SqlConnection(parameter.Values["ConnectionString"]))
{
try
{
await connection.OpenAsync();
response.Success = true;
}
catch (Exception ex)
{
response.Success = false;
response.Message = ex.Message;
response.StackTrace = ex.ToString();
}
}
return response;
}
}
}
HttpWebRequestWatcher
class code:
using System;
using System.Threading.Tasks;
using ServiceMonitor.Common.Contracts;
namespace ServiceMonitor.Common
{
public class HttpRequestWatcher : IWatcher
{
public string ActionName
=> "HttpRequest";
public async Task<WatchResponse> WatchAsync(WatcherParameter parameter)
{
var response = new WatchResponse();
try
{
var restClient = new RestClient();
await restClient.GetAsync(parameter.Values["Url"]);
response.Success = true;
}
catch (Exception ex)
{
response.Success = false;
response.Message = ex.Message;
response.StackTrace = ex.ToString();
}
return response;
}
}
}
PingWatcher
class code:
using System.Net.NetworkInformation;
using System.Threading.Tasks;
using ServiceMonitor.Common.Contracts;
namespace ServiceMonitor.Common
{
public class PingWatcher : IWatcher
{
public string ActionName
=> "Ping";
public async Task<WatchResponse> WatchAsync(WatcherParameter parameter)
{
var ping = new Ping();
var reply = await ping.SendPingAsync(parameter.Values["Address"]);
return new WatchResponse
{
Success = reply.Status == IPStatus.Success ? true : false
};
}
}
}
ServiceMonitor.WebAPI
This project represents RESTful API for service monitor, so We'll have two controllers: DashboardController
and AdministrationController
. Dashboard has all operations related to end user results and Administration contains all operations related to save information (create, edit and delete).
Dashboard
DashboardController
class code:
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;
using ServiceMonitor.Core.BusinessLayer.Contracts;
using ServiceMonitor.WebAPI.Responses;
namespace ServiceMonitor.WebAPI.Controllers
{
#pragma warning disable CS1591
[Route("api/v1/[controller]")]
[ApiController]
public class DashboardController : ControllerBase
{
protected readonly ILogger Logger;
protected readonly IDashboardService Service;
public DashboardController(ILogger<DashboardController> logger, IDashboardService service)
{
Logger = logger;
Service = service;
}
#pragma warning restore CS1591
[HttpGet("ServiceWatcherItem")]
[ProducesResponseType(200)]
[ProducesResponseType(500)]
public async Task<IActionResult> GetServiceWatcherItemsAsync()
{
Logger?.LogDebug("'{0}' has been invoked", nameof(GetServiceWatcherItemsAsync));
var response = await Service.GetActiveServiceWatcherItemsAsync();
return response.ToHttpResponse();
}
}
}
Administration
AdministrationController
class code:
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;
using ServiceMonitor.Core.BusinessLayer.Contracts;
using ServiceMonitor.WebAPI.Responses;
namespace ServiceMonitor.WebAPI.Controllers
{
#pragma warning disable CS1591
[Route("api/v1/[controller]")]
[ApiController]
public class DashboardController : ControllerBase
{
protected readonly ILogger Logger;
protected readonly IDashboardService Service;
public DashboardController(ILogger<DashboardController> logger, IDashboardService service)
{
Logger = logger;
Service = service;
}
#pragma warning restore CS1591
[HttpGet("ServiceWatcherItem")]
[ProducesResponseType(200)]
[ProducesResponseType(500)]
public async Task<IActionResult> GetServiceWatcherItemsAsync()
{
Logger?.LogDebug("'{0}' has been invoked", nameof(GetServiceWatcherItemsAsync));
var response = await Service.GetActiveServiceWatcherItemsAsync();
return response.ToHttpResponse();
}
}
}
ServiceMonitor
This project contains all objects for Service Monitor Client, in this project, we have added the package Newtonsoft.Json
for JSON serialization, in ServiceMonitor.Common
there is an interface with name ISerializer
and that's because I don't want to force to use a specific serializer, you can change that at this level. :)
ServiceMonitorSerializer
class code:
using Newtonsoft.Json;
using ServiceMonitor.Common.Contracts;
namespace ServiceMonitor
{
public class ServiceMonitorSerializer : ISerializer
{
public string Serialize<T>(T obj)
=> JsonConvert.SerializeObject(obj);
public T Deserialze<T>(string source)
=> JsonConvert.DeserializeObject<T>(source);
}
}
Next, we'll be working on MonitorController
class, in this class, we'll perform all watching operations and save all results in database throught AdministrationController
in Service Monitor API.
MonitorController
class code:
using System;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
using ServiceMonitor.Clients;
using ServiceMonitor.Clients.Models;
using ServiceMonitor.Common;
using ServiceMonitor.Common.Contracts;
namespace ServiceMonitor
{
public class MonitorController
{
public MonitorController(ILogger logger, IWatcher watcher, IServiceMonitorWebAPIClient client, AppSettings appSettings)
{
Logger = logger;
Watcher = watcher;
Client = client;
AppSettings = appSettings;
}
public ILogger Logger { get; }
public IWatcher Watcher { get; }
public IServiceMonitorWebAPIClient Client { get; }
public AppSettings AppSettings { get; }
public async Task ProcessAsync(ServiceWatchItem item)
{
while (true)
{
try
{
Logger?.LogTrace("{0} - Watching '{1}' for '{2}' environment", DateTime.Now, item.ServiceName, item.Environment);
var watchResponse = await Watcher.WatchAsync(new WatcherParameter(item.ToDictionary()));
if (watchResponse.Success)
Logger?.LogInformation(" Success watch for '{0}' in '{1}' environment", item.ServiceName, item.Environment);
else
Logger?.LogError(" Failed watch for '{0}' in '{1}' environment", item.ServiceName, item.Environment);
var serviceStatusLog = new ServiceStatusLogRequest
{
ServiceID = item.ServiceID,
ServiceEnvironmentID = item.ServiceEnvironmentID,
Target = item.ServiceName,
ActionName = Watcher.ActionName,
Success = watchResponse.Success,
Message = watchResponse.Message,
StackTrace = watchResponse.StackTrace
};
try
{
await Client.PostServiceEnvironmentStatusLog(serviceStatusLog);
}
catch (Exception ex)
{
Logger?.LogCritical(" Error on saving watch response ({0}): '{1}'", item.ServiceName, ex.Message);
}
}
catch (Exception ex)
{
Logger?.LogCritical(" Error watching service: '{0}': '{1}'", item.ServiceName, ex.Message);
}
Thread.Sleep(item.Interval ?? AppSettings.DelayTime);
}
}
}
}
Before running console application, make sure about these aspects:
ServiceMonitor
database is available ServiceMonitor
database has the information for service categories, services, service watchers and users ServiceMonitor
API is available
We can check the returned value for url api/v1/Dashboard/ServiceWatcherItems:
{
"message":null,
"didError":false,
"errorMessage":null,
"model":[
{
"serviceID":1,
"serviceEnvironmentID":1,
"environment":"Development",
"serviceName":"Northwind Database",
"interval":15000,
"url":null,
"address":null,
"connectionString":"server=(local);database=Northwind;user id=johnd;password=SqlServer2017$",
"typeName":"ServiceMonitor.Common.DatabaseWatcher, ServiceMonitor.Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
},
{
"serviceID":2,
"serviceEnvironmentID":3,
"environment":"Development",
"serviceName":"DNS",
"interval":3000,
"url":null,
"address":"192.168.1.1",
"connectionString":null,
"typeName":"ServiceMonitor.Common.PingWatcher, ServiceMonitor.Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
},
{
"serviceID":3,
"serviceEnvironmentID":4,
"environment":"Development",
"serviceName":"Sample API",
"interval":5000,
"url":"http://localhost:5612/api/values",
"address":null,
"connectionString":null,
"typeName":"ServiceMonitor.Common.HttpWebRequestWatcher, ServiceMonitor.Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
}
]
}
As we can see, API returns all services for DefaultUser
, please remember the concept about one user can subscribe more than one service to watch, obviously in this sample, our default user is suscribed to all services but we can change this link in ServiceUser
table.
Program
class code:
using System;
using System.Threading.Tasks;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;
using ServiceMonitor.Clients;
using ServiceMonitor.Clients.Models;
using ServiceMonitor.Common;
using ServiceMonitor.Common.Contracts;
namespace ServiceMonitor
{
class Program
{
private static ILogger Logger;
private static readonly AppSettings AppSettings;
static Program()
{
Logger = LoggingHelper.GetLogger<Program>();
var builder = new ConfigurationBuilder().AddJsonFile("appsettings.json");
var configuration = builder.Build();
AppSettings = new AppSettings();
configuration.GetSection("appSettings").Bind(AppSettings);
}
static void Main(string[] args)
{
StartAsync(args).GetAwaiter().GetResult();
Console.ReadLine();
}
static async Task StartAsync(string[] args)
{
Logger.LogDebug("Starting service monitor...");
var client = new ServiceMonitorWebAPIClient();
var serviceWatcherItemsResponse = default(ServiceWatchResponse);
try
{
serviceWatcherItemsResponse = await client.GetServiceWatcherItemsAsync();
}
catch (Exception ex)
{
Logger.LogError("Error on retrieve watch items: {0}", ex);
return;
}
foreach (var item in serviceWatcherItemsResponse.Model)
{
var watcherType = Type.GetType(item.TypeName, true);
var watcherInstance = Activator.CreateInstance(watcherType) as IWatcher;
await Task.Factory.StartNew(async () =>
{
var controller = new MonitorController(Logger, watcherInstance, client, AppSettings);
await controller.ProcessAsync(item);
});
}
}
}
}
Once we have checked the previous aspects, now we proceed to turn console application, console output is this:
dbug: ServiceMonitor.Program[0]
Starting application
sr trce: ServiceMonitor.Program[0]
06/20/2017 23:09:30 - Watching 'Sample API' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:30 - Watching 'Northwind Database' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:30 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:35 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:37 - Watching 'Sample API' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:39 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:42 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:43 - Watching 'Sample API' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:45 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:47 - Watching 'Northwind Database' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:48 - Watching 'Sample API' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:48 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:51 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:53 - Watching 'Sample API' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:54 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
06/20/2017 23:09:57 - Watching 'DNS' for 'Development' environment
Now we proceed to check the saved data in database, please check ServiceEnvironmentStatus
table, you'll get a result like this:
ServiceEnvironmentStatusID ServiceEnvironmentID Success WatchCount LastWatch
-------------------------- -------------------- ------- ----------- -----------------------
1 4 1 212 2018-11-22 23:11:34.113
2 1 1 78 2018-11-22 23:11:33.370
3 3 1 366 2018-11-22 23:11:34.620
(3 row(s) affected)
How it works all together? Console application takes all services to watch from API and then starts one task per watch item in an infinite way inside of MonitorController
, there is a delay time for each task, that interval is set in Service definition, but if there isn't a defined value for Interval, the interval is taken from AppSettings
; so after of perform of Watch
action, the result is saved on database through API and the process repeats itself. If you want to perform a watch
operation for other types, you can create your own Watcher
class.
Points of Interest
DatabaseWatcher
works with SQL Server, so how you connect to MySQL, PostgreSQL, Oracle and others DBMS? Create your Watcher
class for specific DBMS and implements the interface IWatcher
and write the code to connect for target database. - Can we host service monitor in non Windows platforms? Yes, since .NET Core is cross platform We can host this project on Windows, Mac OS and Linux.
- As I know, there isn't native support for ASMX in .NET Core but we can monitor both kind of services, simply adding the rows in
Service
table, the ASMX ends with .asmx. - Why console client and API aren't one single project? To prevent common issues on publishing, I think it's better to have two different projects because in that case We can run service monitor in one server and host API in another server.
- In this initial version, there isn't any configuration about security because it's better you add that implementation according to your scenario; you can make work this project with Windows Authentication, custom Authentication or add external service for authentication, that makes sense?
Code Improvements
- Add Identity Server
- Add notifications for administrators on critical errors during services watching (email, sms, etc.)
- I think it's better to have
TypeName
in ServiceCategory
instead of ServiceWatcher
- Add UI project to show in a pretty way the status of services for end users, using some front-end frameworks such as
Angular
You can check Projects section to know about on way improvements.
History
- 17th January, 2017: Initial version
- 20th June, 2017: Addition of environment for services
- 19th October, 2017: Update for Services (Business Layer)
- 15th July, 2018: Update for .NET Core 2
- 22th November, 2018: Removing repository pattern
- 10th January, 2019: Addition of Help Page for Web API
- 27th January, 2019: Code refactor