Diagnosing Windows Services might sometimes be cumbersome – especially when errors occur during the service start. In this two-part series, I am going to show you different ways to handle such problems in production. In the first part, we will focus on “exceptions discovery” techniques which very often are enough to figure out why our service is not working. In the second part, we will setup a debugging environment and attach a debugger to our service. Let’s start then.
Logging
Collecting traces is the less invasive way of diagnostics and, if done correctly, is surprisingly effective. Loggers are one of the first things you should configure when developing a new application. Think what you should log, what would be the format of your logs (must be consistent) and which verbosity levels you would use (for instance VERBOSE for debug logs, ERROR for swallowed exceptions, CRITICAL for unhandled exceptions, etc.). It is worth writing down those rules and including later in the release documentation. My logging library of choice is NLog and I’ve already published few articles how to configure it, so I won’t repeat myself but provide you with some links:
If you prefer System.Diagnostics namespace, I also have something for you. :)
Try to log as much as you can in a given context – as for exceptions do not always rely on Exception.ToString() method. It is worth writing a separate catch block for an exception which is known to have a meaningless string representation (such as DbUpdateException or DbEntityValidationException) and iterate through its properties in the log.
In Quest for the Exception Log (Introducing inwrap.exe)
When the service you need to diagnose does not generate any logs, the situation becomes much more difficult. In such a case, you may start with collecting traces using sysinternals tools. Keep in mind though that sysinternals log is scarce for .NET applications and it won’t reveal more than a name of the exception being thrown. Also, by default, sysinternals tools do not provide you with a way to diagnose a faulty start of a service (I will write more about this in the second part of the series). These are the reasons why inwrap.exe was born. Inwrap is a simple application which wraps the original service and logs all the managed exceptions (including the handled ones) that occur in it. Additionally, you may create your own exception handlers which will be called by InWrap, allowing you to collect application-specific information required to diagnose a given exception. To trace exceptions in a standalone application, just call:
inwrap.exe your-app.exe
To trace a Windows service, we need to install inwrap as a wrapper for the .exe file of the service. It may sound difficult, but is just a matter of calling:
inwrap.exe -install your-service.exe
From now on, anytime you start the service, it will be wrapped by inwrap.exe. To uninstall, change -install to -uninstall in the above command. Simple, isn’t it?
By default, inwrap writes logs to console and Windows debug output, but you can easily change this by modifying inwrap.exe.config file and adding any TraceListener you like. Inwrap has embedded the Essential.Diagnostics library so using one of their listeners is a matter of adding few lines to the configuration file. For example, if you would like to change System.Diagnostics.ConsoleTraceListener to Essential.Diagnostics.ColoredConsoleTraceListener, you could use the following configuration (no additional assemblies are needed):
="1.0"="utf-8"
<configuration>
<system.diagnostics>
<trace autoflush="true"/>
<sharedListeners>
<add name="colored-console"
type="Essential.Diagnostics.ColoredConsoleTraceListener, Essential.Diagnostics" />
</sharedListeners>
<sources>
<source name="LowLevelDesign.Diagnostics.InWrap" switchValue="Verbose">
<listeners>
<add name="colored-console" />
</listeners>
</source>
</sources>
</system.diagnostics>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
</startup>
</configuration>
Finally, the most interesting part of inwrap are exception handlers. You specify a folder where handlers are stored by using the -handlers <handlers-folder-path> switch. On start inwrap scans the handlers folder for assemblies which names end with Exception or Exceptions. When it finds such an assembly, it will check if it contains any static class with a method accepting System.Diagnostics.TraceSource as a first parameter and a derive from System.Exception as a second. If inwrap finds such a method, it will call it when exception of a given type occurs in the examined application. Example DbUpdateException handler might look as follows:
using Dapper;
using System.Data;
using System.Data.Entity.Infrastructure;
using System.Data.SqlClient;
using System.Diagnostics;
using System.Linq;
namespace Trader.Domiporta.Zolas
{
public static class DbUpdateExceptionHandler
{
public static void HandleException(TraceSource logger, DbUpdateException ex)
{
foreach (DbEntityEntry dbEntityEntry in ex.Entries)
{
FlapiAdvert flapiAdvert = dbEntityEntry.Entity as FlapiAdvert;
if (flapiAdvert != null)
{
using (SqlConnection sqlConnection = new SqlConnection("data source=....."))
{
sqlConnection.Open();
if (Enumerable.FirstOrDefault<ApplicationUser>
(SqlMapper.Query<ApplicationUser>((IDbConnection) sqlConnection,
"select * from users where id = @UserId", (object) flapiAdvert,
(IDbTransaction) null, true, new int?(), new CommandType?())) == null)
logger.TraceEvent(TraceEventType.Error, 0, "User {0}
could not be found for advert {1}", (object) flapiAdvert.UserId,
(object) flapiAdvert.AlladsId);
}
}
}
}
}
}
It is a sample from my work diagnostics case – we had a problem in production in our synchronization service. I could see that it was throwing DbUpdateExceptions but didn’t know exactly why. As I didn’t want to modify and deploy the whole service, I simply run it under inwrap and from the log figured out which entities in our database were problematic.
Does It Work with Topshelf Services?
Unfortunately, not from the box. Topshelf is very strict about what’s passed to the service on the command line and throws exception on unregistered parameters and switches. It also does not allow running a service under the remote debugger (more about this in the second part of the series). To overcome those issues, I prepared a small Nuget package: Topshelf.Diagnostics which contains all the necessary classes to make Topshelf services debuggable. For inwrap to work with your service, you just need to add call to the ApplyCommandLineWithInwrapSupport in your host configuration section, example:
class Program
{
private static readonly Logger logger = LogManager.GetCurrentClassLogger();
private static void Main() {
HostFactory.Run(hc => {
hc.UseNLog();
hc.Service<SampleService>();
hc.SetServiceName(typeof(SampleService).Namespace);
hc.SetDisplayName(typeof(SampleService).Namespace);
hc.SetDescription("Sample service.");
hc.ApplyCommandLineWithInwrapSupport();
});
}
}
Note, that if inwrap is not present, this method behaves in the same way as Topshelf’s default ApplyCommandLine so all the command line switches should work just fine. In case you run into any problems, just let me know. Inwrap can be downloaded from the diagnettoolkit github repository.
Filed under: CodeProject, Diagnosing exceptions in .NET
