Add HTTP Request/Response Logging to an ASP.NET Framework Web Application With No Code

Introduction

Have you ever been troubleshooting an ASP.NET web application and wishing that you could see what was going on underneath the hood?

IIS logs are great - they can give you an indication of whether a request has reached the server, how long it took to process the request, what the source IP was, what the response status code was, etc.

What it doesn't give you is that little bit extra that you need sometimes - for example:

• What was passed into the application via the querystring or in the HTTP body?
• What the response was for that specific request?

But how does one enable this level of logging if it hasn't already been built into the web application?

One option is to find the source code, add a logging framework (log4net, Serilog, etc.) and a bunch of logging statements and redeploy the application. This is not always pleasant, possible or feasible.

Another option is to intercept the request/response in the IIS pipeline by using an HTTP Module, and that is precisely what this article is about.

Let's bear in mind that this only works for web applications that participate in the IIS pipeline (read: .NET Framework web applications or web services). Typically, .NET Core web applications do not participate in the IIS pipeline (unless you've added the ANCM module). So, if you're still banging around in the ASP.NET framework world, this is for you!

Background

I've been developing and working with web services for a few years now - from SOAP to REST to WebAPI and even a bit of GraphQL.NET. What I learnt early on was to add request/response logging to the services. These logs are invaluable when troubleshooting - especially when dealing with third parties. They would say that they sent x and got y and a quick dig in the logs could confirm (or refute) their claims. Any overhead incurred by logging is justified by the time saved digging for issues that simply don't exist.

It is also helpful for when you "inherit" some legacy applications that don't have any logging and you need to take a peek at the request/response details.

Using the Code

• Download RequestResponseModule.zip and extract the files.

Add the RequestResponseModule.dll file to the bin directory of the ASP.NET web application or service you wish to add logging to.

Add these keys to the <appSettings> section of the web.config file (change to suit your requirements):

<!-- Enable/disable logging (disable when not required, you'll save some overhead) -->
<add key="requestResponseLogger.enabled" value="true" />

<!-- Required (wildcards are not supported) -->
<!-- Comma separated path values to include in the logs. -->
<!-- If the URL path contains this value, the request will be logged. -->
<add key="requestResponseLogger.path.include" value="api/customer, api/order" />

<!-- Comma separated path values to exclude from the logs -->
<!-- If the URL path contains this value, the request will not be logged. -->
<add key="requestResponseLogger.path.exclude" value="api/payment,.aspx" />

Register the module by adding/amending the <system.webServer> section as follows:

<system.webServer> 
  <modules> 
    <add name="RequestResponseModule" type="RequestResponseModule.Logger" /> 
  </modules> 
</system.webServer>

That's it - you're ready to start logging.

Log files will be created in your web application's root directory (~/Logs/RequestResponse/).
A new file is created daily and any exceptions will be logged to the Errors.log file.

Note: Be sure to allow the web application's service account (the user context under which the web application runs - usually ASPNET or NETWORK SERVICE) sufficient privileges (Modify) to be able to create the logging directory structure and files.

The logs contain the following data in JSON format:

• Url
• Method
• TimeStamp (yyyy-MM-dd HH:mm:ss.fff)
• RequestBody
• ResponseBody
• ProcessingTime (hh:mm:ss.fffffff)
• HttpStatusCode

Here is an example of a single entry in the log file:

{"Url":"/api/customer/card/14","Method":"GET","TimeStamp":"2024-01-09 14:32:36.815",
"RequestBody":"","ResponseBody":"{\"CardHolderName\":\"LT FORD 14\",
\"Token\":\"FF3D60F8-6418-4729-AD06-BBB946462A8C\",\"ExpiryDate\":\"07/2025\",
\"IssuedDate\":\"2023-11-10T14:32:36.8151265+02:00\"}",
" ProcessingTime ":"00:00:0009941","HttpStatusCode":200}

The module also adds a response header to indicate if the request was flagged (based on the path filters) to be logged:

Points of Interest

Theoretically, one can make ASP.NET Core web applications run in the IIS pipeline by adding the ASP.NET Core Module for IIS. I haven't tried it though.

Credit to "mckamey" in this post for his solution to accessing the request body.

I tried a few methods to optimize logging performance and settled on the SemaphoreSlim class for writing to the log file in a synchronized manner. I'm open to suggestions on how this may be improved.

In future, I'd like to enhance the module to allow writing to a user-defined or shared location (UNC path).

The source code is available at https://github.com/thurstonford/RequestResponseModule.

History

• 12^th January, 2024: Initial version