Introduction
ASP.NET is a high productivity framework for building Web Applications using Web Forms, MVC, Web API and SignalR (this is the official definition). It is an ideal platform for developing RESTful applications on the .NET Framework or building great Web sites and Web applications using HTML, CSS and javascript.
But, if browser detection is a primary concern for your business, you certainly noticed that ASP.NET fails to correctly perform such a task: we shall illustrate this point in depth in the first part of this article. We shall then see in the second part how to elegantly circumvent this issue by amending the default configuration with browser files.
What is ASP.NET?
ASP.NET is a free web framework for building Web sites and Web applications using HTML, CSS and JavaScript. It also enables us to create Web APIs, mobile sites and use real-time technologies with SignalR. It is developed and maintained by Microsoft.
We shall develop in the remainder of this article an ASP.NET Web API application, but the following can be applied, without loss of generality, to all other components of the ASP.NET platform. ASP.NET Web API is just the solution developed by Microsoft for providing data services over HTTP. It facilitates plumbing code like content negotiation and leverages the power of HTML. ASP.NET Web API particularly shines with rich-client web applications (whether it be a Silverlight application or any single-page application like AngularJS) or mobile applications which needs a backend. It is also well suited for a platform for IoT.
Be aware of some shortcomings
ASP.NET is powerful and makes building great Web applications a piece of cake. Nonetheless, it comes with a downside that you must be aware of: browser detection is not its strong suit.
We shall illustrate this point by developing a simple application aimed at building a very simplified web analytics service similar to Google Analytics. We shall gather the browser used by the client and this short example will emphasize that the default configuration of ASP.NET for browser detection fails to provide correct results.
Setup our sample ASP.NET Web API application
We use throughout this article Visual Studio 2013 with the framework 4.5.1. This sample is inspired by the tutorial in the ASP.NET site (here).
Installation of the server side
- Start Visual Studio 2013 and select New Project from the Start page.
- In the Templates pane, select Installed Templates and expand the Visual C# node. Under Visual C#, select Web. In the list of project templates, select ASP.NET Web Application. In the New ASP.NET Project dialog, select the Empty template. Under "Add folders and core references for", check Web API. Click OK.
- In Solution Explorer, right-click the Controllers folder. Select Add and then select Controller.
- In the Add Scaffold dialog, select Web API Controller - Empty. Click Add. Keep the default controller name (DefaultController).
Installation of the client side
We add now an HTML page that uses AJAX to call the web API. We shall use jQuery to make the AJAX calls.
- In the Add New Item dialog, select the Web node under Visual C#, and then select the HTML Page item. Name the page "index.html".
- Replace everything in this file with the following:
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Example</title>
</head>
<body>
<div>
<h2>Lorem ipsum...</h2>
</div>
<script src="http://ajax.aspnetcdn.com/ajax/jQuery/jquery-2.0.3.min.js"></script>
<script>
var uri = $(document).ready(function () {
// We send an AJAX request.
$.getJSON(uri).done(function (data) { });
});
</script>
</body>
</html>
- Click on F5 to run the solution and check that everything is all right.
Detect browsers with an ASP.NET application
Once installed and configured, ASP.NET Web API enables us to easily access some basic properties of the incoming request: browser, platform and so on. The code below shows for example how to find out the browser used by the client (add it in the DefaultController class).
public IHttpActionResult Get()
{
HttpRequestBase baseRequest = ((HttpContextWrapper)Request.Properties["MS_HttpContext"]).Request as HttpRequestBase;
HttpBrowserCapabilitiesBase browser = baseRequest.Browser;
string b = browser.Browser + " " + browser.Version;
return Ok();
}
We determine here the browser name with the associated version.
Now, let’s look at what we obtain with some common browsers.
When I use Chrome
The browser is correctly inferred.
When I use Firefox
OK, everything is fine.
When I use Safari
Perfect.
When I use Internet Explorer
So far, so good…
When I use Edge
(Edge is a web browser developed by Microsoft and included in the company's Windows 10 operating systems, replacing Internet Explorer.)
Hum, hum. Edge is recognized as Chrome (with an incorrect version). Argh.
And it’s not over. Let’s now look at what we obtain with some rarer browsers.
When I use Opera
Opera is recognized as Chrome.
When I use Silk (Amazon)
(Silk is mainly distributed on Kindle Fire.)
Silk is recognized as Safari (with a strange version).
When I use Maxthon
(Maxthon is a freeware web browser developed in China.)
When I use SeaMonkey
(SeaMonkey is based on the same source code as Firefox.)
With SeaMonkey, it is even stranger, we get Mozilla 0.0.
And we could multiply examples with other browsers.
We can thus see that, if you want to finely detect browsers, the ASP.NET default configuration is not appropriate because it only infers the most common ones (and even badly detects Edge). After all, it can be enough for basic usages but if your objective is to more accurately qualify your users, this issue must be tackled.
But why does ASP.NET fails to correctly detect these browsers?
How the browser is actually inferred by ASP.NET
Enter user agent
Every time your browser connects to a website, it sends a string (that is, a line of text) identifying it and the operating system it runs on. This string is called a user agent. You will find below some examples of typical user agents for different browsers (based on different OS).
|
Chrome 47
|
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36
|
|
Firefox 43
|
Mozilla/5.0 (Windows NT 10.0; WOW64; rv:40.0) Gecko/20100101 Firefox/43.0
|
|
Safari 8
|
Mozilla/5.0 (iPhone; CPU iPhone OS 8_4_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12H321 Safari/600.1.4
|
|
IE 11
|
Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko
|
|
Edge
|
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240
|
|
Opera
|
Opera/9.80 (Windows NT 6.0) Presto/2.12.388 Version/12.14
Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.73 Safari/537.36 OPR/34.0.2036.25
|
|
Silk 2
|
Mozilla/5.0 (Linux; U; en-us; KFTT Build/IML74K) AppleWebKit/535.19 (KHTML, like Gecko) Silk/2.1 Safari/535.19 Silk-Accelerated=true
|
|
Maxthon 4
|
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Maxthon/4.4.1.5000 Chrome/30.0.1599.101 Safari/537.36
|
|
SeaMonkey 8
|
Mozilla/5.0 (Windows; U; Windows NT 10.0; WOW64; rv:1.8.0.7) Gecko/20110321 MultiZilla/4.33.2.6a SeaMonkey/8.6.55
|
Describing the exhaustive structure of a user agent is beyond the scope of this article. But you can already see that Chrome 47 contains the term Safari or that Edge contains terms Chrome and Safari. And so, detecting browsers is not as simple as finding the string Chrome in the user agent for Chrome 47 or the string Edge for Edge.
The contents of the user agent field vary from browser to browser and from OS to OS. Each browser has its own, distinctive user agent. Eventually, a user agent is a way for a browser to say to a web server Hi, I’m Firefox on Windows or Hi, I’m Safari on an iPhone.
This is this user agent that ASP.NET uses to find out the browser. Let us see how this is performed.
OK, and in concrete terms?
This looks really simple in theory, but it is a bit more complicated in practice. Because user agents have become a mess over time and are rather different from one browser to another and from one OS to another, ASP.NET uses a complex (yet powerful) mechanics to detect browsers.
We have just listed above a few examples of user agents. From this list, we could begin writing browser detection in pseudo-code in an imperative manner:
- Find if the user agent contains the string Opera. If yes, terminate, the browser is Opera.
- Find if the user agent contains the string Trident. If yes, terminate, the browser is IE.
- Find if the user agent contains the string Firefox. If yes, terminate, the browser is Firefox.
- Find if the user agent contains the string SeaMonkey. If yes, terminate, the browser is SeaMonkey.
- Find if the user agent contains the string Safari. If yes, terminate, the browser is Safari.
- Find if the user agent contains the string Silk. If yes, terminate, the browser is Silk.
- Find if the user agent contains the string Edge. If yes, terminate, the browser is Edge.
- …
- Otherwise, the browser is Chrome.
As a picture often speaks better than words, it is maybe clearer in the figure below.
You can see that this code is pretty cumbersome and rather inelegant. What if we had to add a new browser or a new rule? What if a browser changed its user agent? ASP.NET comes to the rescue with browser files. Browser files are a way to detect browsers, but instead of using a procedural view like the aforementioned code, they define hierarchical structures in a more natural fashion. You will find more details on browser files on the MSDN (here).
But how are browser files organized and structured ?
You can see from the above listing that user agents are rather different from browsers to browsers but they still follow a certain rationality. For example:
- Almost everyone begins with the string Mozilla (Opera is the exception).
- Firefox and IE does not contain the string AppleWebKit contrary to the others.
- …
We can derive from these remarks and the pseudo-code developed above a hierarchical structure of user agents (see below).
Here is how to read and use this tree:
start at the root (Default node)
if (user agent contains Opera) return Opera;
else if (user agent contains Mozilla) then
if (user agent contains AppleWebKit) then
if (user agent contains Chrome) return Chrome;
else if (user agent contains Safari) return Safari;
else return WebKit;
else if (user agent contains Firefox) return Firefox;
else if (user agent contains Trident) return IE;
else return Mozilla;
else return Unknown;
Browser files are just XML files that mimic this hierarchical structure as far as possible. And thus:
- We shall have an XML file that will mimic the Default behavior of the previous figure.
- We shall have an XML file that will mimic the Opera behavior of the previous figure.
- And so on …
ASP.NET comes with default browser files that you can find in the C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\Browsers folder on Windows 7 (see below). These browser files give an overview of what they look like in practice.
If you open one of these files (chrome.browser for example), you will find the following content.
<browsers>
<browser id="Chrome" parentID="WebKit">
<identification>
<userAgent match="Chrome/(?'version'(?'major'\d+)(\.(?'minor'\d+)?)\w*)" />
</identification>
<capabilities>
<capability name="browser" value="Chrome" />
<capability name="majorversion" value="${major}" />
<capability name="minorversion" value="${minor}" />
<capability name="type" value="Chrome${major}" />
<capability name="version" value="${version}" />
<capability name="ecmascriptversion" value="3.0" />
<capability name="javascript" value="true" />
<capability name="javascriptversion" value="1.7" />
<capability name="w3cdomversion" value="1.0" />
<capability name="supportsAccesskeyAttribute" value="true" />
<capability name="tagwriter" value="System.Web.UI.HtmlTextWriter" />
<capability name="cookies" value="true" />
<capability name="frames" value="true" />
<capability name="javaapplets" value="true" />
<capability name="supportsCallback" value="true" />
<capability name="supportsDivNoWrap" value="false" />
<capability name="supportsFileUpload" value="true" />
<capability name="supportsMaintainScrollPositionOnPostback" value="true" />
<capability name="supportsMultilineTextBoxDisplay" value="true" />
<capability name="supportsXmlHttp" value="true" />
<capability name="tables" value="true" />
</capabilities>
</browser>
</browsers>
We can see that there is a parentID tag filled with WebKit. This tag refers to another browser file (generic.browser) showed below and that is the parent of the chrome.browser file.
<browser id="WebKit" parentID="Mozilla">
<identification>
<userAgent match="AppleWebKit" />
</identification>
<capture>
<userAgent match="AppleWebKit/(?'layoutVersion'\d+)" />
</capture>
<capabilities>
<capability name="layoutEngine" value="WebKit" />
<capability name="layoutEngineVersion" value="${layoutVersion}" />
</capabilities>
</browser>
This file itself refers to another browser (Mozilla) and this schema continues until the root is reached (until there is no more parentID tag in the browser file). This is how browser files translate into XML files the hierarchical structure described above.
(At the same time, you will notice how a user agent is matched with the corresponding input via the identification tag (and how this one can also take into account version of the browser with the use of regular expressions).)
Once this hierarchical structure is revealed, we can sum up how a browser is inferred by our ASP.NET Web API application.
- The user agent is extracted from the request made by the client.
- The user agent is compared to the root browser file (here Default.browser) and if it matches, the HttpBrowserCapabilities object is populated with the capabilities associated with the file.
- The user agent is then compared to each child the browser file can have. If one of them matches, the HttpBrowserCapabilities object is overriden with the new capabilities. If none matches, the capabilities are not modified.
- And so on, until the process reaches the bottom of the hierarchical tree.
The good news is that we can override this default configuration: what we have to do is just adding custom browser files in the App_Browsers folder in the ASP.NET Web API application. We can then change the default behavior and correctly detect browsers. More on this later.
Enough theory, code please !
We saw earlier that Edge was incorrectly detected as Chrome. Let’s explain why with what we have just described.
- The Edge user agent is Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10136.
- This user agent is compared to Default. As no match is conditioned, the browser is set to Unknown 0.0.
- This user agent is compared to Mozilla. The user agent matches because it contains Mozilla. The browser is set to Mozilla 0.0.
- This user agent is compared to IE. The user agent does not match (it does not contain Trident).
- This user agent is compared to WebKit. The user agent matches because it contains AppleWebKit. The browser is set to Mozilla 0.0 (the name is not overriden).
- This user agent is compared to Chrome. The user agent matches because it contains Chrome. The browser is set to Chrome 42.
- As there are no browser files under Chrome, the process stops.
That explains why Edge is recognized as Chrome.
So, how to correct this behavior? As we said earlier, it is possible to add custom browser files to do it.
Follow the steps below:
- Add the App_Browsers folder in your application (Add -> Add ASP.NET Folder -> App_Browsers).
- In this folder, add a new file named edge.browser.
- Fill in this file with the following content.
<browsers>
<browser id="Edge" parentID="Chrome">
<identification>
<userAgent match="Edge/(?'version'(?'major'\d+)(\.(?'minor'\d+)?)\w*)" />
</identification>
<capabilities>
<capability name="browser" value="Edge" />
<capability name="majorversion" value="${major}" />
<capability name="minorversion" value="${minor}" />
<capability name="type" value="Edge${major}" />
<capability name="version" value="${version}" />
</capabilities>
</browser>
</browsers>
To be more precise, we are just telling ASP.NET that an extra step must be performed in the process: ASP.NET Web API has to check if the user agent contains Edge. As we state that the parentID is Chrome, this operation is only completed if all the requirements for Chrome have already been fulfilled (Mozilla, AppleWebKit, Chrome),
We can see that the result is better than earlier.
And we could apply the same reasoning for the other browsers. For example:
For SeaMonkey
- Add a seamonkey.browser file in the App_Browsers folder.
- Fill in with the following content.
<browsers>
<browser id="SeaMonkey" parentID="Firefox3Plus">
<identification>
<userAgent match="SeaMonkey/(?'version'(?'major'\d+)(\.(?'minor'\d+)?)\w*)" />
</identification>
<capabilities>
<capability name="browser" value="SeaMonkey" />
<capability name="majorversion" value="${major}" />
<capability name="minorversion" value="${minor}" />
<capability name="type" value="SeaMonkey${major}" />
<capability name="version" value="${version}" />
</capabilities>
</browser>
</browsers>
For Maxthon
- Add a maxthon.browser file in the App_Browsers folder.
- Fill in with the following content.
<browsers>
<browser id="Maxthon" parentID="Chrome">
<identification>
<userAgent match="Maxthon/(?'version'(?'major'\d+)(\.(?'minor'\d+)?)\w*)" />
</identification>
<capabilities>
<capability name="browser" value="Maxthon" />
<capability name="majorversion" value="${major}" />
<capability name="minorversion" value="${minor}" />
<capability name="type" value="Maxthon${major}" />
<capability name="version" value="${version}" />
</capabilities>
</browser>
</browsers>
And so on for all other browsers and for those that will be released in the coming months.
But what’s happening in production ?
Browser detection was a primary concern for us. That is the reason why we implemented the guidelines above. But in production, we quickly experimented a very strange and unpredictable phenomenon.
For example, the user agent Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36 was sometimes recognized as a Chrome browser (this is the correct answer) but sometimes was detected as a Maxthon browser. What does this farce mean ?
We ended up finding the problem: instead of achieving at each request the process described above to find out the browser, ASP.NET keeps in memory (=in cache) a temporary key-value dictionary of user agents with the corresponding browser.
This is what happened on a simple example:
- Visitor 1 with user agent us1 comes to the site. ASP.NET looks into its cache to see if this user agent is already registered. As it is not the case, ASP.NET determines the browser with the process described above (based on browser files). The cache is then updated with this value. Here is what it contains.
- Visitor 2 with user agent us2 comes to the site. ASP.NET looks into its cache to see if this user agent is already registered. As it is not the case, ASP.NET determines the browser with the process described above (based on browser files). The cache is then updated with this value. Here is what it contains.
|
Key
|
Value
|
|
us1
|
browser1
|
|
us2
|
browser2
|
- Visitor 3 with user agent us1 (the same as Visitor 1) comes to the site. ASP.NET looks into its cache to see if this user agent is already registered. As this the case, ASP.NET does not need to recompute the browser and can immediately return the result from the cache.
This modus operandi seems a priori very powerful because it improves performance. But there is a small glitch in this machinery: the key used in the dictionary is not the user agent itself but only the first 64 characters. So:
- When a visitor with user agent Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36 comes to the site, the key used in the cache is Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.36 (KHTML, like Ge. The browser detected is Chrome.
- Consequently, when another visitor with user agent Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.36 (KHTML, like Gecko) Maxthon/4.4.8.1000 Chrome/49.0.2623.87 Safari/537.36 comes one second later to the site, the key used is once again Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.36 (KHTML, like Ge. As the key is stored in the cache, the classical process based on browser files is not run and the browser is mistakenly recognized as Chrome.
Fortunately, this cache key length is configurable in the web.config (in the <system.web> section). Add the following code:
<system.web>
<browserCaps userAgentCacheKeyLength="256" />
</system.web>
The default setting is to take the first 64 characters of the user agent, we configure it here to take the 256 first ones.
Things go back to the normal after this modification and browsers were eventually all correctly detected.
To go further
Browser files can also be used to detect platforms and not only browsers.
Summary
If browser detection is of paramount importance for your business, the ASP.NET default configuration is not sufficient and can display incorrect results.
Two main changes have to be made so as to circumvent this issue :
- adding suitable browser files in the App_Browsers folder
- modifying the userAgentCacheKeyLength attribute in the web.config file
