Introduction
In this article, we will look at implementing SSL and Client Certificates in Windows environment. However, even if you do not belong to the Microsoft world, this article will give you a good insight into few of the core concepts in certificate based security.
I would really love your feedback and corrections (if any).
Background
This article, if read in continuance with my previous article (here), will give you enough background to start with. However, if you already have basic understanding of SSL, you may safely skip reading it.
Contents
We will discuss the following topics:
- Configuring SSL
- Configuring IIS for SSL
- Configuring SSL in a self-hosted application
- Client authentication using Client Certificates
- CRL, CTL and Cert Stores
- Configuring IIS for Client Certificate Validation
- Accessing and validating Client Certificates in code (.NET)
- Client Certificate validation in WCF
Configuring SSL
Configuring SSL on Sockets
.NET Framework provides the SslStream class that provides a stream used for client-server communication that uses SSL to authenticate the server and also the client. More details and code can be found here. You will notice that in this case, since we are playing with the Transport layer in socket programming, we will have to do some of the SSL plumbing manually in code. Please note how we are not talking about HTTPS here but SSL over sockets.
Configuring HTTPS (through SSL) on Web Applications
Digital Certificate Requirements for Setting Up HTTPS
As mentioned in my previous article, HTTPS uses Digital Certificates and the PKI at its core. It can be used over any network traffic utilizing reliable stream transport protocol (TCP). HTTPS is one of the most popular implementation of SSL. A server is "identified" and the traffic is encrypted using SSL. How is the server "identified" by HTTPS? It is through the "Issued To" property of the certificate used by the server.
Historically, the "Issued To" property of the certificates used to be the website address (for e.g., "blah.com"). This could then be assigned to an IP address and port. So when the operating system receives https traffic, it (http.sys/IIS using the underlying Schannel SSP) can then select the website’s related certificate and use its private key to decrypt the traffic. Remember that the only 'visible' part of such encrypted raw https traffic for the OS is the IP address and port number (this changed with introduction of SNI in TLS, we will come to this later). This used to cause major headaches in hosting multiple websites on the same IP address and port (80). Please Google "Chicken and Egg problem with SSL" to learn more.
However a certificate's "issued to" property can now have a wildcard character to reflect its association with a domain. For e.g., *.blah.com could be used to apply https on www.one.blah.com and www.two.blah.com. This again is not optimal as there may be requirement (as in case of web hosting companies) of using a single certificate for multiple sites with different domain names.
Then came the multi-domain, Subject Alternative Name (SAN) certificates. These have a list of DNS names in "Subject Alternative Name" field of such certificate. Hence, the OS can now decrypt the https traffic using such certificate and find out the host header address for further processing. On the other end, the client can also positively verify that the server is on the Subject Alternative Name list. However, this was again not the best solution, since every time a change was required (e.g., remove/change one of the entries in the SAN list), the certificate had to be revoked and a new certificate is required to be issued by the CA with the changes in it.
With TLS came an extension called "Server Name Indication" (SNI). SNI is an added visible part of the https traffic, and contains the host header information that the server can use to pick the correct certificate for decryption. However, please remember that since SSL/TLS relies on both: server and client, both parties should be complaint when it comes to any advancement in the protocol stack. IE on Windows XP doesn't support TLS, hence SNI is not supported on it.
Configuring https on IIS Hosted Websites
On IIS, the steps for configuring SSL (for a Website/ Web Api/ Web Service) are relatively simple. Let us assume that you have a valid digital certificate for use in SSL. Either such certificate is minted by your local CA or is given to you by a well-known external CA. Let us assume that the certificate is installed correctly in the Computer’s Personal certificate store.
We will discuss three scenarios here (there can be plenty others):
Scenario #1: Hosting a website with address "test.dummyblah.com" and we have a server certificate installed on the server with "issued to" property set to "*.dummyblah.com".
This is a common scenario where an organisation is hosting multiple websites (with same domain name).
The first step in setting up SSL is to create "Site Bindings" for https. Please refer to the image below; you will see how a new https binding is created with a Wildcard SSL certificate. Please remember that the "Host name" field in "Edit Site Bindings" window will only make sense if the certificate used is a Wildcard certificate. Hence, this field is enabled only if such certificate is selected in the "SSL Certificate" dropdown.
However, this site is still open to http request (there is a binding present for it). If we want to restrict the site to only cater to https requests, we will have to do that setting from "SSL Settings" for that site.
Along with this, some steps would be required to redirect users from http to https (probably through some URL rewrite rules or HSTS header).
Scenario #2: Hosting an HTTPS website on a Shared Hosting provider (GoDaddy, Winhost etc) server
This will depend on whether the hosting provider has SNI capability. If yes, you won't need your website hosted with dedicated IP (remember the chicken and egg problem?) which is generally more expensive. The hosting providers generally provide a self-service website (CPANEL for e.g.) that you can use to buy and link a certificate to your website.
Scenario #3: Hosting an HTTPS website on local IIS on a development machine
You (as a developer) may like to host your website/webapi on your computer (running IIS 7 for e.g.) and play with HTTPS during development phase. To do this, you will need a certificate valid on only your own machine. The easiest way to get such certificate is to get IIS to generate it for you. If you click on the server name (top level) in IIS Management Console, you will see a "Server Certificate" action button. Opening this will let you create Self-Signed certificate. This will generate a self-signed server certificate and install it to your local computer's Personal certificate store.
You can then use this certificate to create https binding for your website. The only glitch is that you will see error messages in browser that you will have to ignore since the self-signed certificate's "issue to" property is set to the machine name and not the website name (localhost).
Please note how I am not using standard port numbers (80 for http and 443 for https) in my example and instead using port numbers 888 and 444 respectively since I may be hosting multiple websites on my development PC. Alternatively, I could have used standard ports and configured host-header in both IIS and "hosts" file (located at {WIN_DIR}\System32\drivers\etc) to host multiple http/https websites.
On your development machine, you can generate and use test certificates using makecert.exe as mentioned in this link.
Scenario #4: Development computer running Visual Studio
This scenario is the simplest. The zest of using Visual Studio is it does take care of lots of developer tasks itself. For example, when you install Visual Studio, it will generate and install a server certificate called "IIS Express Development Certificate" that has the "issued to" field set to "localhost". (Alternatively, if you want to do the same manually -without using VS- you may have to use some certificate generation utility like Makecert, etc. to generate such certificate.)
Now, whenever you want to enable SSL on a website, just enable it from the Properties window for the website/webapi project.
Configuring https on Self-hosted (OWIN/Katana) Websites
When a website is self-hosted using services provided by Katana (Microsoft implementation of OWIN), you obviously don't get the nice features offered by IIS. Remember that Katana in essence uses the "HttpListener" class, which in turn utilizes the kernel mode "http.sys" driver provided by the Windows OS. Hence, if we have to configure HTTPS over self-hosted sites, we will have to configure it at the OS level.
A common way of configuring SSL at OS level is to use the powerful "netsh" command.
netsh http add sslcert ipport=0.0.0.0:443 appid={12345678-db90-4b66-8b01-88f7af2e36bf}
certhash="83ae3ca2c538b95e5037e033a873002e1890f5ba"
appid and certhash values can be found by running the following command:
netsh http show sslcert
Here, all 0s means all IP addresses on this machine. It can be a specific IP as well.
Note: Don’t forget to restart your server/ computer after such changes have been made through netsh (as these apply to Kernel mode processes).
There is a nice utility available here that provides UI to view and manage http.sys bindings. The article also explains about url acls which is more relevant in case of self-hosted application / web services.
Client Authentication Using Client Certificates
SSL can be configured so as to allow server to authenticate client using client certificates. However, in this case, there are few additional configuration steps required on the server and the client.
In this mode, the Server (during SSL handshake) can be configured to send a list of acceptable (trusted) certificate issuers to the client browser (if it is a browser) in the form of a hint. The browser then checks for the available client certificates in the user’s personal certificate store (with access to private key) that have any of the CAs from the server sent hint in their issuer chain. If it finds any such certificates, the user is presented with a screen where the user then selects the certificate to send to the server.
Alternatively, if no such hint is sent by the server, the browser shows all client certificates (with private keys) the user has in ‘Personal’ certificate store. Once the user selects a certificate for client validation, the browser then sends the public key of this certificate in https request. Upon successful validation of the client certificate on the server, the request is either granted or denied access.
There is a bit of a grey area around the actual implementation of client certificate validation in Windows OS. The client certificate is 'negotiated' by http.sys at the transport layer. Http.sys can be optionally configured (using netsh command) to always do this negotiation during SSL handshake. Or, IIS (/application host) can request this negotiation to happen anytime to http.sys (for example, if a sub-directory's access is restricted by client certificate based authentication) if this negotiation has not already happened. In this case, SSL session is re-negotiated; this time, with client certificate requirements. Http.sys then does client certificate validation (once passed to it by client/browser) based on CRL and CTL (or cert stores) and can also be configured to map the client certificate to an AD user. The HTTP request along with the client certificate is then passed to IIS (and the application).
Tip: If you want to see what is sent from the server during SSL handshake, please download and install OpenSSL and then OpenSSL tool for Windows from here. If, let's say, you want to test https://dummyblah.com requiring client certificate authentication, use the following commands (one at a time) in the "OpenSSL" command prompt:
s_client -connect dummyblah.com:443 -prexit
GET /
Before we begin, let's discuss a few important concepts:
Certificate Revocation List (CRL)
CRL is a list of serial numbers of the certificates that a CA has revoked (cancelled). It is the responsibility of a CA (that has issued a certificate) to provide a facility for clients to know if a particular certificate has been revoked. And it is the responsibility of the client to check with the CA has revoked a certificate it has issued. This last statement is important, because even if a CA has revoked a certificate, if the client (browser for e.g.) is not checking the revocation list, the certificate will still appear valid to the client. Different browsers have different behaviour when it comes to CRL check. Some important issues have been pointed here.
The CA maintains and hosts such CRL files on a location clients can access. The URL to this location can be found under the "CRL Distribution Points" extended property of the certificate.
The CRL files can be a base file and a number of delta files (smaller intermediate additions to base file). The client can download these files and can cache it for some time. However, this mechanism of downloading files, however large they are, can mean a small lag in SSL handshake. Besides CRL files published for public access through URLs are prone to denial of service attacks (DOS / DDOS). To avoid these issues, there is an alternate (newer) way of checking if a certificate is revoked, called 'Online Certificate Status Protocol (OCSP)'. OCSP enables checking a single certificate for revokation. A client can use OCSP or fall back to CRL files based check.
Whenever IIS receives a client certificate in http request (and is configured to accept and validate it), it does a CRL check too. Please remember that this is an essential step, and if connectivity between the host (IIS) and CRL link is broken, the client certificate validation will fail with error code 403.13. However, in case of self-signed certificates, CRL check doesn't make sense, hence the "Authority Information Access" and "CRL Distribution Point" fields of the certificate would not be required in the certificate and CRL check won't happen.
Certificate Trust List (CTL)
Certificate Trust List is a list of trusted CAs. The server may be configured to trust a number of global- root CAs and a number of internal/external CAs. IIS can be configured to authenticate a client certificates against these trusted certificates authorities. However, this will mean that any client certificate issued by any such well known root (/intermediate) CAs will be considered valid. To avoid this, a 'CTL' could be created and configured (at system level or website level) which acts as a 'filter' for validation. If so, the client certificate must be issued by one of the CAs in the configured CTL. Remember that the CRL check is still done.
CTLs were traditionally (pre-IIS6) also being sent to the browser, during SSL handshake, to suggest (hint) browser what client-certificate the server is looking for. The browser will then show a message like below for the user to select an appropriate certificate. However since IIS6, Microsoft has changed this behaviour. IIS does not send the CTL as a hint to the browser, but instead, sends a list of all Trusted Root CAs (confusing enough, this list is also referred to as CTL in some technical docs).
This behaviour of sending a hint to the client (during client certificate validation) can be controlled by changing the value of "SendTrustedIssuerList" (DWord) key in Windows registry located at:
HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Control/SecurityProviders/SCHANNEL
By default, before Windows 8/ Windows Server 2012, this value used to be 1 (true). If this behaviour is turned off (by setting this key to 0), the browser will then get no hint and hence present the user with all client certificates on the user’s personal certificate store (even those not trusted by the server).
Note: There can be some complications if the number of ‘Trusted Root CAs’ store on the server is too large. If so, the browser hint would be broken or overflowing and the browser will show the user a blank message window (with no suggested certificates) or a truncated list (to confuse the hell out of you). Hence, on the safe side, it is recommended not to send CTL to the client unless you are very sure of keeping the trusted root CA store under a certain size limit. Some details here: http://netsekure.org/2011/04/tls-client-authentication-and-trusted-issuers-list/
If you want to create a CTL for authentication of client certificates (even if not to hint the client), it can be created using a tool called MakeCTL.exe as pointed in this post. Surprisingly, Microsoft didn’t provide good UI based tools to create CTL files since Windows 2003 Server, I guess due to popular and easier alternative provided by .NET framework (and WIF) to authenticate client certificates in code on the server (I will explain this later in this article).
With Windows Server 2012 and Windows 8, Microsoft has changed this client certificate validation model. CTL based validation is unsupported. Instead, certificate stores are used for this purpose. There is now a new dedicated store named "Client Authentication Issuers (ClientAuthIssuer)" that acts as a placeholder for listing CAs for Client Certificate based authentication at machine level. Besides, custom credential stores can be defined and used to authenticate individual websites. So firstly, the website's custom credential store is checked (if configured), if it does not exist, then the ClientAuthIssuer store is checked (if configured). At last, the "Trusted Root Certification Authorities" store is checked by default. Also, as mentioned previously, no hint is sent to the browser by default and the default value of "SendTrustedIssuerList" registry key is 0.
As mentioned in this article, you can use the following command to bind the ClientAuthIssuer certificate store to an SSL port for client certificate validation:
netsh http add sslcert ipport=0.0.0.0:443 certhash=GUID hash value appid={GUID application identifier} sslctlstorename=ClientAuthIssuer
Notes:
- Since this will effect Kernel mode processes (in
http.sys), you will have to reboot your computer to apply the changes. - Even though
ClientAuthIssuer store is configured to client cert validation, the CA must also be in the "Trusted Root Certificate Authorities" store.
On a side note, observe the presence of a store named “Web Hosting” which is added as an addition to “Personal” store and features scalability for accommodating higher number of certificates.
Configuring IIS for Client Certificate Validation
IIS needs to be configured to "Accept" or "Require" the client certificate as shown in the image below:
- If "Accept" is selected, and if client certificate is provided, IIS will accept the certificate, validate it, and forward the HTTP request to the application with the certificate.
If client certificate is not provided, IIS will still forward that HTTP request (without certificate) to the application. - If "Require" is selected, and if client certificate is provided, IIS will accept the certificate, validate it, and forward the HTTP request to the application with the certificate (just like "Accept" option). However, if client certificate is not provided, IIS will throw Http Error 403.7 – Forbidden to the client.
You can set these values in configuration files (web.config) by using:
<system.webServer>
<security>
<access sslFlags="ssl">
</security>
</system.webServer>
Here "ssl" is equivalent to "Require SSL" option in IIS, "sslNegotiateCert" is equivalent to Client Certificates -> "Accept" and "sslRequireCert" is equivalent to Client Certificate -> "Require".
Further to just validating client request through client certificates, IIS can be used to configure authenticate and map client certificate to a Windows user. If configured so, the request then gets impersonated as the Windows user account by IIS. There are two native modules named "CertificateMappingAuthenticationModule" and "IISCertificateMappingAuthenticationModule" in IIS 7+ that lets you do so. The first one requires the presence of Active Directory (AD) domain whereas the later gives flexibility in working without it (AD). The configuration can be done through web.config using either the <clientCertificateMappingAuthentication> tag or the <iisClientCertificateMappingAuthentication> tag respectively. Besides, IISClientCertificateMapping provides capability to do either One-to-One mapping or Many-to-One mapping. I won’t go into describing how to do this (as this article will be too long), however, the following articles will give you a good understanding of these:
- How to configure Client Certificate Mapping Authentication
- How to configure IIS Client Certificate Mapping Authentication
- How to map a certificate to a user account in AD
Can you see the value in using IIS for Client Certificate based authentication yet? Yes, the entire pain in authenticating and managing users over the internet can be taken off your web-application (.NET or non-.NET based) and assigned to IIS and AD (and the infrastructure team in your organisation, yey!!! ). Of course, this won’t always be the use-case, but where the authentication is meant for organisational users or the user base is limited, this can be a blessing (and an alternative to VPN in a narrow scope).
The biggest issue (or call it lack of facility) with using IIS for client certificate authentication is that you won’t have finer control over exception handling, logging (apart from IIS logs) and alerts. This is one of the most common reasons why most developers prefer to validate client certificates in server side code (within application).
Validating Client Certificates in Code
Client certification validation (and authentication) can be much involved than we have seen till now; for e.g., we may need to extract out more information from the incoming client certificate and use it for further authentication (or authorisation). In modern day web based information technologies, federated security and claims based identity play a major role. The client can be an application (rather than a user) talking to some resource server after being authenticated by an authentication server. Client certificates play a major role in establishing 'trust' between client and server or server and server, and manipulating client certificates in code becomes a necessity.
I have attached three projects with this article for some samples on how to validate and process client certificates in code. These may not necessarily the best samples, however, they will get you started.
Validating Client Certificates in WCF Service
IIS Architecture
In IIS 7+, the most important changes as compared to the previous versions of IIS are the inclusion of WAS and ability to independently host and process HTTP, MSMQ, TCP and NamedPipe based services (look at the possible vertical slices in the image). So you can host a net.tcp WCF service with no dependency on HTTP.SYS and W3SVC and can even switch off or uninstall these driver and service without affecting the WCF service.
Few things to note:
- HTTP.SYS is a Kernel mode process. It has (as expected) security around who can register ports with it and ask it to pass http traffic. This access is controlled by something called "URL access control lists (ACLs)". A user must be listed against a port (/ url) for him to "reserve" the rights to "register" a process (application) to access traffic coming on that port. By default, only administrator users have full rights. Use "
netsh http add urlacl…" command (with proper parameters) to add non admin users to the url ACL. This is especially true in case of self-hosted application and WCF services. Please Google "URL Reservation and Registration" for further details. - At any one time, only one process can "bind" to a port. For e.g., by default
http.sys consumes port 80 and 443. It can then bind to additional ports (if available) when for e.g., you create bindings from IIS management console. However for any reason, if you don’t get access over a desired port, and you want to know which process is using it, you can run the following command (in admin mode) and then map the PIDs with currently running processes as seen from Windows Task Manager:
netstat -o -n -a
Even http.sys can be turned off (for e.g., if you want to install Apache and turn of IIS), by going to Control Panel -> Device Manager ->(View->Show hidden devices) -> Non-Plug and Play Drivers-> HTTP and changing Startup type from "Demand" to "Disable" in Driver tab. - .NET's
HttpListener class sits directly on top of http.sys. Hence, it automatically achieves all the feature richness present in it. This has a lot of advantages over using something like TcpListener class or sockets for network programming. However, if you want to use TCP for any reason, WCF is a preferable option (over TCPListener /Sockets) considering its rich out-of-the-box features (in net.tcp). Besides, net.tcp can be configured to do port sharing (using "net.tcp port sharing service"), similar to http.sys sharing ports 80 and 443.
WCF Client Certificate Configuration
WCF is versatile, powerful and huge. I would just touch its security features superficially and that too certificates specific.
A WCF service can be configured to use "Transport" Security, "Message" Security or a mix of both called "TransportWithMessageCredential" security. In short, Transport security gets implemented at transport layer and Message security gets implemented at application layer. There are pros and cons to both, the biggest cons being, Transport security has to be configured at each hop and is protocol dependent, whereas Message security is a bit slow as it cannot use hardware acceleration. The biggest advantage of Message based security is that it can be configured for custom authentication (e.g., username/password or some security token), again as it is at application layer. "TransportWithMessageCredential" offers the best of both words.
Below are samples of configuration at Server and Client sides when using Transport security. In this case, I am using WSHttpBinding, but BasicHttpBinding can be used as well. I am omitting mex binding imagining I won’t need to auto generate proxies in Visual Studio.
Server Side (WCF service being called)
<system.serviceModel>
<bindings>
<basicHttpBinding>
<binding name="basicHttpBindingConfiguration" >
<security mode="Transport">
<transport clientCredentialType="Certificate" />
</security>
</binding>
</basicHttpBinding>
<wsHttpBinding>
<binding name="wsHttpEndpointBinding">
<security mode="Transport">
<transport clientCredentialType="Certificate" />
</security>
</binding>
</wsHttpBinding>
</bindings>
<behaviors>
<serviceBehaviors>
<behavior name="BlahServiceBehavior">
<serviceMetadata httpGetEnabled="false" httpsGetEnabled="true" />
<serviceDebug includeExceptionDetailInFaults="true" />
<serviceCredentials>
<clientCertificate>
<authentication certificateValidationMode="PeerTrust"
trustedStoreLocation="LocalMachine" revocationMode="NoCheck" />
</clientCertificate>
</serviceCredentials>
</behavior>
</serviceBehaviors>
</behaviors>
<services>
<service name="BlahDataService" behaviorConfiguration="BlahDataServiceBehavior">
<endpoint address="" binding="basicHttpBinding"
bindingConfiguration="basicHttpBindingConfiguration" contract="IBlahDataService" />
<endpoint contract="IMetadataExchange" binding="wsHttpBinding"
address="mex" bindingConfiguration="wsHttpEndpointBinding" />
</service>
</services>
</system.serviceModel>
Here,
PeerTrust (Forces a public key of the client certificate to be present in the 'Trusted People' certificate store on the service side)ChainTrust (Certificate must validate according to the complete certificate chain)
On Client side (which calls the service):
<system.serviceModel>
<client>
<endpoint address="https://www.blah.com/BlahService.svc"
behaviorConfiguration="ClientCertificateBehavior"
binding="basicHttpBinding" bindingConfiguration="BasicHttpBinding_IBlahService"
contract="BlahServiceReference.IBlahService" name="BasicHttpBinding_IBlahService" />
</client>
<bindings>
<basicHttpBinding>
<binding name="BasicHttpBinding_IBlahService" >
<security mode="Transport" />
</binding>
</bindings>
<behaviors>
<endpointBehaviors>
<behavior name="ClientCertificateBehavior">
<clientCredentials>
<clientCertificate findValue="BlahCertificate"
storeLocation="LocalMachine" storeName="My" x509FindType="FindBySubjectName" />
</clientCredentials>
</behavior>
</endpointBehaviors>
</behaviors>
</system.serviceModel>
This article provides details on how to implement client certificate authentication with Message security.
As I mentioned previously, WCF allows a powerful combination of security and binding types. The following snippet of configuration shows netTcp binding with TransportWithMessageCredential using Windows authentication at Transport layer and Certificate authentication at Message (application) layer.
<bindings>
<netTcpBinding>
<binding name="customBinding">
<security mode="TransportWithMessageCredential" >
<transport clientCredentialType="Windows" />
<message clientCredentialType="Certificate" />
</security>
</binding>
</netTcpBinding>
</bindings>
Conclusion
I hope you enjoyed this article. To repeat, please give your valued suggestions and corrections. Few more articles on security coming soon...
History
- 22nd April, 2016: Initial version
