Demystify http.sys with HttpSysManager

• Go to HttpSysManager on Codeplex

Demystify http.sys with HttpSysManager

• Spotlight on HTTP.SYS
• Routing, registration, URL ACL and delegation
• The case of HTTPS
• Show me the code
• Roadmap

Spotlight on HTTP.SYS

Before Windows server 2003, life was simple for http servers creator : just open a socket on an endpoint (IP:Port), listen incoming traffic and parse it.
All was fine and good, except that all applications, for firewall reason, wanted to use port 80 (http) and 443 (https).
As you might know, with the socket model, only one application can listen an endpoint at any given time (IP:Port).
Everybody claimed port 80 and 443, and terrible wars started on internet to know what the ruler would be.

Then Windows server 2003 server released, and a new kernel driver borned, Microsoft gave him the name of http.sys.
This driver is meant to listen http traffic and dispatch based on the URL to processes : now multiple processes will be able to listen HTTP traffic on the same port.
Here is a simple proof :

A public windows API was created around the new born called HTTP Server API, and also a set of tools making use of it like httpcfg (the old way) or netsh http (the new way).

IIS depends on http.sys, as well as our beloved class HttpListener which is in really a simple wrapper around the HTTP Server API and so is WCF Http(s)TransportBinding.

You don't believe me ? You think it is a conspiracy ? Check it yourself.

Translation : Impossible to get any information about the owner. (because a kernel driver, http.sys, is not a process)

And what are the implication for you, fellow developer ? The implication is wierd production deployment crash like this one.

For google referencement and all desperate developers that will ask google to save their soul, I copy the message for you, in plain english :
Unhandled Exception: System.ServiceModel.AddressAccessDeniedException : HTTP could not register URL http://+:80/634800377949733185/. Your process does not have access rights to this namespace (see http://go.microsoft.com/fwlink/?LinkId=70353 for details). ---> System.Net.HttpListenerException: Access is denied

The question I will first respond, is the one you ask just after seeing this message. This is a very famous question among developers, and an endless source of debate: Why in the hell did it work on my machine ?

The show answer is also very common: It worked because you was in the local administrators group of your machine.

The long answer is that it does not work because the user in production environment was not admin and the user in production environment did not have right on the URL http://+:80/.

Routing, registrations, URL ACL and delegation

When IIS, or any application that use HTTP Server API listen on some HTTP request path, they need to register a url prefix on http.sys, we call this process registration.

When an incoming request is picked by http.sys, it needs to deliver the message to the right registered application, we call that routing.

As you have seen, when the application register every thing works fine if you are local administrator. If you are not, URL ACL are checked.

An URL ACL is just a URL prefix associated to an ACL.

.

You can play with it with command line netsh http show|add|delete urlacl, but I don't recommend it, because I created a -almost- nice UI around HTTP Server API : HttpSysManager.

With some familiar UI to set ACLs... (Tout le monde = Everyone)

Some of you may recognize that this is the ACL dialog for the callback url for duplex HTTP scenario in WCF... this ACL is set during WCF installation, so that every user can register callback addresses.

Given my current ACLs, a registration to http://+:80/ReportServer/test/blabla will use the ACL http://+:80/ReportServer.
On the other hand a registration to http://+:80/test/blabla will use the ACL of http://+:80/.

Always remember : the longest match rule applies.

You can see there is two different rights in the ACL window : register and delegate (Special authorization does not apply), and we covered register.

Delegation means that you give rights to group or user to add, modify and remove url acl for sub request path.

Example, here is the rights for http://+:80/ and Guest user :

If I run HttpSysManager as Guest and try to add the user NICO to http://+:80/ReportServer's acl, here is the result :

In plain english : it does not like it, and I can say that's the best case... My application did not like and crashed ! " src="http://www.codeproject.com/script/Forums/Images/smiley_smile.gif" />

Go back in admin user, and give delegation rights to Guest on http://+:80/:

Run again in Guest user, and now you can add Nico to acl of http://+:80/ReportServer.

The case of HTTPS

http.sys can listen both, http and https traffic. And it ships with some really nice feature specific for https.

Similar to the HTTP case, IIS, WCF and the world relies on HTTP Server API for HTTPS. (except people that just use good old socket)
The most obvious feature that http.sys expose is the possibility to bind a certificate to an endpoint.(IP:Port)

The certificate should be installed to the LocalMachine certificate location, because http.sys does not run in the context of any user.

Then you can use netsh http add|delete|show sslcert certhash= to bind the endpoint to the server certificate, but HttpSysManager makes it easier.

Here is how to import or select the certificate in a store and bind to endpoint with HttpSysManager.

You can see in this screenshot an ssl binding already registered by IIS on port 444, and I am adding a new one on port 443.

Click on Select certificate, and choose your certificate (with private key) from file or from your machine's store.

And that's done, now all https traffic from this endpoint will use the certificate HttpSysManager to authenticate and encryption.

A note on the Negotiate client certificate checkbox :

Once an ssl session is established, the application can choose exactly when to ask client's certificate for mutual authentication.
For example, you can configure IIS to require client certificates only when they are accessing the virtual folder Secure_Data...

At this point, the server and client negociate a new SSL session (renegociation) and the client sends its certificate.

All is well, except in some case, some of your clients might not support to renegociate the SSL session.

In this case, you can ask http.sys to always require the client to send certificate when an SSL session is establishing by checking Negociate Client Certificate.

A note for the second option : Directory Service client certificate Mapping

If checked, http.sys will ask the AD to give the access token (Windows identity) corresponding to the client certificate, and will pass it to the application. (For impersonation, or authorization purpose)

For more information on that, check this link.

Show me the code

The code makes heavy use on pInvoke to interact with the HTTP Server API. For the Url Acl part I was lucky enough to find a code snippet on google that wrapped all SecurityDescriptor stuff from Process Hacker V1.
I end up referencing all the library, and modifying the sources to add some methods.

The high level class that wrap the HTTP Server API is HttpApiManager along with UrlAcl and SSLInfo.

No surprise on this part, properties of these object reflects what you see in the UI.

Note that UrlAcl.SecurityDescriptor, hold informations about the ACL, the SecurityDescriptor is a beat on itself... I borrowed the class created by Process Hacker V1

The HttpApiManager just depends on tons and tons and tons of pInvoke wrapper functions around Win API.

I pass the details, it is not very interesting and was a pain in the ass to code.

The interesting part is how I reused windows UI for my puropse again, for example, for Url ACL, I reused Process Hacker V1 classes, especially the SecurityEditor.EditSecurity method.

public static void EditSecurity(IWin32Window owner, ISecurable securable, string name, IEnumerable<AccessEntry> accessEntries);

private void Permissions_Click(object sender, RoutedEventArgs e)
{
	var acl = (UrlAcl)((FrameworkElement)sender).DataContext;
	SecurityEditor.EditSecurity(null, acl, acl.Prefix, GetAccessEntries());
}

The ISecurable interface are callback functions to implement that the ACL window use to get current ACLs, and set new ACLs. That is a fully managed one from Process Hacker that I implemented on UrlAcl...

Here is what I would have to implement without Process Hacker V1 :

[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
[Guid("965fc360-16ff-11d0-91cb-00aa00bbb723")]
public interface ISecurityInformation
{
	HResult GetAccessRights(ref Guid ObjectType, SiObjectInfoFlags Flags, out IntPtr Access, out int Accesses, out int DefaultAccess);
	HResult GetInheritTypes(out IntPtr InheritTypes, out int InheritTypesCount);
	HResult GetObjectInformation(out SiObjectInfo ObjectInfo);
	HResult GetSecurity(SecurityInformation RequestedInformation, out IntPtr SecurityDescriptor, bool Default);
	HResult MapGeneric(ref Guid ObjectType, ref AceFlags AceFlags, ref int Mask);
	HResult PropertySheetPageCallback(IntPtr hWnd, SiCallbackMessage Msg, SiPageType Page);
	HResult SetSecurity(SecurityInformation SecurityInformation, IntPtr SecurityDescriptor);
}

I am certainly confident with my coder's skill, it is feasable, but I am not masochist.
All the classes in this interface are also composed of IntPtr... It is not plain old C# object, but plain old C object.

Here is what I needed to implement thanks to Process Hacker.

public interface ISecurable
{
	SecurityDescriptor GetSecurity(SecurityInformation securityInformation);
	void SetSecurity(SecurityInformation securityInformation, SecurityDescriptor securityDescriptor);
}

And that's all plain old .NET object my friend !

Here is the implementation in UrlAcl:

SecurityDescriptor ISecurable.GetSecurity(ProcessHacker.Native.Api.SecurityInformation securityInformation)
{
	return SecurityDescriptor;
}

public void SetSecurity(SecurityDescriptor securityDescriptor)
{
	((ISecurable)this).SetSecurity(SecurityInformation.Dacl, securityDescriptor);
}
void ISecurable.SetSecurity(ProcessHacker.Native.Api.SecurityInformation securityInformation, SecurityDescriptor securityDescriptor)
{
	if(securityDescriptor.ToString() == SecurityDescriptor.Empty.ToString())
		return;
	var manager = new HttpAPIManager();
	manager.SetUrlAcl(Prefix, securityDescriptor);
	Update(manager.GetAclInfo(Prefix)._Acl);
}

The other interesting part is how I reused the certificate selection windows :

This one was also simple, .NET provide a full manager wrapper around certificates structures : X509Certificate2.

I created a method : Security.SelectCertificate(), to show certificates from the LocalMachine/My store.

Take a look at how I use X509Store and X509Certificate2, to get informations that the unmanaged function Security.CryptUIDlgSelectCertificateW(intputPtr); uses.
I commented the interesting parts.

public static X509Certificate2 SelectCertificate()
{
	bool storeOpened = false;
	var store = new X509Store(StoreName.My, StoreLocation.LocalMachine);
	IntPtr result = IntPtr.Zero;
	IntPtr intputPtr = IntPtr.Zero;
	var array = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(IntPtr)));
	try
	{
		store.Open(OpenFlags.ReadOnly);
		storeOpened = true;
		Marshal.WriteIntPtr(array, store.StoreHandle); //array hold the list of Store to select from. I use X509Store.StoreHandle to get the handle that the unmanaged function need.


		int size = Marshal.SizeOf(typeof(CRYPTUI_SELECTCERTIFICATE_STRUCTW));
		CRYPTUI_SELECTCERTIFICATE_STRUCTW input = new CRYPTUI_SELECTCERTIFICATE_STRUCTW();

		input.dwSize = (uint)size;
		input.cPropSheetPages = 0;
		input.dwDontUseColumn = 0;
		input.dwFlags = 0;
		input.hSelectedCertStore = IntPtr.Zero;
		input.hwndParent = IntPtr.Zero;
		input.pDisplayCallback = IntPtr.Zero;
		input.pFilterCallback = IntPtr.Zero;
		input.pvCallbackData = IntPtr.Zero;
		input.rghDisplayStores = array;
		input.cDisplayStores = 1;
		input.rghStores = IntPtr.Zero;
		input.cStores = 0;
		input.rgPropSheetPages = IntPtr.Zero;
		input.szDisplayString = null;
		input.szTitle = null;


		intputPtr = Marshal.AllocHGlobal(size);
		Marshal.StructureToPtr(input, intputPtr, false);
		result = Security.CryptUIDlgSelectCertificateW(intputPtr);
		if(result == IntPtr.Zero)
			return null;
		return new X509Certificate2(result); //I create X509Certificate2 from the handle I got from Security.CryptUIDlgSelectCertificateW
	}
	finally
	{
		if(storeOpened)
			store.Close();
		if(intputPtr != IntPtr.Zero)
		{
			Marshal.DestroyStructure(intputPtr, typeof(CRYPTUI_SELECTCERTIFICATE_STRUCTW));
			Marshal.FreeHGlobal(intputPtr);
		}
		if(array != IntPtr.Zero)
			Marshal.FreeHGlobal(array);
		if(result != IntPtr.Zero)
			Security.CertFreeCertificateContext(result); //X509Certificate2 duplicate the handle internally
	}
}
}

Roadmap

In fact, there is much more possibility with the HTTP Server API, and here are some of the next steps I will implement.

• script (.bat) generation
• CTL (Certificate Trust List) Creation and binding to SSL endpoint
• HTTPS SNI (only windows 8)
• Support other store than MY