Rationalizing access checks with HMAC:ed URLs

Introduction

This is a method (no complete implementation) to offload the database server hosting protected objects delivered through ASP.NET (Web services will work too) by simply signing a URL to the page actually delivering the material.

Example: A web site where the access checks are very complicated. When browsing, the members only see objects that they have access to, therefore we know when making img-HREFs that the member actually have access, which is why we may sign the HREF so we don't need to check again when delivering the thumbnail previews.

Note: I'm going to use the terms "HMAC" and "signature" interchangeably, meaning HMAC only. This article doesn't deal with real signatures at all.

Background

I have a photo site where members may publish photos to a limited audience. In a very sophisticated (as I'd like to think) way. Photos may be put in several categories, which have, possibly inherited, different levels of permissions for groups of members. Which makes the access lookup quite slow. Although, I have already optimized the database a little so that inheritance can be ignored when checking access because permissions are calculated for every category in beforehand. Resulting permissions for every user is also calculated.

The problem is that when a user browses a category, the server first has to list all photos that the user has access to in that particular category, then will download all the preview images causing the server to take another round doing the same thing. What makes it worse is that there has to be a separate query for each preview since they don't come in an array.

Solution

The simple solution is to sign all the URLs to the images before sending them to the client.

A random and cryptographically strong key is stored in the session object, and is left there for the entire session. Then the blobIDs together with an expiration time (in ticks) can be signed and appended to the URLs. Then, when the server receives a request for a preview image with a signature, it can be verified without asking the database for anything.

In my case, access information is bound to photo objects which have related blob objects. This method also saves me from looking up the intended blob ID.

href=http://myserver.com/blob.aspx?blobid={guid}&validity=ticks&signature=signature

There are three cases I can think of when the signature fails to validate.

• The expiration has timed out, normal. No red lights here, happens when the user saves the HTML in some way. Return a descriptive image asking the user to refresh the page.
• The session doesn't have a key. (There's no reason to create one now. Save the cycles.) Although this would be a sign that something might be wrong, we may draw no conclusions, since it happens when users don't support your session scheme. (The user might have turned off cookies.) This is no problem either, since without the user being logged in, there're no permissions to sign.
• The signature is invalid. This might also be a legitimate case, so don't panic, but beware.

One, rather big, flaw of this scheme which fortunately may be addressed, is client side caching. Client side caching here is everybody's friend, and would be impossible if we constantly gave different URLs to the same thing. Even if we manually set expiration dates.

The solution to the client caching problem is to increment the expiration dates by large steps. Minutes or hours. Obviously, you aren't hurting security by giving the users very long expiration times, since the signatures are invalidated together with the session. And also, you can never stop the users from saving the pictures to their hard drives.

One extra benefit of this scheme is that it can easily be extended to support load balancing elegantly. Any server with access to session data can check the validity of the access. (The session keys can be stored centrally.) You can even use completely separate systems delivering the actual data if it is just signed correctly.

Unfortunately, every hundred or so URL upsets the standard ASP.NET 1.1 request validation. Automatic request validation is good, but has some very bad sides too. For example, it throws an exception when the query strings contain escaped characters, like the Swedish �, �, and � which are actually very common here. And also, sometimes, valid signatures.

To turn off automatic input validation, enter <pages validateRequest="false"/> into web.config. You should really know what you're doing though, and limit this setting (with the <location> tag) to pages that you are sure validate on their own.

Finally, I know that I shouldn't assume that this is water proof, since I'm no security expert. Rather, I would very much appreciate any comments on the security of this scheme.

Using the code

There is no working sample with this article, since my implementation is quite specific, but it should easily be adjusted for other implementations.

I leave it to a future update to provide a more general solution.

// Example code for putting in a resource lister control.

// The blobIds must already be access checked, since their content is

// by signature granted to the browser.

public void ListResources(TextWriter writer, Guid[] blobIds)
{
    string[] signedBlobUrls = 
      Hallman.Hugo.WebSecurity.UrlSignerTool.SignBlobs(blobIds);
    foreach(string signature in signedBlobUrls) {
    writer.Write("<a href="fullResource.aspx?someParam=val");
    writer.Write("<img src=\"blob.aspx?");
    writer.Write(signature);
    writer.Write("\" /></a>");
    }
}

//

// The Blob.ASPX code. Validates the signature, and if it's valid, returns the blob.

// References to Database must be changed, and you should return someimage

// describing the error when the signature is legimitally invalid.

//

public class Blob : System.Web.UI.Page
{
    private void Page_Load(object sender, System.EventArgs e)
    {
        try
        {
            string data        = Request["data"];
            string signature    = Request["sig"];
            long expiryInTicks;
            Guid blobId = UrlSignerTool.EnforceBlobSignature(data, 
                                  signature, out expiryInTicks);
            string blobFilename = Database.BlobFileName(blobId);
            Response.Clear();
            Response.ContentType = "image/jpeg"; //TODO!

            Response.ExpiresAbsolute = new DateTime(expiryInTicks);
            Response.Flush();
            Response.WriteFile(blobFilename);
            Response.End();
            return;
        }
        catch(SessionKeyCreatedException err)
        {
            throw err;
            //TODO! return some descriptive image.

        }
        catch(SignatureExpiredException err)
        {
            throw err;
            //TODO! return some descriptive image.

        }
        catch(SignatureInvalidException err)
        {
            Log.LogSecurity(3, "Potentially tampered blob signature", err);
            //Don't throw. The exception is already logged.

        }
    }
}

using System;
using System.Security.Cryptography;
using System.IO;
using System.Text;

namespace Hallman.Hugo.WebSecurity
{

    class UrlSignerTool
    {
        private static RandomNumberGenerator rnd = new RNGCryptoServiceProvider();
        private static bool ArrayEquality(byte[] left, byte[] right)
        {
            if(left.Length != right.Length)
                return false;
            for(int i=0;i<left.Length;i++)
                if(left[i] != right[i])
                    return false;
            return true;
        }


        /// <summary>

        /// Puts a cryptographically strong random key

        /// in the session collection if it

        /// is not already there.

        /// </summary>

        /// <returns>The key as a byte array.</returns>

        private static byte[] EnsureSessionSecret(bool throwIfNotExists)
        {
            lock(System.Web.HttpContext.Current.Session.SyncRoot)
            {
                const string hashkey = "secretKey";

                object key = System.Web.HttpContext.Current.Session[hashkey];
                if(key != null)
                    return (byte[])key;

                lock(rnd)
                {
                    if(throwIfNotExists)
                    {
                        throw new SessionKeyCreatedException();
                    }
                    byte[] k = new byte[16];
                    rnd.GetBytes(k);
                    System.Web.HttpContext.Current.Session[hashkey] = k;
                    return k;
                }
            }
        }

        private static KeyedHashAlgorithm GetSessionKeyedHasher()
        {
            return GetSessionKeyedHasher(false);
        }
        private static KeyedHashAlgorithm 
                       GetSessionKeyedHasher(bool throwIfNotExists)
        {
            HMACSHA1 s = new HMACSHA1(EnsureSessionSecret(throwIfNotExists));
            return s;
        }

        private const int blobSignatureMaxValidMinutes = 30;
        private const int blobSignatureMinValidMinutes = 5;

        public static string[] SignBlobs(Guid[] blobIds)
        {
            KeyedHashAlgorithm signer = GetSessionKeyedHasher();
            string[] blobUrls = new string;

            //We could use (int) Environment.Ticks to save some cycles, 

            //but that's local to the machine.

            long ticks = DateTime.Now.Ticks;
            //Chop the least significant part off 

            //by max blobSignatureMaxValidMinutes.

            //This is so that the url is the same for a while 

            //so the content can be cached.

            ticks = ticks - ticks % (TimeSpan.TicksPerMinute * 
                                blobSignatureMaxValidMinutes);

            //Add a few minutes so that the client at least has time 

            //to fetch the data at least once.

            ticks += TimeSpan.TicksPerMinute*(blobSignatureMinValidMinutes+ 
                                             blobSignatureMaxValidMinutes);

            for(int i=0;i<blobIds.Length;i++)
            {
                if(Guid.Empty == blobIds[i])
                    continue;

                byte[] id = blobIds[i].ToByteArray();
                byte[] data = new byte[16 + 8]; //8=sizeof(long)


                Array.Copy(id, 0, data, 0, id.Length);
                Array.Copy(BitConverter.GetBytes(ticks), 0, data, 16, 8);
                id=null;

                byte[] signature = signer.ComputeHash(data);
                
                blobUrls[i] = string.Concat("data=", 
                 System.Web.HttpUtility.UrlEncode(Convert.ToBase64String(data)), 
                 "&sig=", 
                 System.Web.HttpUtility.UrlEncode(Convert.ToBase64String(signature))
                 );
            }

            return blobUrls;
        }
        
        public static Guid EnforceBlobSignature(string blobToken, 
                   string signaturetoken, out long expiryInTicks)
        {
            try
            {
                // throw if we need to generate the key,

                // since there's no way we could verify the 

                // signature if that is the case.

                KeyedHashAlgorithm signer = GetSessionKeyedHasher(true);

                byte[] data = Convert.FromBase64String(blobToken);
                byte[] signature = 

                Convert.FromBase64String(signaturetoken);
                if(!ArrayEquality(signature, signer.ComputeHash(data)))
                    throw newSignatureInvalidException(
                        "blobToken="+blobToken+",signature="+signature);

                long ticks = BitConverter.ToInt64(data, 16);
                if(ticks < DateTime.Now.Ticks)
                    throw new SignatureExpiredException();

                byte[] id = new byte[16];
                Array.Copy(data, 0, id, 0, 16);
                Guid blobId = new Guid(id);
                expiryInTicks = 
                ticks; return
            blobId;
            }
            catch(WebSecurityException)
                {
                throw;
            } catch(Exception exc)
            {
                throw newSignatureInvalidException("blobToken="+ 
                   blobToken+",signature="+signaturetoken, exc);
            }
        }
    }


    [Serializable]
    public class WebSecurityException : Exception
    {
        public WebSecurityException(string message, Exception innerException)
            :base(message, innerException)
        {
        }
        public WebSecurityException(string message)
            :base(message)
        {
        }
        public WebSecurityException()
            :base()
        {
        }
    }
    [Serializable]
    internal class SessionKeyCreatedException : WebSecurityException
    {
        public SessionKeyCreatedException() {}
    }
    [Serializable]
    internal class SignatureExpiredException : WebSecurityException
    {
        public SignatureExpiredException() {}
    }
    [Serializable]
    internal class SignatureInvalidException : WebSecurityException
    {
        public SignatureInvalidException(string additionalData, 
                                      Exception innerException)
            :base("additional data: "+additionalData, innerException)
        {
        }
        public SignatureInvalidException(string additionalData)
            :base("additional data: "+additionalData)
        {
        }
        public SignatureInvalidException() {}
    }
}

Performance

Naturally, you may choose any hashing algorithm you see fit, but my empirical test showed that all hash algorithms in the System.Security.Cryptography namespace except MACTripleDES were very fast. In my system, they performed 17 (TripleDES performed only 2 times better!) times better than the database lookup, and since database server's CPU-time is often more worth than the web server's, we would gain even if it wasn't faster. Also, this test is unfair in the sense that the database had extremely much better opportunities to cache what we needed and nothing else, which doesn't happen in reality. I actually expect the comparison in a production environment to be between one and five hundred times better.

Also, you might want to use only one Hash object stored statically, and instead of using a session key for the hash object, append the key to the data to be hashed. It doesn't provide the same cryptographic strength but should be good enough for most applications. (The theory is quite complicated.)

History

This is the first version of my first CodeProject article.