HTML5 and CSS3 Part 10: Web Storage Wizardry

Introduction

Storing data on the client side of a website may not be something you have though of before. If you have, you probably looked into cookies and left it at that. In HTML5, there is so much more you can do with client-side storage of data! I am going to walk you through some of the benefits of client-side storage and then dive into the relatively easy process of storing and retrieving data. We will look at the different options you have, the safety of the stored data and what you can do to take web storage to the next level.

What is Web Storage

Web storage is a mechanism in HTML5 that allows us to store string data in key/value pairs on the client. For example, you could put a person's first name in storage using the key of "FirstName" and the value of "Sue". If there was not a key called "FirstName" already, it will be created. If there was a key named "FirstName" already, it will be overwritten without warning. However, this is not typically a problem, since web storage is so specific about how it stores data.

Storage of data in web storage is sandboxed. That means that sites only have access to their own sandbox. They cannot gain access to any other site's storage. A site, and thus its sandbox, is defined by:

• The domain name (e.g. stackoverflow.com)
• The sub-domain name (e.g. careers.stackoverflow.com)
• The protocol (e.g. http or https)
• The browser (e.g. Chrome or FireFox)
• The session type (e.g. normal or in-private)

In order for a website to have access to web storage data, all of the above items need to be the same. Session storage is further segmented to the same tab as well, meaning the same site open on two different tabs of a browser cannot gain access to the same sessionStorage sandbox. They would each get their own.

Web storage is also broken up into two different types of storage: Local Storage and Session Storage. Local storage is persistent on the computer "forever". In reality, it only stays until the storage is cleared by the user (clear cache). Session storage persists only as long as the session persists. This means that when you close the web page, the storage goes away. Be aware, though, that this might not always be the case. For instance, if a browser crashes, it may restore the session and the session data. Also, some browsers allow closed tabs to be reopened with their session data intact.

Another important thing to know about web storage is that it is quite spacious. We can store up to 5 MB of data in any one sandbox. Some browsers may extend this limitation even further but 5 MB is the recommended size. Since we are storing string data, that is an enormous amount of space, especially compared to the 4k limit on cookies. It is also important to note that, unlike cookies, web storage data is not transmitted to the server automatically. Since we can store up to 5 MB, that is a very good thing.

Finally, when using web storage, you need to be aware that it is not secure storage. Other sites do not have access to the data, but that does not mean that the data is protected. Data in web storage can be read on the client using basic web tools. That means that you should never store sensitive data in web storage without first encrypting it.

When to Use Web Storage

Storing data on the client is not something we are going to use every day. However, there are a number of use cases where web storage becomes amazing. Let's look at a few examples of where we can use web storage effectively:

Long Surveys

Filling out response after response can be tiresome and frustrating. Users tend to wander away from the page. Maybe they they close the browser down accidentally. If they were to come back to the survey page later, they might not complete it because of all of the lost work. If you had local storage in place, you could restore all of the questions they already answered and allow them to pick up where they left off. You could even make that a feature of the survey: take the time you need and come back to it later - your work will still be here.

Infrequently-changing Data

If you have a page that is data-driven, but the data does not change often, you can reduce the calls to your database by storing the data in local storage. For example, say you have a dropdown box that lists the countries that your company provides service to. That list can be stored at the client side. That way you do not need to load it from the database every time. You could store a variable that tells you when to update the list or you could pass that with your initial page load. That would mean a significant reduction in database traffic if you have a high repeat visitor rate.

Site Settings

You can allow visitors to customize your site (theme, important links, layout, etc.) You can store this customization data in web storage. That way, every time they come to your site they get the look they want. You could make your website very different depending on the relationship a visitor has with the website. Amazon does this quite a bit using a mix of server-side and client-side information to determine what products to show on the homepage. You could do the same using web storage.

Form Submission Backup

Form data is typically important to the owner of a website. You really do not want to throw away information that the user was willing to give you. However, what happens when a user fills out your form and the site goes down, even for a second, or the submission has a glitch? All of that data is lost. The best case scenario is that the user fills out all of the data again and re-submits it. However, that probably will only be a small percentage of users. Worse yet, if the form had a field that asked for feedback in paragraph form, the second time the user submits the data they will probably include less details because they do not want to type all of that information over again. This is where local storage or session storage can be used to great effect. You can store the data as they move between fields. Then, if the submission fails, you can retry the submission or, if you used local storage, ask the user to come back later to submit the data.

Offline / Cached Data

If the data service that you rely on goes down, you can use a backup of the data that you stored in web storage from the last time the data was requested. This might not work for all data, but it might be better than nothing. It also might reduce requests to the web service. For example, if the user does a search for the same term more than once you could return the cached data instead of hitting the web service again.

Using Web Storage

For simplicity sake, from now on, I will refer to session storage and local storage collectively as storage or web storage. In my examples, I will show you how to do the same task with each type of storage and I will point out any differences as they come up.

Let's first look at how to store data using web storage (each line represents one method - any are acceptable):

// Local Storage
localStorage.setItem('key', 'value');
localStorage.key = 'value'
localStorage['key'] = 'value'

// Session Storage
sessionStorage.setItem('key', 'value');
sessionStorage.key = 'value'
sessionStorage['key'] = 'value'

Now let's look at how to retrieve data:

// Local Storage
var value = localStorage.getItem('key');
var value = localStorage.key;
var value = localStorage['key'];

// Session Storage
var value = sessionStorage.setItem('key');
var value = sessionStorage.key;
var value = sessionStorage['key'];

The really cool thing is that you don't have to remember which method you used for saving the data in order to retrieve it. Any of the three methods will retrieve data stored using any of the three methods. There are some speed differences in the three methods, but that is a changing target. It really comes down to which you decide best fits your workflow.

If you want to remove an item from storage, you can do so like this:

// Local Storage
localStorage.removeItem('key');
delete localStorage.key;
delete localStorage['key'];

// Session Storage
sessionStorage.removeItem('key');
delete sessionStorage.key;
delete sessionStorage['key'];

Each of these removes the item from storage, so if you were to try to access that key you would get "undefined". If you decided that you wanted to wipe all of your storage at once, you could so so using the clear method like so:

// Local Storage
localStorage.clear();

// Session Storage
sessionStorage.clear();

To check to see how many objects are stored in storage, you can run the following command:

// Local Storage
var itemsCount = localStorage.length;

// Session Storage
var itemsCount = sessionStorage.length;

That will give you an integer value that indicates how many items are currently being stored in the storage mechanism of your choice.

Web Storage Limitations

The specification for web storage says that 5 MB should be reserved for each sandbox of storage. Some browsers may allow for extra space or they may allow the user to specify how much space each sandbox gets, but this is not something that the website can change or affect in any way. Since we have a limit, we have to plan for the eventuality that we hit the limit.

When you exceed the storage limit for your particular sandbox, you will get a QUOTA_EXCEEDED_ERR error. You should keep an eye out for this in your code to be sure you handle this event, should it occur. To do so, you can use a try/catch like so:

// Local Storage
try {
   localStorage.key = value;
} catch (e) {
   if (e.name === 'QUOTA_EXCEEDED_ERR') {
      // The quota has been exceeded. Maybe put a clear() here 
      // or make some room in some other way
   } else {
      // A different error occurred
   }
}

// Session Storage
try {
   sessionStorage.key = value;
} catch (e) {
   if (e.name === 'QUOTA_EXCEEDED_ERR') {
      // The quota has been exceeded. Maybe put a clear() here 
      // or make some room in some other way
   } else {
      // A different error occurred
   }
}

The trick here is that not all browsers will implement the proper error message. It may be safest to wrap just the set storage line in a try/catch like we have done here and then just assume that the error means that the storage is full.

Since we can hit an error when the storage is full, it would be nice to know when we are getting close to our limit. In Internet Explorer, there is a "remainingSpace" property on the storage object that gives the remaining space in bytes. Unfortunately, IE is the only browser to implement this. Unless you want to develop your site for IE-only, you shouldn't use this property. Instead, the best way to figure out how much space you have left is to figure out how much space you have used and then assume a 5 MB storage limitation. For example, here is a function that will tell you how many bytes you have used so far in storage:

// Local Storage
function localStorageUsed() {
    var output = 0;
    for(var x in localStorage){
        output += (localStorage[x].length * 2);
    }
    return output;
};

// Session Storage
function sessionStorageUsed() {
    var output = 0;
    for(var x in sessionStorage){
        output += (sessionStorage[x].length * 2);
    }
    return output;
};

You could calculate the remaining space by subtracting that number from five million. Even though 5 MB is technically 5.3+ million bytes, at least Internet Explorer uses the rounded 5 million for the storage space instead. It is better to underestimate your space requirements than to overestimate them.

The final major limitation of web storage is the implementation of it. Not all browsers implement web storage. Most notably, Internet Explorer 6 and 7 do not implement it at all and IE 8 has a limited implementation. To verify that the client browser supports web storage, you can use the following code:

// Verifies both Local Storage and Session Storage
if(typeof(Storage) !== "undefined") {
   //Both localStorage and sessionStorage are supported
} else {
   // Web storage is not supported
}

The only problem with this test is if someone defines a Storage object. In that case, the test would pass without truly verifying web storage. Modernizr goes a bit further in order to get around a few edge cases by putting a try/catch around a setItem function call and a removeItem function call to localStorage. If either of these commands fail, the browser does not support web storage. If both succeed, then the browser supports web storage.

Web Storage Event

When you store data in web storage, an event gets triggered. This event indicates which key was affected, what the old value was and what the new value now is. This event can be used to watch what other pages are doing. That brings up the biggest limitation of this event: it does not fire on the page that raised the event. This means that for all practical purposes, this event is limited to local storage.

To capture this event, first you need to register a listener to the event like so:

window.addEventListener('storage', handleStorageEvent, false);

Next, you need to create the handleStorageEvent function (or whatever you called it) like so:

function handleStorageEvent(e) {
   // Use e.key for the key that was changed
   // Use e.oldValue to get the old value of the key
   // Use e.newValue to get the new value of the key
}

This event will now fire whenever another page in the same sandbox (typically another tab) changes the data.

Advanced Web Storage

Storing a first name, a last name or even a paragraph of text is great, but there are times when it just isn't enough. While web storage does have the limitation that it only stores strings, there are ways to make the most of the strings we store. Let's look at a couple different examples of how to make the most out of web storage.

Sets of Data

In JavaScript, we typically deal with objects that contain a number of properties. For instance, an object might contain a first name, a last name, and a zip code. This data might be used to bind to the inputs on our form. Here would be an example of that in JavaScript:

var model = {
   firstName: 'Tim',
   lastName: 'Corey',
   zipCode: '18411'
};

Say you want to store this data in web storage. You could store each property in its own key/value pair like so:

// Local Storage
localStorage.firstName = model.firstName;
localStorage.lastName = model.lastName;
localStorage.zipCode = model.zipCode;

// Session Storage
sessionStorage.firstName = model.firstName;
sessionStorage.lastName = model.lastName;
sessionStorage.zipCode = model.zipCode;

That can quickly get unwieldy. Even this example is ugly. To make this simpler, we can store the entire JavaScript object in one key/value pair. The key is to use the JSON.stringify() method to first convert the object into a string:

// Local Storage
localStorage.model = JSON.stringify(model);

// Session Storage
sessionStorage.model = JSON.stringify(model);

Later, when you want to re-populate your model, you will need to use the JSON.parse() method like so:

// Local Storage
model = JSON.parse(localStorage.model);

// Session Storage
model = JSON.parse(sessionStorage.model);

The cool thing here is that it does not matter how many properties you have on your model. You don't even need to know all of the names of your properties. You just save the entire model and you are good to go. It is less code for you to maintain. It also handles any number of properties on your model without needing to know about them. You could write some more complex code to look at every property attached to your model (that was not inherited) and then store each one individually (hoping not to step on the toes of any currently-used key) but that seems like a lot of work. This solution is much cleaner.

Images and the Canvas

While there are better ways to cache images (see App Cache), there may be times when you want to store a canvas object in web storage. For example, if you allow a user to create an image on the canvas, you may want to save that image for later use. To do this, you would need to export the canvas JavaScript object using the toDataURL() method like so:

// JavaScript setup
var canvas = document.getElementById('canvasId');
var context = canvas.getContext('2d');

// Do any work on the canvas here

// Export the canvas object to string
var canvasString = canvas.toDataURL();

// Local Storage
localStorage.canvasId = canvasString;

// Session Storage
sessionStorage.canvasId = canvasString;

To later gain access to that canvas, you would need to do something like this:

// Local Storage
var canvasString = localStorage.canvasString;

// Session Storage
var canvasString = sessionStorage.canvasString;

// JavaScript setup
var canvas = document.getElementById('canvasId');
var context = canvas.getContext('2d');

// Sets up a new image that will use the canvas text
// as the image source
var canvasImg = new Image();

// Fires an event once the image loads that actually puts the image
// onto the canvas
canvasImg.onload = function () {
   context.drawImage(this, 0, 0);
};

// Assigns the text to the source of the image
canvasImg.src = canvasString;

That is a bit more complicated and it is definitely an edge case. However, in the rare case where something like this is needed, web storage will get the job done.

Conclusion

Web storage isn't for every site, but in certain circumstances it is very powerful and easy to use. With it, we can save calls to our database, improve the user experience and recover from crashes elegantly. In this short article, we were able to walk through what web storage is, how to use it and how to make the most of its capabilities. Now all that is left is for you to implement this technology in your site.