A New JavaScript Wrapper Library for HTML5′s IndexedDB

What is IndexedDB?

One of the exciting new benefits of HTML5 technologies is the greatly expanded ability to work with local copies of data saved on a user’s machine directly from within a web application.  Since W3C has identified the Web SQL database as obsolete, IndexedDB is quickly becoming the preferred solution for creating and managing off-line / local database solutions. As of today however, the IndexedDB API is currently implemented by only two of the major browsers (Chrome and FF).  In addition, none of the mobile browser platforms currently provide support for IndexedDB.  Nonetheless, it appears that IndexedDB will be widely supported within the near future.  The current implementation status of this and other HTML5 technologies can always be checked here.

IndexedDB is not a relational database and could be more accurately referred to as a NoSQL ObjectStore. The technology is described by W3C as “a database of records holding simple values and hierarchical objects. Each record consists of a key and some value. Moreover, the database maintains indexes over records it stores. An application developer directly uses an API to locate records either by their key or by using an index. A query language can be layered on this API. An indexed database can be implemented using a persistent B-tree data structure.” [1]

Does IndexedDB need a wrapper library?

According to W3C, IndexedDB was actually designed so a more user friendly or extended wrapper library could be “layered” on top of the API to add various features or functionality.  For example, the IndexedDB API provides no form of query language which can make complex data extractions challenging for anyone who is used to working with SQL.  These users may find a wrapper library that includes a query language layer very useful.  However, some developers who are more familiar with using object-relational mapping frameworks such as Microsoft’s Entity Framework or NHibernate may find that they actually prefer working directly with IndexedDB’s openCursor() or get() functions to extract data via an IndexedDB ObjectStore.

Since the IndexedDB API is a client-side database, data synchronization immediately stands out as an important wrapper feature candidate.  In use cases where web applications may offer clients off-line data entry, forms, record updates, or any other type of CRUD operations, data synchronization will be required to ensure that data changes made while disconnected from the internet are immediately updated to the server database once a connection becomes available.  Furthermore, data changes  from other users which may have occurred on the server must also be reconciled by the same client upon re-connection.

Writing cross-browser compatible code can also prove challenging when working with relatively new technologies.  For instance, Chrome and Firefox currently differ in the levels of their IndexedDB API implementations.  This becomes most obvious when writing JavaScript to open or create an IndexedDB database.  While Firefox is using the latest W3C specification’s onupgradeneeded event to determine if a database should be created or upgraded, Chrome is still using the older and now obsolete setVersion method.  Writing code that functions correctly on either browser can quickly become complicated.  For this and other such cases, a wrapper library for IndexedDB seems to make a lot of sense.

The ixDbEz.js wrapper library for IndexedDB

While working on a recent side project of my own, I realized that I was quickly building up a rather large collection of JavaScript for IndexedDB.  I decided to gather up all of this code and place it into one easy to use library called  ixDbEz.js (“IndexedDB Ez”).  The library is currently very simple in scope.  It is intended to accomplish the following two primary goals:

1. Allow users to easily create, modify, and update IndexedDB databases with only a few lines of code.
2. Provide data synchronization hooks for each of the ways that IndexedDB allows users to modify data within an IndexedDB database.

Goal#1 is very straightforward.  The primary goal of ixDbEz.js is to allow users to include the JavaScript library within a web application, and then quickly begin to create and use an IndexedDB database  in a very simplistic fashion.  The ixDbEz library exposes the primary functions required to create, upgrade, and modify data via the IndexedDB API.

Here is a quick JavaScript example of how to define and create or open an IndexedDB database using ixDbEz.js:

<!--The ixDbEx library is used like any other JavaScript library by including a script tag in the head section of your html…-->
<script src="scripts/ixDbEz.js" type="text/javascript"></script> 

The following code can be used in a web page’s onload event or elsewhere to open, upgrade, or create an IndexedDB database on a client machine:  

//Define the local IndexedDB database structure in a function

//This code ONLY executes anytime:
// 1. The database does not exist.
// 2. The version number passed to StartDB is greater than the
// current version number.
// 3. The version number passed to StartDB is not a valid integer.
var ixdbDDL = (
  function () {
       //Create an IndexedDB ObjectStore (a Table) and Indexes via ixDbEz
       ixDbEz.createObjStore("demoOS", "keyField", false);
       ixDbEz.createIndex("demoOS", "ixField1″, "Field1″, true);
  }
);

//Create or Open the local IndexedDB database via ixDbEz when the
// web application is accessed by the user
//Notice that the ixdbDDL variable function above is passed into the
// startDB command below.
ixDbConnection = ixDbEz.startDB("demoDb", 1, ixdbDDL);

How is this easier than using IndexedDB directly?

Although covering the nuts and bolts of the JavaScript contained within ixDbEz.js is beyond the scope of this article, the library’s code is very well documented for anyone who wants to take a closer look at or improve upon the ixDbEz.js code.  However, I think it is important to discuss a few details related to how the IndexedDB and ixDbEz internals work.  Specifically, gaining a greater understanding of IndexedDB database creation, upgrades, and connections (i.e. opening a database) are critical since these processes are slightly different than one might expect after working with other server-side databases.

The structure of an IndexedDB database can only be modified during what is called a “versionchange” transaction.   This means that the only time ObjectStores (aka tables) or indexes can be created or removed is during the ”versionchange” transaction.  Basically, a “versionchange” transaction is automatically created by the IndexedDB API anytime a database is opened via the open method and one of the following two conditions occur:

1. The requested database does not exist.
2. The requested database’s version number is greater than the version number of the database on the local client’s machine.

During these situations, the latest version of the IndexedDB API requires that the onupgradeneeded event be fired.  Within this event, developers can create or make changes to ObjectStores (aka tables) or indexes.  This is, in fact, the only time that these structures can be updated.  As previously mentioned, writing cross-browser compatible code to support this process in both Chrome and Firefox can be challenging since Chrome does not yet fire the onupgradeneeded event.

The ixDbEz library uses a JavaScript variable to contain the current structure of any IndexedDB database.  This variable, shown in the example above as “ixdbDDL”,  is a complete JavaScript function which should always contain the commands required to build the entire IndexedDB database.  In addition, the ixDbEz.startDB command accepts three arguments which include the database name, version, and the “ixdbDDL” variable that contains the current IndexedDB database structure function.  When ixDbEz.startDB executes, the process first attempts to open the database.  In cases where the database does not exist or the version number indicates that an upgrade is required, ixDbEz.startDB behaves differently depending on which browser it is executing in.  Chrome triggers the older IndexedDB setVersion command, and Firefox responds by triggering the latest IndexedDB specification’s onupgradeneeded event.  However, no actual browser detection is used to facilitate this process.  So, when Chrome does eventually support the new onupgradeneeded event, ixDbEz will continue to work with no modifications required.  During the ixDbEz.startDB process, the library also defines a connection object and common error / process logging objects.  All other processes within the library take advantage of these common objects.

The current process of IndexedDB database creation, upgrades, and connections (i.e. opening a database) can quickly become complex.  The ixDbEz library achieves Goal #1 by allowing users to easily create, modify, and update IndexedDB databases with only a few lines of code.  Anyone can compare the example shown above with the ixDbEz.startDB source code to quickly understand how using the library saves time and is easier than using the IndexedDB API directly.

What are the data synchronization hooks in ixDbEx.js?

Data stored within any IndexedDB database can be quite volatile when compared to data stored within other server-side database technologies.  For instance, Chrome, Firefox, and most other browsers offer users complete control over the data that is stored on their own device.  When an IndexedDB database is created for the first time on a user’s device, the browser prompts the user requesting permission to store data on the user’s local machine.  Furthermore, browser settings allow the user to remove or “clean up” browser data at anytime with only a few clicks.  This means that developers cannot depend on these databases for critical data without a rock-solid data synchronization plan in place.

In cases where data is not entirely critical or seldom updated, local databases can be great for reducing server traffic and round-trips.  Local databases can also be used in combination with other HTML5 off-line features to offer users access to web applications even when they are not connected to the internet.  This functionality would be great for reference sites and other applications that may require users to capture information in remote locations where internet access is not available.  When data is important or changing frequently, web applications with local databases will require the ability synchronize data changes between various online / off-line users and the server database which would typically represent a master copy of the data.  In these scenarios, the data synchronization hooks within ixDbEz allow developers to quickly plug into the library and trigger their own custom data synchronization processes whenever data changes occur within an IndexedDB database.

All ixDbEz functions include two optional event style variables called onsuccess and onerror.  These variables can optionally be used to pass entire custom JavaScript functions into any ixDbEz wrapper function providing the perfect hook for any custom data synchronization, error handling, or other processes that should execute upon success or failure of an ixDbEz function call within a web application.

The following example demonstrates how a new record can be inserted into the “demoOS” ObjectStore (aka table) that we created in the previous example above:

//Define a new record using a JavaScript object.
//Notice that the primary key value (defined as "keyField" when "demoOS"
// was created above) is passed with the newRecord object.
var newRecord = {};
newRecord.keyField = "ThePrimaryKey";
newRecord.Field1 = "Field1_Value";
newRecord.Field2 = "Field2_Value";
newRecord.Field3 = "Field3_Value";

//Add new record to the local database using the ixDbEz.js add command.
ixDbEz.add("demoOS", newRecord);

Creating a new data synchronization hook for this process would be as simple as defining a function and passing it through the onsuccess argument of the ixDbEz.js add function:

//Create a function and add it into a new variable named "myCustomHook".
var myCustomHook = function () {
                     alert("Data synchronization logic could go here!");
                   };

//Add new record and data synchronization hook to the local database
// using the ixDbEz.js add command and onsuccess argument.
//Notice the JavaScript value of undefined is used in this example since
// the Key argument must is not needed.
ixDbEz.add("demoOS", newRecord, undefined, myCustomHook);

Conclusion

It is important to note that the current hooks provided in ixDbEz are just simply standard JavaScript callback functions that execute upon success or failure of the requested ixDbEz function.  They do not perform actual data synchronization between a client IndexedDB database and a server database.  Instead these “hooks” are intended to allow developers the ability to easily utilize any function within ixDbEz.js, and then to further extend that function as needed to achieve the requirements of their specific web application.  A future addition to the ixDbEz.js family will be the ixDbSync.js wrapper library which will actually perform client and server-side data synchronization for all ixDbEz.js function calls.

The ixDbEz wrapper library can save developers time and effort during the development and deployment of client side databases using the IndexedDB API.  

Project Resources 

• ixDbEz JavaScript
• ixDbEz Web Demo  (Only works on the latest versions of Chrome or Firefox)
• ixDbEz GitHub Project

References and Useful Links

1. http://www.w3.org/TR/IndexedDB/
2. http://www.caniuse.com/
3. http://mobilehtml5.org/
4. http://www.codeproject.com/Articles/325135/Getting-Started-with-IndexedDB
5. http://html5-demos.appspot.com/static/html5storage/index.html#slide30
6. http://www.jakemdrew.com/
7. http://www.jakemdrew.com/Blog.aspx