Introduction
About six months ago, my manager approached me with a question: "We're going to start a new consumer facing web application, what technologies should we use?"
I'm a longtime fan of rich internet application (RIA) for creating line of business web applications, but because of some thoughts of Steve Jobs and our web app needs to be accessible by increasingly popular consumer hand held web-enabled devices, I had to table the options of using Flash/Flex/Silverlight to build the entire site. The attention turned to those classic web server page technologies, like ASP.NET, ASP.NET MVC, JSP, PHP, Ruby on Rail, ColdFusion.... Since we've already had RESTful services that can be leveraged for business logic and data access in the middle tier, plus each of those web server frameworks tend to couple the presentation with data access tier with their own scripting languages, after evaluating, the conclusion turned out to be that those server pages technologies are not very helpful for building a loose coupled, rich-interactivities enabled, service oriented, widely accessible web application.
Additionally, modern browsers have more exciting programmable features and Internet Explorer becomes more web standard compliant since IE8, JavaScript libraries (jQuery and jQuery UI, Prototype and Prototype UI, MooTools, ExtJs, Yahoo! UI Library (YUI), the Dojo Toolkit, DHTMLX, to name a few popular ones) are more compact, flexible and powerful to build and manipulate DOM on the fly, CSS and CSS3 are more capable building visually compelling layout and visuals, HTML 5 is on the way to come, more and more service oriented architecture is applied to applications, client side computing capability is becoming much more powerful by multi-core client machines, etc. All these factors helped me to confirm that a web app can be built without any of those server pages frameworks.
This made me to think, how about building a "RIA" without server page and with no dependency on browser plug-ins? Wouldn't it achieve high scalability by having web server just serve static HTML/JavaScript/CSS/Images files? It could also enable basic access by mobile devices, better leverage existing already tested RESTful services as back end. The idea started to become a very interesting topic and prompted me to dig deeper.
I'm absolutely certain about manipulating DOM at runtime via Ajax, creating page content dynamically on the client should be no issue. However, with static files of HTML/CSS/JavaScript and without the help from server pages, how to handle user session? How to navigate securely? How to deal with user book marked URL after logout? How to transfer user data from page to page? How to detect cookie/JavaScript settings in end user's browser? Besides, more advanced topic, how to share common HTML contents like header and footer across multiple static HTML files? (Client side master page?) How to handle user preferences by applying different experiences after user logs in?
After I answered all those questions and had a working prototype, I replied my manager with my recommendation: we can build the web app with web standard technologies ---- HTML/CSS/Ajax plus Apache ---- to build a RIA without web server pages generating any HTML on the server and it doesn't require browser plugins like Flash Player/Silverlight/JavaFX, our existing RESTful service can be reused as backed with very little or no change at all, and client and server stay loose coupled.
This is where SOHA - Service Oriented HTML Application originated, let's talk more about it.
What is SOHA?
SOHA, Service Oriented HTML Application, is a web application programming model that leverages HTML as Model template, CSS as View and JavaScript as Controller, utilizes Ajax to interact with web services. It promotes web standards, achieves high scalability by eliminating the need for classic web server pages (JSP, ASP, PHP, Ruby On Rail, ColdFusion, etc.) on server slide, and also enables rich interactivities without browser plug-ins (Flash/Flex, Silverlight, JavaFX, etc...) on client side.
Service Oriented HTML Application is an extension to Ajax that has 3 major tenets:
- App Server hosts a presentation agnostic service and data access tier that serves data via HTTP, it focus on data access and business logics. Its API usually designed in RESTful style, responses client request in either JSON or XML format, and also can serve different client applications (web, desktop, mobile) at the same time;
- Web Server only handles HTTP, no server page framework further processes HTTP request for generating HTML, web front tier consists of a set of static files of HTML, CSS, JavaScript, and other digital assets, web server is a pure http server to serve these static files upon request, in other words, HTML is served, not generated on the server;
- Client side promotes web standard technologies (HTML, CSS, JavaScript) to create content layout, visual effects and rich interactive user interfaces, it only requires JavaScript enabled at client browser, no browser plugin is required, including Flash player, Silverlight runtime or JavaFx.
SOHA extends the scope of Ajax from building page's regions and interacting with services asynchronously to a higher application scope, including: manages application navigation, dynamically constructs pages by reusing shared parts at client side, changing entire application look and feel and behavior by conditionally apply CSS and JavaScript, manages secure page's user authorization, keep track of authenticated user's data for from page to page data transfer, etc.
With SOHA, web services and data access tier can use WCF and IIS with .NET 4.0 to create RESTful style services, the request and response for services are usually designed in JSON format, mainly because JavaScript will be its primary consumer. The key consideration for service tier is that it need to expose service API in a standard way, as long as the service API is in RESTful style, payloads in JSON format, can be consumed by different applications on varies platforms, we would say this is a SOHA-friendly web services. As for the implementation, the service can be implemented by WCF, it can also be done by ASP.NET MVC with RESTful endpoint, or by another technologies that can help building RESTful service effectively.
At work, I created the service tier using Ruby on Rail (no HTML in the service project), then later converted to Java EE services on GlassFish by other team members. It's not important what to use, it's important how the services are exposed. Usually it's a good idea to have application architecture depend on standards rather than a specific vendor's framework; it would generate high confidence on flexibility and predictability in architectural level. To design web service API in RESTful style and JSON format for its payload is an important consideration in SOHA to enable the reusability and presentation agnostic aspect at the service tier.
Behind the service tier, it usually has database for persistence. Once again, SOHA has no interests on which database engine to use, it only emphasis on data access needs to be exposed by REST service. In reality, when a service framework is picked for the technology stack, database engine can be finalized based on the factors of how efficient the database would work with the service framework and how much expertise on the selected database within the team. With the project at work, it starts with SQLite with Ruby on Rail, then migrated to MySQL while data capacity grows, eventually switched to Oracle when we have dedicated service and database player within the team. During database updates, migrations and switches, the changes are pretty much well encapsulated in the service tier, service API stays fairly stable, therefore there wasn't any change required for client Ajax code.
Service and data access tier is usually hosted on an App Server, database server is physically separated, it's behind the firewall and only accessible by service code. Since most service frameworks have their own host environment (like IIS+.NET, TomCat or GlassFish, JBoss, etc.), it's usually separated out from Web Server. In order to enforce the same origin policy, the Web Server needs to configure as a proxy to the App Server.
In our web application, we use Apache as web server, it can be easily configured as a proxy to our GlassFish App Server base on URL pattern matching, this would ensure the service URL with the same host, protocol and port to pass the same origin test.
Web Server sits in front of the service tier in SOHA. It doesn't generate HTML, it doesn't run any application logic either. It's a pure HTTP server that only responses with requested static files for HTML, CSS, JavaScript and other digital assets, like images or sounds files. Classic server page technologies do not have a role in SOHA, free the developer's concern for scalability at the front tier, web server achieves maximum scalability by default, only the service tier needs to worry about how to scale.
In SOHA client, it encourages to use standard web technologies to build user interface. In order to better leverage HTML as a main vehicle to download the necessary application bits to browser, we intentionally selected XHTML 1.0 transitional as the DocType for each HTML files in our site. This selection essentially turned HTML as XML application; we further designed each XHTML document with the principle of XHTML template.
The XHTML Template file doesn't have concrete or user specific content as web server page usually does, it only has a "skeleton" or "placeholder" of XHTML markup about what to display based on user data at runtime. Compare to RIA by Flex/Silverlight, we lose the power of data binding in SOHA, but with the intention of using XHTML as Template, we can build our own rendering logic around those templates to gain productivity. This principle and design also requires "Clean HTML" from day one, no inline styles and no inline scripts, all it has is structural clean XHTML that represents the page template.
Since SOHA XHTML only represents structural data and templates, all visual aspects, including layout (size, float, margin, padding...), visuals (background color and images, fonts, borders, dividers...), plus certain level of interactivities (show/hide on hover, pseudo classes) are all handled by CSS. In MVC architecture, layout and visuals are considered as View, in SOHA, we can say View is controlled by CSS.
Certainly, JavaScript can create and apply/update CSS on the fly, but we purposely restrict the usage of JavaScript to create CSS rules. All CSS rules are created in external linked CSS file; JavaScript can conditionally update CSS on DOM element, but never create rules. This design also follows the rules of separations of concerns, CSS serves as View, and JavaScript serves as Controller.
With SOHA, JavaScript as Controller, it not only controls/updates views, but also controls/updates data modal, also controls entire application's visual themes and reusability of shared HTML regions. This is performed by dynamically loading HTML/CSS/JavaScript at runtime based on user data or UI logic, jQuery has excellent support on this topic.
Ajax, the power of JavaScript for manipulating DOM, plus new HTML 5 features, these technologies can create rich interactivities in web pages while maintain the reach-ness and accessibility as broadly as possible, any web enabled devices can access the site without much limitations. This is the most important reason for SOHA to promote web standards on the client side, no dependencies on browser plugin to render the web page. Especially for line of business web applications, the UI has text and data intense content, SOHA can enable an rich internet application without requiring browser plugin. Just as there are no server pages on web server, no browser plugin required on the client side is another distinguished feature in SOHA.
One note worth in depth discussion is that SOHA is not suitable for all web applications. Say your site has streaming video or audio as key content, then the web site architecture needs to be content oriented and plugin depended, SOHA would not help in this case. Same reason as HTML 5 is not ready for prime time yet, if the main feature of web application is to let the user creates graphics intense content, editing images for example, SOHA is not suitable either. In most LOB web applications, web page content is pretty much about information (rather than multimedia content), text and data heavy, SOHA makes a lot of senses in terms of separation of concerns, less dependencies, great rich-ness for interactivities and also has broad reach-ness for accessible by varies of platforms and devices, our consumer facing web application is one of this type of web applications.
OK, enough about high level discussion, let's look at some challenges and solutions when applying SOHA to a real world application.
Fundamentals and Detections
To enable Service Oriented HTML Applications, there are some common tasks that are particularly important in the client side because there is no help from server pages, these tasks include:
- Make sure user browser JavaScript is enabled, otherwise the application cannot run. Since JavaScript is the controller, we need to prompt the user to enable JavaScript on their browser
- Detect cookie settings in browser, if not enabled, prompt the user to enable it. Because SOHA navigation mechanism depends on it (more on this topic later)
- Detect local storage if browser doesn't support (like Safari 3 and Opera 10.1), since SOHA page to page data transfer (DOM is rebuilt when navigate to new page) relies on it
- The ability to load HTML pieces dynamically in order to share HTML regions (like page header and footers) from the client side
- Being able to load CSS and JavaScript files at runtime in order to change look and feel and behaviors dynamically
All items above, are solved by a jQuery plugin, detailed in my last article "Web App Common Tasks by jQuery", it has more discussions about implementation details, all source code and a demo HTML page are provided, you can take a closer look at how it works with jQuery. From SOHA perspective, there are some notes worth reiterating.
Client side master page and user control
"Client side master page" can be achieved by dynamically loading HTML from client. This technique expands Ajax's scope to cross pages usage, it enables consistent web page structure in a site, like page header, footer and navigation menus. Traditionally, server pages help the reusability in the forms of master page, user controls, etc. With this extension to Ajax, same structure can be achieved without server pages. The demo html page with my last article has a working example and all source code for dynamically loading page header and footer, and the site's navigation items are part of the header, it can be shared by multiple pages too.
loadDivHTML: function(divID, htmlURL, onSuccess)
{
$('#' + divID).load(htmlURL, onSuccess);
}
Here below is the demo page's script that uses loadDivHTML to dynamically load page header and footer:
$(function ()
{
var paramHF = qs['headerfooter'];
if (paramHF == 'both')
{
$.cbexp.loadDivHTML('pageHeader', 'html/_header.html', function() {
$.cbexp.loadDivHTML('pageFooter', 'html/_footer.html');
});
}
});
Page layout with CSS
Another aspect of master page is about page layout, like header is on top, footer on bottom with certain width and height, page content is always centered within browser window. Within page content, it could be 2 columns layout. One may be with side bar on the left and detail content on the right, one reversed layout can be side bar on the right while detail content locates at left. This type of common layout can be defined by external CSS files, without any change in HTML, by dynamically loading different layout CSS file, page layout can be changed on the fly.
Again, one working example with source code can be found in the demo html page. It achieves dynamic layout change without changing HTML, only applies different CSS files.
The following layout.base.css is shared by all the pages in the prototype site:
#pageContainer
{
margin: 0px auto 0px auto;
width: 960px;
}
#pageHeader
{
margin:0px auto 0px auto;
width:960px;
height:100px;
text-align:left;
}
#contentContainer
{
margin: 0px;
padding-top: 20px;
padding-bottom: 20px;
min-height: 500px;
}
#pageFooter
{
margin: 0px auto;
padding-bottom: 20px;
width: 960px;
position: relative;
}
In addition to layout.base.css, here below is the 2 columns layout by CSS:
@import url('cbexp.layout.base.css');
#main_grid-nav {
margin-left: 10px;
margin-right: 10px;
width:220px;
float: left;
position: relative;
}
#main_grid-content {
margin-left: 10px;
margin-right: 10px;
width:700px;
float:right;
position: relative;
}
When loading the following reversed 2 columns layout CSS file dynamically, it changes page layout without changing the page HTML:
@import url('cbexp.layout.base.css');
#main_grid-nav {
margin-left: 30px;
margin-right: 30px;
width: 200px;
float: right;
position: relative;
}
#main_grid-content {
margin-left: 30px;
margin-right: 0px;
width: 670px;
float: left;
position: relative;
}
Site themes in SOHA
As discussed above, CSS is the View in SOHA, and CSS is not only about layout, it also about visuals. With the same demo page, it shows the page's theme (color, font, images, layout, etc.) can be dynamically changed without changing HTML (this is another reason why HTML should not have inline style). It's implemented in a way that separating out layout from visuals by creating different CSS files (visual.base.css, layout.base.css), see the article for more information.
When multiple pages in a site controls layout and visual themes in a consistent way, and the dynamic CSS/HTML data is associated with user account, then "user preference" logic can be achieved in the client side based on service's data.
The following demo.blue.css is pure visual style sheet rules for a blue theme:
body
{
background: #3275ac url(../img/bk_texture_blue.png) repeat-x;
}
#pageHeader
{
background: transparent url(../img/bk_header_blue.png) no-repeat 0 top;
}
#navigation ul li a:hover {
background-color: #3275ac;
}
#contentContainer
{
background-color: #f4f4f4;
}
A pure visual style sheet (no layout) for a green theme might look like this:
body
{
background: #6f8881 url(../img/bk_texture_green.png) repeat-x;
}
#pageHeader
{
background: transparent url(../img/bk_header_green.png) no-repeat 0 top;
}
#navigation ul li a:hover {
background-color: #6f8881;
}
#contentContainer
{
background-color: #f4f4f4;
}
The following code demonstrates how to load layout style sheet and theme(visual) style sheet on demand:
loadPageCSS: function (cssUrl)
{
$('<link />').appendTo('head').attr({
rel: "stylesheet",
type: "text/css",
href: cssUrl
});
}
$(function ()
{
var qs = $.cbexp.getQueryString();
var paramCT = qs['theme'];
if (paramCT == "green")
loadGreenTheme();
else if (paramCT == "blue")
loadBlueTheme();
});
function loadGreenTheme()
{
$.cbexp.loadPageCSS('css/cbexp.demo.green.css');
$.cbexp.loadPageCSS('css/layout/cbexp.layout.2column.css');
}
function loadBlueTheme()
{
$.cbexp.loadPageCSS('css/cbexp.demo.blue.css');
$.cbexp.loadPageCSS('css/layout/cbexp.layout.2column_r.css');
}
Dynamic loading JavaScript
Another real world use case for dynamically loading HTML/CSS/JS is to offer context sensitive content, like different promotions for different users. Technically, each different promotion would be just different combinations of HTML/CSS/JS files, by leveraging the technique discussed above, we can offer them conditionally on a page. Coupled with appropriate tracking mechanism, this would be a lightweight while powerful data source for further business intelligence analysis.
The source code for "Web App Common Tasks by jQuery" has a method named loadPageScript, it leverages jQuery's support to load script file on demand. One use case for dynamically loading script would be changing behavior on a common "interface" (methods implemented by different scripts file, loaded conditionally); Another one is language specific "dictionary" files under language name folder, each script file has the same name for the dictionary object and same key for each entry, but different definition. Main page's controller JavaScript code can load the right dictionary file based on which locale the user is in.
All these features are not new in a web application, what's new is that they are implemented at the client side without any web server pages' help. As you can see from the source code, when building with jQuery, the actual code is very concise and easy to master.
In our prototype web app, we have a config.js file that defines APP_LOCALE, like: var APP_LOCALE = "en-us", "en-us" is the language folder name underneath JavaScripts directory. Within the language folder, it has language specific formatters (like currency and date formatter) and dictionaries (like the error code to error message dictionaty).These language specific scripts can be loaded dynamically based on the APP_LOCALE settings:
loadPageScript: function (jsUrl, onSuccess)
{
$.getScript(jsUrl, onSuccess);
}
$(function ()
{
$.cbexp.loadPageScript("javascripts/" + APP_LOCALE +
"/cbexp.formatter.js", function ()
{
$.cbexp.loadPageScript("javascripts/" + APP_LOCALE +
"/cbexp.error.dictionary.js");
});
});
Stay Tuned
This is the first part of discussion on Service Oriented HTML Application. Please note that client side processing for sharing HTML content among multiple pages (client side master page and user control), separation of layout and visuals in CSS, user preferences and themes, dynamic building page content and changing look and feel and even behaviors, etc., are all done without server page and no plugin required on client side. Without the help of server pages, there are going to be some challenges for managing sessions, user authentication and authorization, processing book marked URLs, sharing data when DOM rebuilds and interacting with RESTful services effectively. In the second part of SOHA, we'll going to discuss some details on the techniques about how to solve those challenges in our SOHA web app. Stay tuned.
[Updated 10/18/2010: Further discussion is posted: SOHA - Service Oriented HTML Application (Session and Security)]
