Introduction
In Steve Souders' book High Performance Web Sites [1], Steve outlines 14 steps for improving the perceived performance of web applications. Several of these are related to managing JavaScript more efficiently. Most of these mechanisms and a few others are discussed in the CodeProject article Speed Up Your Website - By Example [2]. These are excellent resources for understanding the reasons for performing some of the improvements implemented here, and therefore won't be repeated.
There are also some existing tools for combining JavaScript files and compression that are available. The need for this is evidenced by the inclusion of composite scripts in ASP.NET 3.5 SP1. This mechanism is described in another CodeProject article Performance Optimization of ASP.NET Applications on Client-side [3]. It also describes the HttpCombiner [4] from Microsoft Code that is defined as an HttpModule which can combine scripts when explicitly requested. Each of these tools solves some of the JavaScript issues described by Steve Souders, but require in-depth understanding of the issue in order to use the solution.
The implementation presented here is designed primarily from a usability standpoint. It does not require an understanding of all of the mechanisms, nor require developers to fundamentally change the way they code. Instead, the only thing to remember during development is to change the declaration of the script tag in ASPX and ASCX files from the traditional script tag to the new HP:Script tag (HP for High Performance). While senior team members should understand why these changes are desirable, it is easier to have all team members use a single mechanism. Finally, this mechanism is designed to be used in the easy/inelegant way that script tags are currently used, without the negative side-effects.
Using the Control
Once installed, integrating into Web Applications is straightforward. The control has equivalents for each of the common mechanisms for including scripts in ASP.NET. The mechanisms mimic the way that ASP.NET includes scripts.
External Scripts in Pages and User Controls (.aspx and .ascx)
Replace the typical script declaration, such as:
<script src="myfile.js" type="text/javascript"></script>
with the Script custom control:
<%@ Register TagPrefix="HP" Assembly="HighPerformanceScript"
Namespace="HighPerformanceScript" %>
<HP:Script src="myfile.js" runat="server" />
Inline Scripts in Pages and User Controls (.aspx and .ascx)
Replace the script tags around the script, such as:
<script type="text/javascript">
function doSomething() { ... }
</script>
with the Script custom control:
<%@ Register TagPrefix="HP" Assembly="HighPerformanceScript"
Namespace="HighPerformanceScript" %>
...
<HP:Script Location="External" runat="server">
function doSomething() { ... }
</HP:Script>
(Note: the Location="External" attribute removes the script from the page and feeds it externally so that it can be cached, this is the default. The attribute Location="Inline" will leave the script inline.)
External Scripts in Code-Behind and Custom Controls (.aspx.cs, ascx.cs, and .cs)
If you are using Page.ClientScript in User Controls to eliminate multiple identical scripts being included, such as:
Page.ClientScript.RegisterClientScriptInclude("myfile.js Specific Key", "~/myfile.js");
it can be replaced with either the inline version above, or with the code base equivalent:
HighPerformanceScript.Script.RegisterClientScriptInclude("~/myfile.js");
How it Works
The control deviates from the use of most custom controls in that it does not inject HTML at the point it is declared. Instead, it acts as a directive to ensure that the script is included at the most appropriate point. This allows it to automatically perform the desired optimizations. The eight core features of the control are:
Make Fewer HTTP Requests (Souders Rule 1)
It is a good practice to split code into logical files during development and maintenance. However, while this is good for development, it is bad for deployment to web clients. Each additional script requires additional latency during the request, and marginally increases bandwidth requirements.
To minimize the number of requests, the control keeps track of the JavaScript files that are requested during the page's lifecycle, and merges all of these script files into a single script. Then, only a single script tag is rendered in the page, with a URL unique to the combination of files. The combined script is maintained in the application cache for delivery, and is not actually written to the file system.
While this mechanism is fairly simple to implement, it does create a few problems, such as order dependencies, which are covered below.
Manage Dependencies
It is natural when breaking JavaScript into logical files that there will be dependencies between them. Unfortunately, JavaScript does not have a defined mechanism for managing these dependencies. In an HTML file, the scripts are simply merged in the order that they are included. In a single page, this is fairly obvious; however, using multiple user or custom controls in a page prevents the complete list of scripts from being seen easily until the page is rendered.
Since the control manages dependencies on the server side, it would make sense to have a syntax similar to C#. Borrowing the 'include' keyword and using a syntax that JavaScript can parse, the control allows the following syntaxes within the .js file:
include("prereq1.js");
The control will scan each file requested in the script tags, and will automatically add any dependencies. Further, this will determine the order in the combined file that the scripts are joined. The order of the script tags in the page does not have any effect on the order in the aggregated script. The include mechanism is the only way to override the default order, which is alphabetic.
(Note: Filenames in dependencies and anywhere else in the control can use absolute or relative URLs, as well as application root specific URLs such as "~/scripts/myscript.js".)
Add an Expires Header (Souders Rule 3)
In a production environment, script files change infrequently. However, the development practice of requesting the script on each page request is the norm. By applying a far future expires header to the script, the client will cache the script and not re-request after the first download. Since this control dynamically creates a modified script, the IIS expires header mechanism can't be used.
All scripts delivered by the control have an expires header that is set to 1 year in the future. This will prevent the client from re-requesting the file. However, it does need to request a new file when the JavaScript changes.
The control creates a URL specific to the content of the combined JavaScript files. This is done by appending a hash code of the content of the combined JavaScript file to the query string. Therefore, any changes to the JavaScript files (ignoring comments and whitespace) will cause the query string to change and signal the client browser to re-load the script.
Put Scripts at the Bottom (Souders Rule 6)
When scripts, either external or inline, are encountered by the client, the rendering of the page is stopped while the script is retrieved. This follows as clients assume the worst case scenario that the script will be changing the content of the page. By placing scripts at the bottom, the page is allowed to be rendered and displayed to the user before the client goes back to server for scripts.
The control does not render the script tag at the point that it is declared. Instead, it waits for the page, all users controls, and all custom controls to load, and then injects a single script tag at the end of the page, just before the closing form tag.
Make JavaScript External (part of Souders Rule 8)
Many pages and controls have their script included directly within the script tags. Since this portion of the page is less likely to change dynamically than the page, performance improvements can be made to create an external file, even if it is just a few hundred bytes.
When script is defined inline in the control, it is deleted from the output of the page, and is added to the combined output script. This way, there is no problem writing scripts inline. In cases where the code is desired to be inline, adding the Location="Inline" attribute will prevent this behavior and will cause the control to function like a normal script tag.
Minify JavaScript (Souders Rule 10)
Portions of JavaScript contain information that does not add value to the browser, such as whitespace and comments. This can easily account for 30-40% of the size of a JavaScript file. Removing this extra information will reduce the bandwidth required during the first request.
The control automatically runs the Minify Process [5] which removes unnecessary data from the JavaScript file.
Some additional gains can be made by compressing the remaining JavaScript. Two general mechanisms exist, but each has its disadvantages, preventing inclusion here. The first mechanism changes function names and attributes to shorter equivalents. This mechanism, however, does not always create compressed JavaScript that is consistent with the original. The second mechanism uses a JavaScript function that decompresses text and then uses the JavaScript eval function to include the actual JavaScript. This mechanism has the negative effect of requiring client processing time, which degrades performance.
Remove Duplicate Scripts (Souders Rule 12)
Many scripts will naturally be needed by more than a single control, e.g., an AJAX enabling script. When this is done in several User Controls, however, it can cause conflicts within the JavaScript. At best, it degrades performance by requesting the same script multiple times. ASP.NET provides a remedy using the ClientScript property of the page:
Page.ClientScript.RegisterClientScriptInclude("Ajax.js Specific Key", "Ajax.js");
Likewise, the HP:Script control automatically reduces multiple script includes to a single include. This effectively duplicates the behavior of RegisterClientScriptInclude while allowing the script to be included in the .aspx or .ascx page.
Auto-Switch for localhost Debugging
Trying to debug an application that has scripts that have been shuffled and obfuscated is not for the faint of heart. However, the control cannot completely disable the above features without changing the way the scripts behave. So, a middle ground is sought that will enable debugging.
When a web page is loaded from localhost (or 127.0.0.1), it will not join files together into a single file, and it will not minify the JavaScript. This will ensure that JavaScript consoles on client browsers will reference the correct filename and the correct line within that file. All other processing is performed. Accessing the local machine by machine name will not be treated as a localhost request (e.g., http://localhost/MyApp and http://mymachinename/MyApp are not treated the same).
Implementation
The system consists of just a few files that achieve these results: a custom control, a web page, and a support file for Minifying [5]. There are a few possible implementations that were considered before the use of a custom control was chosen.
One possible implementation would use an HttpModule to intercept the page after it has been rendered and before it has been sent to the client. This module would then parse the HTML and extract the script tags, combine them, and emit a single script at the end. This requires a slightly more difficult implementation, as parsing would have to be done just right. Further, there would be no explicit control over when to disable optimizations. Finally, it requires a setting in the Web.config file to attach the module. Normally, this isn't a problem, but an accidental removal of the module would likely not change the behavior of the system, it would just destroy performance.
Another possible implementation is a more manual process that allows complete control over the optimizations. This is the path taken by HttpCombiner [4]. This process, however, requires that users are always vigilant about the use of HttpCombiner. This, and the fact that a suitable tool already exists, prevented this model.
So, the final model was to create a custom control to manage the scripts. This control creates a link back to another page which serves the modified JavaScript from memory instead of from a file system. Installation, then, only requires that the page and the control be copied into a web application. The control could easily be moved into a control library if desired, but the attached sample uses a single project.
Script Custom Control (Script.cs)
Whenever a script is loaded during the life of a request, it is registered into a list of all scripts on the page. The first control that enters PreRender will iterate through the list and compile all the information required for the aggregated script. This aggregated script is placed in an application level cache. This cache reduces repeat calculation of aggregated scripts, and when running on a single machine, will act as a cache for the script serving page. It then uses RegisterStartupScript to emit the script include tag at the end of the form. Just before leaving PreRender, it clears the list so that other controls in PreRender will not repeat this process. Finally, during the Render phase of each control, nothing is output.
Script Resource Server Page (ScriptResource.aspx.cs)
The aggregated script that is created by the control was previously stored in an application cache, with a key that corresponds to the combined names of the scripts. The script resource server page will deliver this cached version, if it is available. When running in a web farm, this page may not be served from the same application. In this case, the information from the query string of the page is sufficient to rebuild the aggregated page. The page will, if necessary, rebuild and re-cache this resource. Finally, it is sent back to the client with the appropriate MIME types. Using a .aspx extension simplifies integration with the web application by not requiring a custom extension handler.
JavaScriptMinifier.cs
Taken from Crockford [5] for minification, minor modification to allow for in-memory minification.
Sample
The attached solution contains the necessary files for the control, along with a host of test web pages. These tests will just load in a browser, and indicate in the browser whether or not the tests pass. The screenshot above is from Google Chrome, showing the lifecycle of a page that includes four separate scripts. The orange bar is the JavaScript resource that was compiled from all four of the resources. Without aggregation, this would be four separate requests of the server. Prior to Chrome, IE 8, and Firefox 3, these four requests would not even be executed in parallel, but would only be requested two at a time.
To test the sample, download the zip and open the Solution file. Running with the ASP.NET Development Server will prevent some tests from succeeding as the resultant page will be in localhost/debugging mode. Start IIS, and create a Virtual Directory to the project directory, then access by machine name and virtual directory to ensure all tests succeed.
References
- S. Souders, High Performance Web Sites, O'Reilly Media, 2007
- A. Baig, "Speed Up Your Website - By Example", The Code Project, 2008-05-27 (2009-02-02)
- K. Shehzad, "Performance Optimization of ASP.NET Applications on Client-side", The Code Project, 2008-12-18 (2009-02-02)
- O. Zabir, "HttpCombiner - combine, compress, and cache multiple CSS, JavaScript, or URL", MSDN Code Gallery, 2008-08-29 (2009-02-02)
- D. Crockford, "The JavaScript Minifier", Douglas Crockford's World Wide Web, 2003-12-04 (2009-02-02)
History
- 2009-05-13
- Fixed cache problem on multiple nested file includes.
- Made scripts nested in page appear after all scripts included.
- 2009-02-05
