Introduction
This article and its accompaning sample demonstrate several useful techniques for Web software development both in server and client sides. The sample presents a simple scrollable table. When user while scrolling its content, comes close to either end or begin of visible part of the table, appropriate adjacent portion of data is downloaded from server and replaces part of the table which has become invisible. The sample works as follows. Browser calls Web server and loads index.html file along with related JavaScript (JS) and CSS files that start single-page Web application. This application intensively communicates with server via either WebSocket (preferred option) or HTTP POST (if WebSocket is not supported by the browser) to obtain data. In the client application JS Promise construct is used in WebSocket creation function. Closure and object augmentation techniques are employed to create chain of function calls to manupulate DOM objects. Two data providers organize data supply to client. Both of them extend the same JS style "base class". The data providers are used by scrollable table objects with a dependency injection mechanism. There are two implementations of data providers: with JS and TypeScript (TS). In the latter case TS files are transpiled to JS, this "Web assembly language".
Two servers based on different technologies with the same functionality are presented. One of them uses ASP.NET Core platform recently released by Microsoft, and another one is developed with node.js. Both servers can run in different operating systems (OS) and have been tested under Windows 10 and Linux Ubuntu 16.04.
The above techniques will be discussed in more detail below.
In this work I use several fragments of code by other authors. I put appropriate references in the article and/or in code. Please forgive me if I inintentionaly missed some references.
Prerequisites
As it was previously stated, the sample in this article illustrates the ability of the servers to run under different OS. But if you would prefer to confine yourself with just Windows then you can skip the next paragraph.
Since the main platform we going to use is Windows 10, in order to run our servers in Linux environment we need to install Linux virtual machine. Oracle VirtualBox was chosen for that matter (only the server environment is actually required). VirtualBox may be downloaded from its Web site and guidance for its installation may be found e. g. here. After the installation of the virtual machine we need to install a tool for file exchange between Windows and Linux. WinSCP does the job (the acronym stands for Windows Secure Copy). The application may be installed from its site. Although VirtualBox display window can be used for inserting commands, PuTTY terminal emulator is very useful for that purpose. It may be downloaded from here.
Installations of node.js and .NET Core. should be carried out in both Windows and Linux OS. node.js for Windows may be installed from here and for Linux from here. Installation of .NET Core (including ASP.NET Core) may be performed from dedicated Microsoft site. It is possible to run the sample in this article without any development tools using only command line actions with just runtime for .NET Core. But to debug the sample and play with its code Microsoft Visual Studio (VS) 2015 Update 3 and .NET Core VS tools will be very handy. To facilitate debugging of node.js code VS tools for node.js may be installed from here.
If you were persistent enough to successfully install all the above stuff (in my case it became possible with kind help of a great expert in multiplatform and mobile software and a good friend of mine Michael Molotsky) you can proceed with our sample. Let's start with the client.
Client
Client is a single-page Web application. It is placed in folder .\webclient (here and below we use either ".\" or "<Root Dir>" to denote our solution root folder where we unzip the sample) and activated through file index.html by a browser. Client is written in JS (when referring to a client side JS in this article we mean JavaScript of ES5 standard). File .\webclient\js\index.js contains main logic of the client application. File .\webclient\js\scrollableTable.js implements scrollabe table. Files related to data providers are placed in the .\webclient\js\dataProviders folder. The sample demonstrates some useful JS techniques, like closure, "class" inheriting simulation, promise, ajax communication, etc. (all these features are implemented in many well-known JS libraries, but to me it was interesting to develop their minimalistic implementation myself). Files implementing the features are located in .\webclient\js\helpers folder.
File closure.js contains closures handy for DOM elements manipulation. The closures allow usage of chains based on augmentation of Object (file objectAugmentation.js). File communication.js is responsible for communication with server. It supports communication via WebSocket (when available), and HTTP (GET and POST methods). Given implementation of WebSocket connection illustrates two important paradigms: singleton (file singleton.js) to hold created WebSocket object, and promise (file promiseC.js). The latter simplifies the syntax of asynchronous callbacks by enveloping it in a synchronous-like wrapper. Most modern browsers support promises out of the box, but not all of them. Due to the lack of promises in some browsers (and because the topic is interesting) a custom simplified implementation of a promise is developed in this work, and that is PromiseC in file promiseC.js. It will be discussed in details below in appropriate chapter.
Scrollable table is separated from its data providers and uses them via dependency injection. There are two data providers in the client side, namely, local DataProviderLocal and provider delivering data from server DataProviderFromServer. Both providers have common "base class" DataProvider. The providers are implemented in two flavors: (a) using JS with "inheritence" mechanism by Douglas Crockford (file inheritance.js), and (b) using TS files (folder .\webclient\ts) following by their transpiling to correspondent JS files. The transpiling is performed with TS compiler tsc.exe (called with command file tsc.cmd) and TS project file tsconfig.json.
In the sample all interactions with server are initiated by client (no asynchronous notifications). Working scenario is following:
- if constant
SERVER in file utilities.js is set to true then the client tries to connect server via WebSocket. In case SERVER is set to false, client does not connect to server at all and shows data generated by local data provider in file dataProviderLocal.js (this is testing mode); - if WebSocket is not supported by browser then attempt to communicate via HTTP is carried out;
- the first request to server is "about" request to obtain information about server's origin (whether ASP.NET Core or node.js). Client displays response information and communication method (either WebSocket or HTTP POST);
- then client requests the first chunk of data and places response to the scrollable table;
- as user scrolls the table in the browser and comes close to end or begin of currently displayed chunk, client requests server for the next chunk of data in the direction of user scrolling;
- newly received chunk is displayed to replace one that went out of the table visible area.
Data exchange between client and server is performed in JSON format allowing usage of standard parsing procedures in both sides. The file index.html defines scrollable table as following:
<div>
<div>
<table class="tbl">
<tr>
<th>Column A</th>
<th>Column B</th>
<th>Column C</th>
<th>Column D</th>
</tr>
</table>
</div>
<div class="scrollBox">
<table class="tbl"></table>
</div>
</div>
In the above fragment the first <table> element provides scrollable table's header with its <th> tags. The second table placed inside <div class="scrollBox"> is actually scrollable and will be filled dynamically. Styles for classes tbl and scrollBox (as well as styles for all other elements) are defined in the .\webclient\\css\styles.css file.
Note: -webkit vendor specific prefix is used in scroller style definition to improve its appearance. This prefix is "understood" by Google Crome and Apple Safari. FireFox and Microsoft browsers do not recognise -webkit and depict scroller is a simple standard way.
Function fillTable() (file .\webclient\js\index.js) is called when document is ready:
var fillTable = function() {
var divScrollBox = allTags('div').byAttribute('class', 'scrollBox').result()[0];
var tables = allTags('table').result();
var tblHeader = tables[0];
var tblData = tables[1];
var maxData = 200;
var tableRowLimit = 50;
var vertScrollerLowerPosInPerCent = 0.9;
var chunkLen = tableRowLimit / 5;
var dataProvider = DataFactory.prototype.dataProviderFactory(
SERVER ? 'fromServer' : 'local',
maxData,
urlProvider,
function (about) {
elementById('lblDataProviderAbout').setText(about);
},
SERVER ? theSingleton.getInstance().getWebSocket()
: undefined);
scrollableTable(tblData, tblHeader, tableRowLimit, vertScrollerLowerPosInPerCent,
chunkLen, divScrollBox, dataProvider,
function(relativePosHorz, relativePosVert) {
var scrollHorzText = '';
if (!isNaN(relativePosHorz)) {
scrollHorzText = 'Scrolled from the left: ' +
relativePosHorz.toFixed(2) *100 + '%, ';
}
elementById('lblScrolling')
.setText(scrollHorzText +
'Scrolled from the top: ' +
relativePosVert.toFixed(2) * 100 + '%')
.setStyle('color: #73AD21');
});
}
The function obtains DOM elements used by scrollable table, sets up requied configuration parameters, creates appropriate data provider and finally invokes function scrollableTable() which consumes the above parameters as its arguments and takes in addition a callback for scroller position change event.
Since browsers provide mostly single-threaded execution model, in order to make UI responsive Web client usually performs lengthy operations (like e. g. communication related operations) asynchronously. This implies usage of callbacks. Code with callbacks in ordinary JS sometimes is not easy to read. It would be much nicer to present asynchronous functions with pseudo-synchronous notation with clear order of callbacks invokation. The construct called "promise" does the job.
Promise
Promise is a useful object to deal with asynchronous function calls. According to the Mozilla Development Network definition "Promise represents a single asynchronous operation that hasn't completed yet, but is expected in the future. A Promise represents a proxy for a value not necessarily known when the promise is created. It allows you to associate handlers to an asynchronous action's eventual success value or failure reason. This lets asynchronous methods return values like synchronous methods: instead of the final value, the asynchronous method returns a promise of having a value at some point in the future."
I'd admit that the above definition (as well as some others I found) does not look to me very clear. So I provide description of Promise as I understand it. The following pseudo-code snippet illustrates functionality of Promise:
When object of Promise is created, Promise() function takes as an argument a function called executer. Arguments of executer are functions resolve() and reject() of Promise object itself. A Promise object has function then() with two arguments: functions onResolve() and onReject(). Executer is invoked by Promise "constructor". Invokation of executer causes call of either the resolve() or reject() function. Usually this is an asynchronous process. For example, in our code we use promise to initialize the WebSocket connection. So if WebSocket is supported then our executer function creates WebSocket object, sets handler for its onopen event and returns. After some time WebSocket's onopen handler is called. The handler in its turn calls the resolve() function of Promise passing some argument (WebSocket object in our case) on to it. If WebSocket is not supported then function reject() with error object as its argument is called.
Call of resolve() or reject() function starts execution of chain of callback arguments of functions then(). Function resolve() causes execution of onResolve callback, and function reject() leads to execution of onReject callback. The callbacks recieve the arguments supplied to resolve() and reject() functions respectively. Each next callback in the chain takes result of the previous one as argument. If the previous callback returns correctly then in current pair onResolve callback is invoked, and in case of exception onReject callback is activated taking error object as its argument.
Promises become popular relatively recently, and old browsers (e. g. Internet Explorer 11 and Safari for Windows 5.1.7) do not support Promise out of the box. So I provide a custom simplified implementation of Promise as PromiseC() function in file .\webclient\js\helpers\promiseC.js. The function is shown below:
function PromiseC(executer) {
var isResolved;
var arg;
var callbackPairs = [];
var resolve = function (val) {
isResolved = true;
arg = val;
invokeCallbacks();
}
var reject = function (err) {
isResolved = false;
arg = err;
invokeCallbacks();
}
var invokeCallbacks = function () {
if (callbackPairs.length > 0 && !(isResolved === undefined)) {
callbackPairs.reverse();
while (callbackPairs.length > 0) {
var aCallbackPair = callbackPairs.pop();
try {
arg = isResolved ? aCallbackPair.onResolveCallback(arg)
: aCallbackPair.onRejectCallback(arg);
isResolved = true;
}
catch (err) {
arg = err;
isResolved = false;
}
}
}
}
this.then = function (onResolveCallback, onRejectCallback) {
callbackPairs.push({
onResolveCallback: onResolveCallback,
onRejectCallback: onRejectCallback
});
invokeCallbacks();
return this;
}
executer(resolve, reject);
}
This is simplified implementation indeed, since standard Promise has more methods apart from then(), but it is enough for our sample. In this implementation function invokeCallbacks() is called by resolve() / reject() and also by then(). Actual callback execution takes place after call of executer and adding onResolve / onReject callbacks (either some or all of them) to callbackPairs array.
To experiment with promise, two then() functions are added to our promise in the file communication.js (they marked with //TEST comment). Web application in case when WebSocket is supported logs the following output to browser developer tool console:
the 1st "then()" - resolved, arg: [object WebSocket]
the 2nd "then()" - rejected, arg: Exception in the 1st "resolve()'!
If WebSocket is not supported (or we'd like to simulate this situation by uncommenting throw() function call inside function createWebSocket()) then the following log is output to browser console:
the 1st "then()" - rejected, arg: TEST to disable WebSocket
the 2nd "then()" - resolved, arg: result of 1st "reject()"
If your browser has a native support to promise then you can use standard implementation of promise instead of mine. For this in th file communication.js in line var promise = new PromiseC(... simply replace PromiseC with Promise (remove letter "C"). The above logs will also be obtained with the standard promise.
Communication
As previously stated, communication between client and server is carried out using either WebSocket or HTTP. The former way is preferred because it allows asynchronous (push) server-initiated messages out of the box. Although we have not used push messages in our sample so far, in the real world this is a very common practice for Web applications. If WebSocket is not supported by the browser (nowadays this is a rare case) then push message effect may be achieved with e. g. HTTP long polling technique. Unfortunately long polling not only requires additional code but also compromises scaling of the server due to threads holding.
Note. There is a commonly used SignalR communication framework optimized for high productivity and scalability that (among other things) automatically selects the best available transport (whether it is WebSocket, HTTP long poll or other). But currently SignalR is available mostly in the Windows environment (in Linux it is avalable with Mono). According to Microsoft, the plans for the next release of ASP.NET Core should include SignalR. So hopefully SignalR availability in .NET Core is a matter of weeks away.
Servers
Two kinds of servers are developed in this work. One is based on recently released ASP.NET Core and another is based on node.js. Both servers demonstrate interoperability across the most wide spread OS such as Windows and Linux. The servers' implementation is rather minimalistic without too many auxiliary details.
ASP.NET Core Server
At the end of June 2016 Microsoft had finally released the first version of .NET Core platform interoperable across all major OS. Its ASP.NET Core subsystem is used in this work to develop server. The server is essentially a normal ASP.NET Web API one. So the learning process for an ASP.NET developer is short and painless.
The server was created by VS2015 equipped with .NET Core development tools. In the FILE -> New -> Project... dialog window chose Templates -> Visual C# -> .NET Core and take ASP.NET Core Web Application (.NET Core) template in the right-hand side panel. Then in the New ASP.NET Core Web Application (.NET Core) dialog window chose Web API template. That's basically it: ASP.NET Core server template has been created. DLLs required for the server should be created in much the same way with Class Library (.NET Core) project template.
Our server is located in the folder .\AspNetCoreServer. Its main part is RESTful Web API ServerApi project. ServerApi provides overall webservice infrastructure and standard mechanism to communicate with client via HTTP (JsonController class). It also starts WebSocket communication in its Startup.Configure() method by calling the RequestProcessor.Instance.ProcessRequest() method of RequestProcessor singleton class of RequestProcessorLib.dll. InterfaceClientContextLib.dll contains IClientContext interface implemented by ClientContext type (ClientContextLib.dll). Such a design allows developer to have in future several client contexts implementing IClientContext interface to provide different models and different data processing algorithms for different clients. Class ClientContextFactory of ClientContextFactoryLib.dll is responsible for creation of the client context objects.
This server and client together adopt Model-View-Controller (MVC) model. Internal class ClientContext.TableRow of ClientContextLib.dll acts as a Model, JsonController class constitutes the only Controller in current server implementation, whereas file index.html with all its JS / CSS stuff stands for the View.
The server supports communication with client via WebSocket and HTTP. The latter is organized in a conventional for ASP.NET Web API applications by way of using appropriate controller JsonController class. Currently it contains only one method PostJsonString() which is responsible for receiving HTTP POST requests and passing them on to one of the two overloaded RequestProcessor.Instance.ProcessRequest() methods of a singleton class for actual processing. The second method is used by WebSocket communication channel. Both methods call the same ProcessText() method of ClientContext class in order to generate response string.
IP adresses and ports on which the server listens to requests are configurable. Section "ServerUrls" in the configuration file appsettings.json contains a list of those IP addresses, and static method Urls() of class Program reads this configuration.
Cross-Origin Resource Sharing (CORS)
Our server supports Cross-Origin Resource Sharing (CORS). According to Wiki CORS "is a mechanism that allows restricted resources (e. g. fonts) on a web page to be requested from another domain outside the domain from which the resource originated. ...embedded web fonts and AJAX (XMLHttpRequest) requests have traditionally been limited to accessing the same domain as the parent web page (as per the same-origin security policy). "Cross-domain" AJAX requests are forbidden by default because of their ability to perform advanced requests (POST, PUT, DELETE and other types of HTTP, along with specifying custom HTTP headers) that introduce many cross-site scripting security issues." In order to lift this restriction appropriate policy is created and then added to IApplicationBuilder app object in Startup class:
public void ConfigureServices(IServiceCollection services)
{
...
services.AddCors(o => o.AddPolicy("CurrentCORSPolicy", builder =>
{
builder.AllowAnyOrigin()
.AllowAnyMethod()
.AllowAnyHeader();
}));
}
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
app.UseCors("CurrentCORSPolicy");
...
}
node.js Server
node.js based server was created using VS2015 equipped with development tools for node.js. In VS2015 the following actions were taken: FILE -> New -> Project..., in the New Project dialog window in the left-hand panel Templates -> Other Languages -> JavaScript. Then in the righ-hand panel Blank Node.js Web Application template was chosen. The template was placed in folder .\nodeScrollableTableServer with the same solution name. Upon template creation three JS files nodeScrollableTableServer.js, messageProcessor.js and utilities.js were put to folder .\nodeScrollableTableServer\nodeScrollableTableServer and added to the project with rigt click menu -> Add -> Existing Item..., and file server.js created by wizard, was deleted. File nodeScrollableTableServer.js was Set as Node.js Startup File using right click menu.
WebSocket is installed locally in folder .\nodeScrollableTableServer\nodeScrollableTableServer with npm (Node Package Manager), the default package manager of node.js:
npm install websocket
(the command is put to command file installWebSocket.cmd for convenience and may be run from there).
The file nodeScrollableTableServer.js contains functions responsible for communication to the client via WebSocket and HTTP, file messageProcessor.js provides parsing client messages and generetion of responses, and file utilities.js contains several handy commonly used utilities. The server's files use lambdas supported by modern versions of node.js (version 4.4.3 was used for testing).
CORS
The CORS problem described in the previous chapter is solved also in this server. For this the following object is included to response header (file nodeScrollableTableServer.js):
{
'access-control-allow-origin': '*',
'access-control-allow-methods': 'GET',
'access-control-allow-headers': 'content-type, accept'
}
Running Sample
Please unzip the sample zip file to the folder below referred to as <Root Dir> and read ReadMe.txt file. To start Web application I used IIS Express Web server. To configure it please add site ScrollableTable from ReadMe.txt to IIS Express standard configuration file ..\Documents\IISExpress\config\applicationhost.config replacing <Root Dir> with actual path. Start ScrollableTable site under IIS Express by running .\startIIS.cmd file as Administrator.
Servers in Windows
The next step is to run server. ASP.NET Core server is started with file .\aspNetCoreServer.cmd activating command
dotnet run
from its main folder .\AspNetCoreServer\src\ServerApi.
Alternatively node.js server may be started by activating file .\nodeScrollableTableServer.cmd containing command
node nodeScrollableTableServer
The server will listen on port 15011 on all available IP addresses.
The two servers cannot run simultaneously since they listen on the same port.
Then you can start your browser and insert IP address http://localhost:15111 to it. Application will start.
Servers in Linux
To be installed in the Linux environment our ASP.NET Core server first should be "published". Publish operation prepares a complete set of resources required to run the server. Before publishing section "ServerUrls" in configuration file appsettings.json should be changed to insert proper IP addresses and ports for the server. In order to publish the server the following command should be executed from its main folder .\AspNetCoreServer\src\ServerApi (for convenience the command is put to command file .\publish_aspNetCoreServer.cmd):
dotnet publish -o publish -c Release
Output of the above command is folder .\AspNetCoreServer\src\ServerApi\publish. This folder contains all resources required to run our ASP.NET Core server in Linux environment.
The next step is coping content of newly created publish folder to Linux folder of .\AspNetCoreServer with WinSCP application. Then from inside this Linux folder run command
dotnet ServerApi.dll
starting our ASP.NET Core server in Linux.
Our node.js server should be first copied to Linux folder .\nodeScrollableTableServer with WinSCP (only JS files and node_modules folder have to be copied). Then the server may be run with the same as in Windows node nodeScrollableTableServer command.
Client
To enble proper communication, the server IP address in client Web application should be changed to match the server IP address: in file .\index.js in anonymous function that returns urlProvider object, value of variable host 'localhost' should be replaced with actual server IP address.
In order to test Web application with data providers in TS please go to folder .\webclient\ts and run file tsc.cmd activating TS transpiler tsc.exe working with tsconfig.json configuration file. Result JS file will be placed to folder <Root Dir>\webclient\js\dataProviders replacing previous files there. Then reload (refresh) page in the browser.
Further Studies
This article shows collection of client and server techniques useful for a "full stack" Web developer. But some important topics related to scope of the article are left out. This is first of all interaction with databases. Indeed, for the sake of simplicity, servers generate data to be displayed in the scrollable table rather than take them from some data repository as it happens in real world. So interaction with databases is an obvious direction for further study (several good works on using databases by ASP.NET Core were published in CodeProject recently). As it was stated above push messages are vital for some Web applications. So it will be nice to demonstrate their implementation with both WebSocket and HTTP long polling.
Thanks
I'd like to express my thanks to a friend and colleague of mine Michael Molotsky for his tremendous help in VirtualBox and .NET Core for Linux installation and useful discussion in the scope of this article. I am grateful to Mark Hotykov for discussion on JS promise. My deep gratitude goes to Liron Levi for his support of my Web studies and valuable suggestions on the topic.
