Know JavaScript... Use node.js server-side

node.js Introduction

node.js firstly is a (unexpectedly) spawned server side execution of Google's V8 JavaScript engine. The V8 engine is said to be also used by the Chrome browser. node.js may be compared to something like VB runtime enabling VBScript.

So this enables JavaScript to be used on server-side.

All browsers have a JavaScript engine which nobody seems to have thought of making into a server-side engine which will allow JavaScript enthusiasts code on the server-side. Now node.js is popular, and Microsoft even supports node.js scripting on their Azure cloud systems (could help tap JavaScript coders to use Azure).

This article aims to quickly dive into some examples of using node.js.

Hello World

Type the below line in a text file, preferably named helloworld.js.

console.log('Hello world!'); 

and run it on the command line with node engine:

node  c:\work\helloworld.js

To do the above... the pre-requisite is that you have node.js installed. (It is similar to installing a desktop application.)

What Can We Use node.js For

In essence of JavaScript being very good for AJAX, the same seems to be the most popular use of node.js as well... making requests or serving requests. Though other uses are possible, most of the samples you find will be to make or respond to server requests.

We will create some examples of them.

A Web Request Example

The below code saved in a file like test.js and executed with node, will do a HTTP request to fetch a page and just print the results... the HTML.

var http = require('http');

var options = {
  host: 'www.google.co.in',
  port: 80,
  path: '',
  method: 'GET'
};

var req = http.request(options, function(res) {
  console.log('STATUS: ' + res.statusCode);
  console.log('HEADERS: ' + JSON.stringify(res.headers));
  res.setEncoding('utf8');
  res.on('data', function (chunk) {
    console.log('BODY: ' + chunk);
  });
});

req.on('error', function(e) {
  console.log('problem with request: ' + e.message);
});

req.write('data\n'); //this line prints first, while the request and the content 
                     //printing happens a bit slow as its an internet call.

req.end();  

The style of script is just the same as JavaScript coding with variables, objects, and namespaces. Yes... that, and experience with jQuery like library usage, is a pre-requisite for readers of this article.

node.js has inbuilt interfaces exposing support for HTTP functions, JSON, etc., which we have used above. Some other details on the above HTTP request example:

http.request - makes the request and its second parameter is a callback event handler in JavaScript callback handlers wiring style.

JSON.stringify - yes, JSON is inbuilt into the node engine.

Events handlers are wired, as below:

res.on('data', callback fn); 

where the response object fires the data event every time it receives data after the http request is raised.

node.js External Libraries & npm

Other than inbuilt interfaces, node.js can also use libraries like the noderss, htmlparser, etc., These libraries can be installed using npm.

npm - is node's package manager. (I guess it might be using the open-source rpm -Redhat Linux package manager internally and so the name, but that's a guess. Somebody let me know if I am right.)

Installed libraries will be in a folder 'node_modules' in your node installation directory or the user directory. Under 'documents and settings' on a windows machine, for example c:\Documents and Settings\username\Application Data\npm\node_modules.

There are numerous node capable libraries added on github by the day... so node.js's capabilities are being improved by the day or if you may, by the hour.. so you may not necessarily wait for another version of node.js to be released to write better code. Another reason for saying that is, since it is JavaScript based, the scripting language being a public standard.. the engine itself might not need/have improvements unless JavaScript versions upgrade.

(Another guess... I think libraries for node are written in JavaScript.. guessing from what I have seen inside node_modules folder. Am I right? So every good JavaScript developer should be able to create libraries for node? Let me know.)

Using xml2js Library and Parsing an RSS Feed

This next example, uses node's inbuilt support of HTTP and file system functions, and an external library xml2js, which you have to install with npm.

Some Pre-requisites.. install npm module

Type the below command on your command prompt to install xml2js:

npm install xml2js

or:

npm install --global xml2js

Troubleshooting 'xml2js not found' errors.

I personally didn't have success using xml2js unless I copied the xml2js package into the root folder of my .js project folder.

• http://stackoverflow.com/questions/5047542/node-js-cannot-find-module-xml2js

The Code for Parsing an RSS Feed

var http = require('http'),
  fs = require('fs'),
  xml2js = require('xml2js');

////////// code to get xml from url
var options = {
  host: 'www.codeproject.com',
  port: 80,
  path: '/WebServices/ArticleRSS.aspx',
  method: 'GET'
};
var req = http.request(options, function(res) {
  console.log('STATUS: ' + res.statusCode);
  console.log('HEADERS: ' + JSON.stringify(res.headers));
  res.setEncoding('utf8');
  
  var fullResponse = "";
  
  res.on('data', function (chunk) {
    //console.log('BODY: ' + chunk);
    fullResponse = fullResponse+chunk;
  });
  
  res.on('end', function(){
    parser.parseString(chunk);
    console.log(item.title + '\n');
  });
});
req.on('error', function(e) {
  console.log('problem with request: ' + e.message);
});

/////////xml load example from xml from file
fs.readFile(__dirname + '/ArticleRSS.aspx.xml', function(err, data) {
    parser.parseString(data);
    
    jsonObj.channel.item.forEach(function(item){
      console.log(item.title + '\n');
    });                            
    
    console.log('Done');               
    
    process.exit(0);
});

//code to parse xml
var jsonstring = "";
var jsonObj;
var parser = new xml2js.Parser();
parser.addListener('end', function(result) {
    jsonstring = JSON.stringify(result);
    jsonObj = JSON.parse(jsonstring);
    
    console.log('Done');
});

About the Above Code

xml2js usage is explained here.

The above code gets the content of the CodeProject article RSS feed, calls a handler to parse the XML, to extract or print the title of all nodes in the RSS.

• shows an example use of forEach
• shows an example of forcefully stopping node program with process.exit(0)
• shows examples for loading an RSS feed from a filesystem file and from a web URL.

Can I Use jQuery Core Inside node.js as an External Library?

Technically yes. You may try installing the npm and start using it.

npm install jquery

I couldn't, because some dependencies were failing for me... still troubleshooting.

Making a Fully JavaScript Written Web Service

This is a service, which accepts a search string, searches codeproject.com and returns the results as an RSS feed.

It is possible to write entire server side applications, service layers, etc., with node.js. Some SOAP + XML external libraries available on npm repository allow you that.

To keep it easy for now (since doing proper WSDL like service for this example's purpose seemed needing more code and testing), I built a quick REST service all with a server... all in JavaScript. Remember, no need for web server, socket programming, etc.

In the below example, we are creating and hosting a web server handling REST service requests.

Uses Four External Libraries

• qs - which allows easily getting and parsing the querystring of a request to our server
• htmlparser - which allows parsing the HTML returned
• soupselect - which allows looping through a set of HTML elements which are the results in CodeProject's HTML output of a search result page
• rss - which allows generating an RSS feed

and of course, the inbuilt HTTP methods. I have written all the code with callback handlers... so the methods don't return anything, so no line of code has to wait to use the returned values. Though this style of coding makes the readability of the code a bit complex, it allows keeping what node.js is good at - non-blocking operations. Yes, JavaScript developers will be implementing parallelism, without writing any sockets or having to do anything special for implementing async execution.

The Code

var http = require('http'),
rss = require('rss'),
qs=require('querystring'),
htmlparser = require("htmlparser"),
htmlselect = require('soupselect');

/////////// Create and start the server to handle requests
http.createServer(function (request, response) {
  if (request.url.indexOf('searchcp')==-1)
  {  
    response.writeHead(200, {'Content-Type': 'text/html'});
    response.end('Invalid call\n');
  }
  else
  {  
    //tell the client that return value is of rss type
    response.writeHead(200, {'Content-Type': 'application/rss+xml'});
    
    //extract the searchquery from the querystring.. this is a REST service
    var str_querystring = request.url.substring(request.url.indexOf('searchcp'));
    var searchString = qs.parse(str_querystring).searchcp;
    console.log('\nsearch string:'+qs.parse(str_querystring).searchcp);
    
    //method call to process the search and print rss
    processquery(searchString, response);
  }
}).listen(8080);
console.log('Server running at http://localhost:8080/');

function processquery(searchString, responseObj)
{ 
  
// code to get xml
var options = {
  host: 'www.codeproject.com',
  port: 80,
  path: '/search.aspx?q='+encodeURIComponent(searchString)+'&sbo=kw',
  method: 'POST'
};
var handler = new htmlparser.DefaultHandler(function (error, dom) {
    if (error)
    {}
    else
    {    
      var cpsearchresults=[];
      //get nodes in HTML that we are interested in
      //and loop through each node.. each being one search result item
      //of codeproject search result HTML.. 
      //**depends on codeproject HTML not having changed 
      //**since this time when I tested.
          
      htmlselect.select(dom, "div.result").forEach(function (ele){
        tmpTitle = htmlselect.select(ele, "span.title a");
        if (tmpTitle != undefined){
          if (tmpTitle[0] != undefined) {
              
            var itemTitle=""; itemURL="";    
            try{
              itemTitle = tmpTitle[0].children[0].data;
              itemURL = tmpTitle[0].attribs.href;
                                                   
              if (itemTitle != "" && itemURL != "")
              {
                tmpObj = {Title:itemTitle, URL:itemURL};              
                cpsearchresults.push(tmpObj);
                
                //print(cpsearchresults[cpsearchresults.length-1].Title);
              }
            }
            catch(err)
            {
              //print('Err:'+err);
               //skip record and continue
            }    
        }}
      });       
      
      ///////////// Generate RSS feed
      var feed = new rss({
          title: 'Codeproject search result Sample RSS Feed',
          description: 'Code project search Results RSS feed through node.js sample',
          feed_url: 'http://codeproject.com',
          site_url: 'http://codeproject.com',
          author: 'You'
      });  
          
      /* loop over data and add to feed    */
      cpsearchresults.forEach(function(item){
        //print(item.Title);
        feed.item({
            title:  item.Title,
            url: 'http://www.codeproject.com'+item.URL          
        });                             
      
      });
                    
      //Print the RSS feed out as response
      responseObj.write(feed.xml());            
      responseObj.end();                    
    }
});
var html_parser = new htmlparser.Parser(handler);

var req = http.request(options, function(res) {
  console.log('STATUS: ' + res.statusCode);
  //console.log('HEADERS: ' + JSON.stringify(res.headers));
  res.setEncoding('utf8');
  
  var alldata = "";
  res.on('data', function (chunk) {
    alldata = alldata+chunk;
  });
  res.on('end', function(){
    html_parser.parseComplete(alldata);
  });  
});

req.on('error', function(e) {
  console.log('problem with request: ' + e.message);
});

req.write('data\n');
req.end();
}

function print(value)
{
  console.log('\n'+value);
}

The above example is similar to the previous example until the step where it fetches content... previous example fetched an RSS, this fetches an HTML page.

Then it extracts + loops through search result nodes in the HTML of CodeProject's search page using the soupselect library which is documented here.

Then create an array of search result titles + links.

Then converts that array into RSS feed, using node.js rss library and prints out as HTML response.

node's Core Objective - Non-blocking Operations

node.js executes line1 and goes to line2... without waiting on line1 even, if say line1 makes a call to a method which takes time to complete.

I am not sure if it is even possible to forcefully make node wait on a line... if at some point of implementation, you even require that. (Is it possible.. anybody? Let me know.)

That's it for now. Thanks for reading!