Ibuki - PUB-SUB Messaging and Data Service for Node.js

• Download demo - 14.4 KB

Introduction

When you create a node.js server side application, soon it grows quite big consisting of several modules. In such an application, it becomes challenging to send data from a module X to a module Y. A requirement is felt to have a centralized messaging cum data service which enables you to send and receive arbitrary chunk of data irrespective of the module or place of code. You can certainly do it using eventEmitter mechanism, but here I am going to show a more versatile, generic and unified way of doing so. Ibuki data service attempts to resolve this problem in a systematic manner. This utility is a must have code snippet for any node.js application.

Background

Way back in year 2016, I was working with a big banking project in backbone.js. There, it was a nightmare to send object messages from one place to another. They had come up with a concept of backbone.radio channel which facilitated sending messages in publisher / subscriber fashion in a great manner. But this had got limitations. Later, in 2017, when I was working with Angular 2, I came across the great rxjs library. This library is a revolution in programming. This unleashes a new way of programming called reactive programming. I realized that the concept of backbone.radio channel can be combined with rxjs to create a very robust and well organized pub / sub service which is fail proof in heavy traffic conditions. I ended up creating this Ibuki service which I used in over 100 projects successfully. Ibuki is a Japanese word which means breath. In a way, Ibuki service is a breath control service for a massive node.js application.

What Business Problem Ibuki Solves

Imagine the situation when your node.js application has grown up to 100+ modules or files. You need to send data from one module to another module and the entire communication needs to be asyncronous. Ibuki data services can be used to send named messages from any module to any other module. I have tested it in heavy traffic conditions and it has never let me down. Ibuki provides a very unique, simplified and generic way to do such communications. Through proper use of Ibuki, you can convert deeply nested asyncronous calls to single level of nesting which can highly simplify the code and make it understandable to other team members.

I have also successfully used Ibuki in several Angular 2 / 4 / 5 / 6 applications using it as an Angular service. I will discuss Ibuki usage in Angular in a separate blog article. For Angular, there are a few small changes to be made in Ibuki code like making use of import statements instead of require statement.

I say Ibuki is not merely a small piece of code but it is a different concept of thinking in an asyncronous world of programming. If you use node.js at server side and JavaScript at client side in single page application, then Ibuki enables to use the same control mecanism and similar code in both client and server side. This highly simplifies the work and increases the throughput. In fact, through proper adjustments, Ibuki concept can be used in any JavaScript based SPA or node.js application.

Why You Need Multiple Modules in node.js Application

Multiple modules make it easy to manage your node.js application when it grows big. Say for example, your node.js application connects to database, it does some file input / output operations. It makes some http calls to REST enabled web services, it does some housekeeping work, it listens to inbound HTTP calls and it also contains entire business logic. If you create a single module or a file to do all the above work, then you will start scratching your head some day.

In such situations, it is suggested that you create modules such as database.js, fileio.js, http-calls.js, houskeeping.js, routing.js, business-logic1.js, business-logic2.js ... and so on. Modulerizing your application like this will simplify its understanding to other developers also. Say for example, all database related code is in database.js module. In node.js paradigm, most of these modules need to communicate with each other in an asyncronous manner transferring an arbitrary size and structure of data from one module to another. This is where Ibuki service comes to the rescue.

An Entire node.js Application using Ibuki

Now I will walk through an entire node.js app which makes use of Ibuki service for intra modules communication. I am keeping the app very simple only to explain the basic idea of object messaging in node.js through Ibuki service. The app has only three modules or files. They are index.js, ibuki.js and http-calls.js.

The index.js is an entry point, ibuki.js is the Ibuki service module and http-calls.js is a module which does all http-calls to external APIs. Flow of the app is as follows:

• User browses the url http://localhost:8080/states
• The index.js module in the app is hit because this is the entry point and this implements the http endpoint for the said url.
• The index.js makes an asyncronous call to http-calls module by using ibuki module and passes it the response object and an external REST API url wrapped in an object.
• The http-calls module calls the external REST API and updates the response object and sends back the response to user's browser.
• The user sees the states in India on the browser screen in JSON format.

index.js

'use strict';
const path = require('path');
const express = require('express');
const ibuki = require('./ibuki');
const httpCalls = require('./http-calls');
const app = express();

// CORS
app.use((req, res, next) => {
    res.header("Access-Control-Allow-Origin", "*");
    res.header('Access-Control-Allow-Methods', ' GET,PUT,POST,DELETE');
    res.header("Access-Control-Allow-Headers", 
               "Origin, X-Requested-With, Content-Type, Accept, Authorization");
    next();
});

app.get('/states', function (req, res) {
    const obj = {
        url: ' http://services.groupkt.com/state/get/IND/all',
        res: res
    };
    ibuki.emit('get:states', obj);
});

app.listen(8080, _ => console.log('Running on port ' + 8080));

ibuki.js

'use strict';
var ibuki = {};
const rx = require('rxjs');
const operators = require('rxjs/operators');
const subject = new rx.Subject();
ibuki.emit = (id, options) => {
    subject
        .next({
            id: id,
            data: options
        });
}
ibuki.filterOn = (id) => {
    return (subject.pipe(operators.filter(d => (d.id === id))));
}
module.exports = ibuki;

http-calls.js

'use strict';
const axios = require('axios');
const ibuki = require('./ibuki');
let httpCalls = {};
ibuki.filterOn('get:states').subscribe(
    d => {
        const obj = d.data;
        axios.get(obj.url)
            .then(result => {
                const states = result.data.RestResponse.result;
                obj.res.json(states);
            })
            .catch(err => {
                obj.res.send(err.message);
            })
    }
);
module.exports = httpCalls;

Explanation of Code

index.js

The following code in index.js is relevant code which receives the http request.

app.get('/states', function (req, res) {
    const obj = {
        url: ' http://services.groupkt.com/state/get/IND/all',
        res: res
    };
    ibuki.emit('get:states', obj);
});

In the above code, an object obj is created and this is published through ibuki.emit('get:states', obj). The obj is arbitrary data which wraps the external REST API url and the response object. Now all the subscribers who have already subscribed to 'get:states' will get the data in form of obj. The http-calls module is the one which already subscribes to 'get:states' hence it will receive the obj.

http-calls.js

Following is the relevant code in http-calls.js which subscribes to 'get:states'.

ibuki.filterOn('get:states').subscribe(
    d => {
        const obj = d.data;
        axios.get(obj.url)
            .then(result => {
                const states = result.data.RestResponse.result;
                obj.res.json(states);
            })
            .catch(err => {
                obj.res.send(err.message);
            })
    }
);

In the above code, the ibuki.filterOn('get:states').subscribe(... is responsible for subscription. So when a call is received at index.js, the data is transferred to http-calls.js through ibuki module in the form of obj. The obj has the information of url to be called. By using the axios library, a get request is made to the url which returns a promise. In the success of promise (.then) the states object is created and this data is pushed to the http response through the line obj.res.json(states); In case of error, the error message is sent through the response object through the code obj.res.send(err.message);

How to Install and Run this Demo Application from my GitHub

Download the code from GitHub:

git clone https://github.com/capitalch/ibuki-node-demo

Change directory in Windows environment:

cd ibuki-node-demo

Install dependencies:

npm install

Node.js should be preinstalled in your system. Run the node.js server:

node index

Now in your browser, navigate to:

http://localhost://8081/states

You will see the following screen:

Source Code Included

I have included all code files in the form of ibuki-node-demo.zip. You can download it. You need to run npm install to install all dependencies in package.json file before running the application.