HTTP 206 Partial Content In Node.js

• Download source code - 2 KB

Table of Contents

• Introduction
• Prerequisites
• A brief of partial content
• Get started to implement in Node.js
• Test the implementation
• Conclusion

Introduction

In this article, I would like to explain the basic concept of HTTP status 206 Partial Content and a step-by-step implementation walkthrough with Node.js. Also, we will test the code with an example based on the most common scenario of its usage: an HTML5 page which is able to play video file starting at any second.

Prerequisites

• Basic HTTP Knowledge
• Intermediate Node.js Skill
• Basic HTML5 Skill
• Basic JavaScript Skill

A Brief of Partial Content

The HTTP 206 Partial Content status code and its related headers provide a mechanism which allows browser and other user agents to receive partial content instead of entire one from server. This mechanism is widely used in streaming a video file and supported by most browsers and players such as Windows Media Player and VLC Player.

The basic workflow could be explained by these following steps:

1. Browser requests the content.
2. Server tells browser that the content can be requested partially with Accept-Ranges header.
3. Browser resends the request, tells server the expecting range with Range header.
4. Server responses browser in one of following situations:
   • If range is available, server returns the partial content with status 206 Partial Content. Range of current content will be indicated in Content-Range header.
   • If range is unavailable (for example, greater than total bytes of content), server returns status 416 Requested Range Not Satisfiable. The available range will be indicated in Content-Range header too.

Let's take a look at each key header of these steps.

Accept-Ranges: bytes

This is the header which is sent by server, represents the content that can be partially returned to browser. The value indicates the acceptable unit of each range request, usually is bytes in most situations.

Range: bytes=(start)-(end)

This is the header for browser telling server the expecting range of content. Note that start and end positions are both inclusive and zero-based. This header could be sent without one of them in the following meanings:

• If end position is omitted, server returns the content from indicated start position to the position of last available byte.
• If start position is omitted, the end position will be described as how many bytes shall server returns counting from the last available byte.

Content-Range: bytes (start)-(end)/(total)

This is the header which shall appear following HTTP status 206. Values start and end represent the range of current content. Like Range header, both values are inclusive and zero-based. Value total indicates the total available bytes.

Content-Range: */(total)

This is same header but in another format and will only be sent following HTTP status 416. Value total also indicates the total available bytes of content.

Here are a couple examples of a file with 2048 bytes long. Note the different meaning of end when start is omitted.

Request First 1024 Bytes

What browser sends:

GET /dota2/techies.mp4 HTTP/1.1
Host: localhost:8000
Range: bytes=0-1023

What server returns:

HTTP/1.1 206 Partial Content
Date: Mon, 15 Sep 2014 22:19:34 GMT
Content-Type: video/mp4
Content-Range: bytes 0-1023/2048
Content-Length: 1024

(Content...)

Request Without End Position

What browser sends:

GET /dota2/techies.mp4 HTTP/1.1
Host: localhost:8000
Range: bytes=1024-

What server returns:

HTTP/1.1 206 Partial Content
Date: Mon, 15 Sep 2014 22:19:34 GMT
Content-Type: video/mp4
Content-Range: bytes 1024-2047/2048
Content-Length: 1024

(Content...)

Note that server does not have to return all remaining bytes in single response especially when content is too long or there are other performance considerations. So, the following two examples are also acceptable in this case:

Content-Range: bytes 1024-1535/2048
Content-Length: 512

Server only returns half of remaining content. The range of next request will start at 1536^th byte.

Content-Range: bytes 1024-1279/2048
Content-Length: 256

Server only returns 256 bytes of remaining content. The range of next request will start at 1280^th byte.

Request Last 512 Bytes

What browser sends:

GET /dota2/techies.mp4 HTTP/1.1
Host: localhost:8000
Range: bytes=-512

What server returns:

HTTP/1.1 206 Partial Content
Date: Mon, 15 Sep 2014 22:19:34 GMT
Content-Type: video/mp4
Content-Range: bytes 1536-2047/2048
Content-Length: 512

(Content...)

Request with Unavailable Range

What browser sends:

GET /dota2/techies.mp4 HTTP/1.1
Host: localhost:8000
Range: bytes=1024-4096

What server returns:

HTTP/1.1 416 Requested Range Not Satisfiable
Date: Mon, 15 Sep 2014 22:19:34 GMT
Content-Range: bytes */2048

With understanding of workflow and headers above, now we are able to implement the mechanism in Node.js.

Get Started to Implement in Node.js

Step 1 - Create a Simple HTTP Server

We will start from a basic HTTP server as the following example shows. This is pretty enough to handle most of requests from browsers. At first, we initialize each object we need, and indicate where the files are located at with initFolder. We also list couple of filename extensions and their corresponding MIME names to construct a dictionary, which is for generating Content-Type header. In the callback function httpListener(), we limit GET as the only allowed HTTP method. Before we start to fulfill the request, server will return status 405 Method Not Allowed if other methods appear and return status 404 Not Found if file does not exist in initFolder.

// Initialize all required objects.
var http = require("http");
var fs = require("fs");
var path = require("path");
var url = require('url');

// Give the initial folder. Change the location to whatever you want.
var initFolder = 'C:\\Users\\User\\Videos';

// List filename extensions and MIME names we need as a dictionary. 
var mimeNames = {
    '.css': 'text/css',
    '.html': 'text/html',
    '.js': 'application/javascript',
    '.mp3': 'audio/mpeg',
    '.mp4': 'video/mp4',
    '.ogg': 'application/ogg', 
    '.ogv': 'video/ogg', 
    '.oga': 'audio/ogg',
    '.txt': 'text/plain',
    '.wav': 'audio/x-wav',
    '.webm': 'video/webm'
};

http.createServer(httpListener).listen(8000);

function httpListener (request, response) {
    // We will only accept 'GET' method. Otherwise will return 405 'Method Not Allowed'.
    if (request.method != 'GET') { 
        sendResponse(response, 405, {'Allow' : 'GET'}, null);
        return null;
    }

    var filename = 
        initFolder + url.parse(request.url, true, true).pathname.split('/').join(path.sep);

    var responseHeaders = {};
    var stat = fs.statSync(filename);

    // Check if file exists. If not, will return the 404 'Not Found'. 
    if (!fs.existsSync(filename)) {
        sendResponse(response, 404, null, null);
        return null;
    }
    responseHeaders['Content-Type'] = getMimeNameFromExt(path.extname(filename));
    responseHeaders['Content-Length'] = stat.size; // File size.
        
    sendResponse(response, 200, responseHeaders, fs.createReadStream(filename));
}

function sendResponse(response, responseStatus, responseHeaders, readable) {
    response.writeHead(responseStatus, responseHeaders);

    if (readable == null)
        response.end();
    else
        readable.on('open', function () {
            readable.pipe(response);
        });

    return null;
}

function getMimeNameFromExt(ext) {
    var result = mimeNames[ext.toLowerCase()];
    
    // It's better to give a default value.
    if (result == null)
        result = 'application/octet-stream';
    
    return result;
}

Step 2 - Capture the Range Header by Using Regular Expression

With the basic HTTP server, now we can handle the Range header as the following code shows. We split the header with regular expression to capture start and end strings. Then use parseInt() method to parse them to integers. If returned value is NaN (not a number), the string does not exist in header. The parameter totalLength represents total bytes of current file. We will use it to calculate start and end positions.

function readRangeHeader(range, totalLength) {
        /*
         * Example of the method 'split' with regular expression.
         * 
         * Input: bytes=100-200
         * Output: [null, 100, 200, null]
         * 
         * Input: bytes=-200
         * Output: [null, null, 200, null]
         */

    if (range == null || range.length == 0)
        return null;

    var array = range.split(/bytes=([0-9]*)-([0-9]*)/);
    var start = parseInt(array[1]);
    var end = parseInt(array[2]);
    var result = {
        Start: isNaN(start) ? 0 : start,
        End: isNaN(end) ? (totalLength - 1) : end
    };
    
    if (!isNaN(start) && isNaN(end)) {
        result.Start = start;
        result.End = totalLength - 1;
    }

    if (isNaN(start) && !isNaN(end)) {
        result.Start = totalLength - end;
        result.End = totalLength - 1;
    }

    return result;
}

Step 3 - Check If Range Can Be Satisfied

Back to the function httpListener(), now we check if the range is available after the HTTP method gets approved. If browser does not send Range header, the request will be directly treated as normal request. Server returns entire file and HTTP status is 200 OK. Otherwise, we will see if start or end position is greater or equal to file length. If one of them is, the range can not be fulfilled. The status will be 416 Requested Range Not Satisfiable and the Content-Range will be sent.

var responseHeaders = {};
var stat = fs.statSync(filename);
var rangeRequest = readRangeHeader(request.headers['range'], stat.size);

// If 'Range' header exists, we will parse it with Regular Expression.
if (rangeRequest == null) {
    responseHeaders['Content-Type'] = getMimeNameFromExt(path.extname(filename));
    responseHeaders['Content-Length'] = stat.size;  // File size.
    responseHeaders['Accept-Ranges'] = 'bytes';

    //  If not, will return file directly.
    sendResponse(response, 200, responseHeaders, fs.createReadStream(filename));
    return null;
}

var start = rangeRequest.Start;
var end = rangeRequest.End;

// If the range can't be fulfilled.
if (start >= stat.size || end >= stat.size) {
    // Indicate the acceptable range.
    responseHeaders['Content-Range'] = 'bytes */' + stat.size; // File size.

    // Return the 416 'Requested Range Not Satisfiable'.
    sendResponse(response, 416, responseHeaders, null);
    return null;
}

Step 4 - Fulfill the Request

Finally the last puzzle piece comes. For the status 206 Partial Content, we have another format of Content-Range header including start, end and total bytes of current file. We also have Content-Length header and the value is exactly equal to the difference between start and end. In the last statement, we call createReadStream() and assign start and end values to the object of second parameter options, which means the returned stream will be only readable from/to the positions.

// Indicate the current range.
responseHeaders['Content-Range'] = 'bytes ' + start + '-' + end + '/' + stat.size;
responseHeaders['Content-Length'] = start == end ? 0 : (end - start + 1);
responseHeaders['Content-Type'] = getMimeNameFromExt(path.extname(filename));
responseHeaders['Accept-Ranges'] = 'bytes';
responseHeaders['Cache-Control'] = 'no-cache';

// Return the 206 'Partial Content'.
sendResponse(response, 206,
    responseHeaders, fs.createReadStream(filename, { start: start, end: end }));

Here is the complete httpListener() callback function.

function httpListener(request, response) {
    // We will only accept 'GET' method. Otherwise will return 405 'Method Not Allowed'.
    if (request.method != 'GET') {
        sendResponse(response, 405, { 'Allow': 'GET' }, null);
        return null;
    }

    var filename =
        initFolder + url.parse(request.url, true, true).pathname.split('/').join(path.sep);

    // Check if file exists. If not, will return the 404 'Not Found'. 
    if (!fs.existsSync(filename)) {
        sendResponse(response, 404, null, null);
        return null;
    }

    var responseHeaders = {};
    var stat = fs.statSync(filename);
    var rangeRequest = readRangeHeader(request.headers['range'], stat.size);

    // If 'Range' header exists, we will parse it with Regular Expression.
    if (rangeRequest == null) {
        responseHeaders['Content-Type'] = getMimeNameFromExt(path.extname(filename));
        responseHeaders['Content-Length'] = stat.size;  // File size.
        responseHeaders['Accept-Ranges'] = 'bytes';

        //  If not, will return file directly.
        sendResponse(response, 200, responseHeaders, fs.createReadStream(filename));
        return null;
    }

    var start = rangeRequest.Start;
    var end = rangeRequest.End;

    // If the range can't be fulfilled. 
    if (start >= stat.size || end >= stat.size) {
        // Indicate the acceptable range.
        responseHeaders['Content-Range'] = 'bytes */' + stat.size; // File size.

        // Return the 416 'Requested Range Not Satisfiable'.
        sendResponse(response, 416, responseHeaders, null);
        return null;
    }

    // Indicate the current range. 
    responseHeaders['Content-Range'] = 'bytes ' + start + '-' + end + '/' + stat.size;
    responseHeaders['Content-Length'] = start == end ? 0 : (end - start + 1);
    responseHeaders['Content-Type'] = getMimeNameFromExt(path.extname(filename));
    responseHeaders['Accept-Ranges'] = 'bytes';
    responseHeaders['Cache-Control'] = 'no-cache';

    // Return the 206 'Partial Content'.
    sendResponse(response, 206, 
        responseHeaders, fs.createReadStream(filename, { start: start, end: end }));
}

Also, the entire workflow can be summarized with the following chart:

Test the Implementation

So how do we test our work? As we just mentioned in the Introduction, the most common scenario of partial content is streaming and playing videos. So we create an HTML5 page which includes a <video/> with ID mainPlayer and a <source/> tag. Function onLoad() will be triggered when mainPlayer has preloaded the metadata of current video. It is used for checking if there is any numeric parameter existing in URL. If yes, mainPlayer will skip to the indicated second.

<!DOCTYPE html>
<html>
    <head>
        <script type="text/javascript">

            function onLoad() {
                var sec = parseInt(document.location.search.substr(1));
                
                if (!isNaN(sec))
                    mainPlayer.currentTime = sec;
            }
        
        </script>
        <title>Partial Content Demonstration</title>
    </head>
    <body>
        <h3>Partial Content Demonstration</h3>
        <hr />
        <video id="mainPlayer" width="640" height="360" 
            autoplay="autoplay" controls="controls" onloadedmetadata="onLoad()">
            <source src="dota2/techies.mp4" />
        </video>
    </body>
</html>

Now we save our page as "player.html" under initFolder along with video file "dota2/techies.mp4". Activate Node.js, execute the script, then open the URL in browser:

http://localhost:8000/player.html

This is how it looks like in Chrome:

Because there are no parameters in the URL, the file will be played starting at 0^th second.

Next is the fun part. Let's try to open this one and see what happens:

http://localhost:8000/player.html?60 

If you press F12 to open Chrome Developer Tools, switch to Network tab and click the detail of latest log. You will notice that the Range header string sent by your browser is something like this:

Range:bytes=225084502-

Pretty interesting, right? When function onLoad() changes the currentTime property, browser computes the corresponding byte position of 60^th second in this video. Because mainPlayer has preloaded the metadata, including format, bit rate and other fundamental information, the start position comes out almost immediately. After that, the browser is able to download and play the video without requesting first 60 seconds. Same thing happens if you click the timeline before mainPlayer reaches the position you just clicked. It works!!

Conclusion

We have implemented an HTTP server in Node.js which supports partial content. We also tested it with an HTML5 page. But this is just a beginning. If you have understood the whole thing about these headers and workflow, you can try to implement it with other frameworks such like ASP.NET MVC and Web WCF Service. But don't forget to enable Task Manager to see CPU and memory usage. Like we discussed in A brief of partial content, server does not have to return all remaining bytes in single response. To find out a balance point of performance will be an important mission.