We are going to implement the technique by using Apple HTTP Live Streaming (HLS) with Video on Demand (VOD) using a desktop application and IIS.
Introduction
What Is Adaptive Bit-Rate Streaming?
“Adaptive bit-rate streaming is a technique used in streaming multimedia over computer networks” - Wikipedia.
The key point is adapting the streaming according to the client machine status like bandwidth up down. By using HLS, we are going to implement the thought.
What is HLS?
Apple HTTP Live Streaming known as HLS, which is a video streaming protocol, based on HTTP.
HLS Supports:
- Live streaming
- Video on demand (VOD)
- Multiple bit rates (MBR) Streaming
- Smart switching streams based on client environment
- Media encryption
- User authentication
Learn more: https://developer.apple.com/documentation/http_live_streaming
We are going to separate the whole process in different steps with the main thought of:
- Transcoding Tools
- Server Configuration & Testing
Transcoding Tools
In this step, we need to encode the input video file to different segment with VOD playlist. Using ffmpeg, we are going to transcode our input file by command.
We are going to use Windows Form application to transcode the video file. Let’s create a new windows form application and prepare the UI.
Open Visual Studio Go to > File > New > Project, then choose Windows Form Application by clicking menu > Windows Desktop.
Go to App.config to add the output folder path.
Manage Encoding Process
Add a class library which will process the task while we are going to transcode the video file. This is a custom solution based on the original source https://github.com/vladjerca/FFMpegSharp.
Which has some modification for multi-bit rate segment command.
FFMpeg
Let’s download the open source ffmpeg libraries from the below download link:
Version Used- ffmpeg-3.1.4. Unzip downloaded file, there will be several folders like the below folder image.
Open bin folder to copy the below three files to application directory, in this example, the files copied to debug folder to access.
Let’s get started with the coding section.
Transcoder
First, we are going to browse the video file (.mp4) with the below code section to display the browsed path in textbox, which is our actual input path.
txtFile.Text = string.Empty;
OpenFileDialog ofdFile = new OpenFileDialog();
ofdFile.Multiselect = false;
ofdFile.Filter = "Video files (*.mp4)|*.mp4";
ofdFile.Title = "Select File.";
if (ofdFile.ShowDialog() == DialogResult.OK)
{
var file = ofdFile.FileNames;
txtFile.Text = file[0].ToString();
}
Next, we are going to process the video file into common video resolution with ffmpeg command:
- 1080p
- 720p
- 480p
- 360p
- 240p
Here’s the different command for each resolution:
1080p
" -i \"{0}\" -vf scale=w=1920:h=1080:force_original_aspect_ratio=decrease
-c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48
-keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 5000k -maxrate 5350k
-bufsize 7500k -b:a 192k -hls_segment_filename " + fileOutput + "1080p_%d.ts "
+ fileOutput + "1080p.m3u8"
720p
" -i \"{0}\" -vf scale=w=1280:h=720:force_original_aspect_ratio=decrease
-c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 -keyint_min 48
-hls_time 4 -hls_playlist_type vod -b:v 2800k -maxrate 2996k -bufsize 4200k -b:a 128k
-hls_segment_filename " + fileOutput + "720p_%d.ts "
+ fileOutput + "720p.m3u8"
480p
" -i \"{0}\" -vf scale=w=842:h=480:force_original_aspect_ratio=decrease
-c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48
-keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 1400k -maxrate 1498k
-bufsize 2100k -b:a 128k -hls_segment_filename " + fileOutput + "480p_%d.ts "
+ fileOutput + "480p.m3u8"
360p
" -i \"{0}\" -vf scale=w=640:h=360:force_original_aspect_ratio=decrease
-c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48
-keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 800k -maxrate 856k
-bufsize 1200k -b:a 96k -hls_segment_filename " + fileOutput + "360p_%d.ts "
+ fileOutput + "360p.m3u8"
240p
" -i \"{0}\" -vf scale=w=426:h=240:force_original_aspect_ratio=decrease
-c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48
-keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 240k -maxrate 240k
-bufsize 480k -b:a 64k -hls_segment_filename " + fileOutput + "240p_%d.ts "
+ fileOutput + "240p.m3u8"
For multiple bitrate file, we need to create a master playlist. The below code section will create a master playlist with all those command information.
string path = fileOutput + "index.m3u8";
File.Create(path).Dispose();
string[] line ={
"#EXTM3U",
"#EXT-X-VERSION:3",
"#EXT-X-STREAM-INF:BANDWIDTH=10000,RESOLUTION=426x240",
"240p.m3u8",
"#EXT-X-STREAM-INF:BANDWIDTH=420000,RESOLUTION=640x360",
"360p.m3u8",
"#EXT-X-STREAM-INF:BANDWIDTH=680000,RESOLUTION=842x480",
"480p.m3u8",
"#EXT-X-STREAM-INF:BANDWIDTH=1256000,RESOLUTION=1280x720",
"720p.m3u8",
"#EXT-X-STREAM-INF:BANDWIDTH=2000000,RESOLUTION=1920x1080",
"1080p.m3u8"
};
File.WriteAllLines(path, line);
Then, with the below code section, start the process by passing the command string with input source.
FFMpeg encoder = new FFMpeg();
encoder.OnProgress += encoder_OnProgress;
Task.Run(() => encoder.ToTS(inputFile, conversionArgs));
Finally, the full Transcoding event:
private void btnTranscode_Click(object sender, EventArgs e)
{
try
{
bool isProcessing = IsProcessRunning("ffmpeg");
if (!isProcessing)
{
if (txtFile.Text != string.Empty)
{
string _rootPath = Environment.CurrentDirectory;
string ffmpegOutput = ConfigurationManager.AppSettings["ffmpegOutput"];
this.Cursor = Cursors.WaitCursor;
this.Text = "Transcoding...";
btnTranscode.Text = "Transcoding..";
string inputFile = txtFile.Text.ToString();
string fileOutput = ffmpegOutput +
toUnderscore(Path.GetFileNameWithoutExtension(inputFile)) + "\\";
if (!Directory.Exists(fileOutput))
{
SetFolderPermission(fileOutput);
DirectoryInfo di = Directory.CreateDirectory(fileOutput);
}
string path = fileOutput + "index.m3u8";
File.Create(path).Dispose();
string[] line ={
"#EXTM3U",
"#EXT-X-VERSION:3",
"#EXT-X-STREAM-INF:BANDWIDTH=10000,RESOLUTION=426x240",
"240p.m3u8",
"#EXT-X-STREAM-INF:BANDWIDTH=420000,RESOLUTION=640x360",
"360p.m3u8",
"#EXT-X-STREAM-INF:BANDWIDTH=680000,RESOLUTION=842x480",
"480p.m3u8",
"#EXT-X-STREAM-INF:BANDWIDTH=1256000,RESOLUTION=1280x720",
"720p.m3u8",
"#EXT-X-STREAM-INF:BANDWIDTH=2000000,RESOLUTION=1920x1080",
"1080p.m3u8"
};
File.WriteAllLines(path, line);
string conversionArgs = string.Format("-hide_banner -y" +
" -i \"{0}\" -vf scale=w=426:h=240:
force_original_aspect_ratio=decrease -c:a
aac -ar 48000 -c:v h264 -profile:v main
-crf 20 -sc_threshold 0 -g 48 -keyint_min 48
-hls_time 4 -hls_playlist_type vod -b:v 240k
-maxrate 240k -bufsize 480k -b:a 64k
-hls_segment_filename " + fileOutput + "240p_%d.ts "
+ fileOutput + "240p.m3u8" +
" -i \"{0}\" -vf scale=w=640:h=360:
force_original_aspect_ratio=decrease
-c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20
-sc_threshold 0 -g 48 -keyint_min 48 -hls_time 4
-hls_playlist_type vod -b:v 800k -maxrate 856k
-bufsize 1200k
-b:a 96k -hls_segment_filename " + fileOutput
+ "360p_%d.ts " + fileOutput + "360p.m3u8" +
" -i \"{0}\" -vf scale=w=842:h=480:
force_original_aspect_ratio=decrease -c:a aac -ar 48000
-c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48
-keyint_min 48 -hls_time 4 -hls_playlist_type vod
-b:v 1400k
-maxrate 1498k -bufsize 2100k -b:a 128k
-hls_segment_filename "
+ fileOutput + "480p_%d.ts " + fileOutput
+ "480p.m3u8" +
" -i \"{0}\" -vf scale=w=1280:h=720:
force_original_aspect_ratio=decrease -c:a aac -ar 48000
-c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48
-keyint_min 48 -hls_time 4 -hls_playlist_type vod
-b:v 2800k
-maxrate 2996k -bufsize 4200k -b:a 128k
-hls_segment_filename "
+ fileOutput + "720p_%d.ts " + fileOutput
+ "720p.m3u8" +
" -i \"{0}\" -vf scale=w=1920:h=1080:
force_original_aspect_ratio=decrease -c:a aac -ar 48000
-c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48
-keyint_min 48 -hls_time 4 -hls_playlist_type vod
-b:v 5000k
-maxrate 5350k -bufsize 7500k -b:a 192k
-hls_segment_filename "
+ fileOutput + "1080p_%d.ts " + fileOutput
+ "1080p.m3u8", inputFile, fileOutput);
FFMpeg encoder = new FFMpeg();
encoder.OnProgress += encoder_OnProgress;
Task.Run(() => encoder.ToTS(inputFile, conversionArgs));
}
}
}
catch (Exception ex)
{
ex.ToString();
}
}
The below code section will show the progress in progress bar.
void encoder_OnProgress(int percentage)
{
try
{
Invoke(new Action(() =>
{
progressBar1.Value = percentage;
this.Text = "Transcoding..." + percentage + "%";
}));
if (percentage == 100)
{
Invoke(new Action(() =>
{
this.btnTranscode.Text = "Transcode";
this.Cursor = Cursors.Default;
}));
}
}
catch (Exception ex)
{
ex.ToString();
}
}
Now run the application & browse .mp4 file, then click on transcode button to start the transcoding process.
Output: Go to > C:\ProgramData\transcode
There’s a single index file with .m3u8 extension with multi-resolution information of sub playlist which is the master playlist.
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=10000,RESOLUTION=426x240
240p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=420000,RESOLUTION=640x360
360p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=680000,RESOLUTION=842x480
480p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1256000,RESOLUTION=1280x720
720p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2000000,RESOLUTION=1920x1080
1080p.m3u8
Sub-Playlist: As you can see from master playlist 240p.m3u8 is listed which has another list of segmented file with .ts extension.
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:5
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:4.004000,
240p_0.ts
#EXTINF:4.004000,
240p_1.ts
#EXT-X-ENDLIST
Learn more at this link.
Streaming Server (IIS) Configuration & Testing
Create a new website in IIS to serve the HLS live stream. We need to add MIME (Multipurpose Internet Mail Extensions) types to our website to play .m3u8 extension.
Double click on MIME Types to add new extension.
As you can see in Web.Config, configuration is added.
<configuration>
<system.webserver>
<staticcontent>
<mimemap fileextension=".m3u8" mimetype="application/x-mpegURL">
</mimemap></staticcontent>
</system.webserver>
</configuration>
Transfer transcoded file in videos folder, then add player in index.html file to play the video from live streaming server.
Next, we are going to add video.js player for live streaming in Index.htm.
Video.Js
<link href="https://vjs.zencdn.net/7.4.1/video-js.css" rel="stylesheet" />
<script src="https://vjs.zencdn.net/ie8/1.1.2/videojs-ie8.min.js"></script>
<script src='https://vjs.zencdn.net/7.4.1/video.js'></script>
<script src="https://unpkg.com/videojs-flash/dist/videojs-flash.js"></script>
<script src="https://unpkg.com/videojs-contrib-hls/dist/videojs-contrib-hls.js"></script>
Here videojs-contrib-hls.js is HLS library for cross browser HLS support.
Player: As you can see, our source path is set to index.m3u8 file from the below source tag.
<video class="video-js vjs-default-skin vjs-big-play-centered"
data-setup='{"controls": true, "autoplay": true,
"aspectRatio":"16:9", "fluid": true}'>
<source src="http://localhost:8081/videos/Sunset_at_Bhawal_Resort/index.m3u8"
type="application/x-mpegURL" />
</video>
Learn more: http://videojs.github.io/videojs-contrib-hls
Finally, Index.htm:
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<title>IIS HLS-VOD</title>
<link href="https://vjs.zencdn.net/7.4.1/video-js.css" rel="stylesheet" />
<script src="https://vjs.zencdn.net/ie8/1.1.2/videojs-ie8.min.js"></script>
<script src='https://vjs.zencdn.net/7.4.1/video.js'></script>
<script src="https://unpkg.com/videojs-flash/dist/videojs-flash.js"></script>
<script src="https://unpkg.com/videojs-contrib-hls/dist/videojs-contrib-hls.js"></script>
</head>
<body style="margin:0;padding:0;">
<div>
<video class="video-js vjs-default-skin vjs-big-play-centered"
data-setup='{"controls": true, "autoplay": true,
"aspectRatio":"16:9", "fluid": true}'>
<source src="http://localhost:8081/videos/Sunset_at_Bhawal_Resort/index.m3u8"
type="application/x-mpegURL" />
</video>
</div>
</body>
</html>
Open browser to test the streaming. Go to URL > http://localhost:8081
Client Test
I am going to use Chrome browser for testing. As you can see from the below screen, the video is playing in browser.
Change the screen size to see the adaptive live streaming like the above image. Hope this post is going to clarify the thought behind adaptive video playing also creating a video streaming server in IIS.
References
