Video Manipulation with the Canvas Element
One of the designers in my company asked me to help him with the creation of a transparent video which runs on top of an image in HTML. So I did some digging and found a very good resource to do that - Manipulating video using canvas. This post will explain how to combine the video and canvas elements in order to create transparency inside a video during runtime using JavaScript. I’ve based the code on the previous link so they deserve all the credit. You can download the demo code from here.
The Demands
The designer has created a simple movie file with a red square that is floating on top of a black background. He wanted that during runtime, the black background will be transparent so the image that he wants to show underneath will be visible. Here is a figure that shows how the movie looks like:
The HTML File
<!DOCTYPE HTML>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>Working with Canvas</title>
<script type="text/javascript" src="main.js"></script>
<link href="style.css" rel="stylesheet" />
</head>
<body onload="handlePageLoad()">
<video id="video" controls="controls" width="997px" height="587px" autoplay>
<source src="vid.mp4" type="video/mp4" />
</video>
<div>
<canvas id="c1" width="997px" height="587px" />
<canvas id="c2" width="997px" height="587px" />
</div>
</body>
</html>
The HTML includes two canvas elements and the video element. I use two canvas elements because one will hold the buffer that I’m working on (and make the black background transparent) and the other will show the movie after the adjustments are finished. I also use the following style.css file:
#c1
{
background-image<span class="code-none">: url(Bg.png)<span class="code-none">;
background-repeat<span class="code-none">: no-repeat<span class="code-none">;
<span class="code-none">}
#c2
<span class="code-none">{
display<span class="code-none">: none<span class="code-none">;
<span class="code-none">}
#video
<span class="code-none">{
display<span class="code-none">: none<span class="code-none">;
<span class="code-none">}</span></span></span></span></span></span></span></span></span></span></span></span></span>
In the CSS, I’m making sure not to display the buffer canvas and the video element and to show the background image in the first canvas element.
The Real Thing – The JavaScript Code
Here is the implementation of the JavaScript file that will help you do the job:
var video;
var ctx1;
var ctx2;
var width;
var height;
function timerCallback() {
if (video.paused || video.ended) {
return;
}
generateFrame();
setTimeout(function () {
timerCallback();
}, 0);
}
function handlePageLoad() {
video = document.getElementById("video");
var c1 = document.getElementById("c1");
ctx1 = this.c1.getContext("2d");
var c2 = document.getElementById("c2");
ctx2 = this.c2.getContext("2d");
video.addEventListener("play", function () {
width = this.videoWidth;
height = this.videoHeight;
timerCallback();
}, false);
}
function generateFrame() {
ctx2.drawImage(video, 0, 0, width, height);
var frame = ctx2.getImageData(0, 0, width, height);
var len = frame.data.length / 4;
for (var i = 0; i < len; i++) {
var r = frame.data[i * 4 + 0];
var g = frame.data[i * 4 + 1];
var b = frame.data[i * 4 + 2];
if (g < 10 && r < 10 && b < 10) {
frame.data[i * 4 + 3] = 0;
}
}
ctx1.putImageData(frame, 0, 0);
return;
}
Let's deep dive into the things that the functions are doing:
- The
handlePageLoad function is responsible to initialize all the canvas contexts and to add a listener to the play event of the video. When it occurs, it will start the timerCallback function. - The
timerCallback function is responsible to stop the generation of frames when the video stops and to generate the current frame using the generateFrame function. - The
generateFrame function is the function that does the heavy lifting of drawing the transparent video on the canvas elements. It draws the video frame to the buffer canvas and then extracts that frame. After extracting the frame, it passes on every pixel and checks whether it is black. If so, it sets its opacity to transparent.
The effect will look like:
Summary
In the post, I showed you how to use a canvas in order to make changes to a running video on the fly. This is only an experiment that I made to show that it is possible to do that. The performance of this implementation isn’t so good and I would have thought about other solutions such as a video that includes the background image and the movie.
