Fun with HTML5 Canvas, WebSocket, JQuery and ASP.NET. End-result: A live white board on a web page!

• Download source - 1.41 MB

Introduction

What is the best communication medium for a team discussion?

Of course, it's the classic white board, with a simple chalk and a duster.

Yeah, that's true. If a few members of your team need to discuss something, and if you all can physically gather in the same place, a white board is the best way to express and share ideas among yourselves. But, this is an era of business outsourcing, and, when you need to discuss something with your stakeholders, one of you could reside in Washington, another in Zurich, and you could reside in Dhaka. No white board is there to help you to discuss something among you people while you all stay in remote places, right?

Well, it was true, until now. I have developed a very simple white board on a web page which could be used to draw anything using the mouse or write whatever using the keyboard to express your statements to other team mates, no matter how far they stay physically. As long as they all are participants into the same white board session, they all can see what you draw or write on the white board right after you are done, and for that they don’t have to refresh their pages. The moment you draw something on the white board , or, write something on the chat window, all other persons are able to see it instantly. Similarly, you are also able to see their instant response when any other participant draws a figure on the white board or writes something on the discussion window. Yes, we are talking about a live white board on a web page!

Imagine how the white board would be used to demonstrate the idea of Pyramid in ancient Egypt!

Here is how the Egyptian King Pharaoh would use the white board to demonstrate the idea of Pyramid to his architect Fukayna.

Fukayna and Pharaoh both logs on to the white board by using their HTML 5 supported browsers and says hello to each other (Assuming they no longer follow the long salutations to honour the kings in this web 2.0 arena).

Fukayna’s White board

Pharaoh’s white board

Pharaoh says: "I want to build a Pyramid!"

Figure: White board demonstration

Fukayna says: "Your excellency, how would be this Pyramid?"

Pharaoh says: "Pyramid is something like this"

Fukayna says : "Hmm..if you pardon me, would this be something like the following if see from the sky?"

Pharaoh says : "Yes, exactly"

You see, how easy it was to demonstrate an idea via the white board? A little drawing helped understanding the idea with a minimal conversation. Pictures really can tell a thousand story!

Running the white board Application

Follow these steps to run the white board application: 

• Download the whiteBoard.zip (Download link at the top of this article) and extract it at a convenient location.
• Create an ASP.NET 4.0 Web site (say, http://whiteboard) in IIS pointing to the web application folder “whiteboard ” within the extracted folder.
• Put an entry in the hosts file so that, the web site name “whiteboard ” points to the local IP.

whiteboard 127.0.0.1

• Browse the white board application by browsing the URL http://whiteboard/whiteboard.aspx using two different browsers windows using one of the following HTML5 supported browsers:
  • IE9
  • Google chrome 
  • Safari 5.1
• Identify yourself by providing your name in the text box and pressing “OK”. There you go!

You can perform the following things using the white board:

• Draw something on the white board by dragging the mouse (If the Pencil mode is selected by clicking on the icon below the white board, which is selected by default).
• Erase drawing on the white board by clicking on the Pencil icon, selecting the Eraser mode (The Eraser icon is being displayed in such situation) and dragging the mouse on the white board.
• Clear the entire white board by clicking on the “Clear white board ” link, which is available below the white board.
• Send/receive message to/from other participants using the conversation window.

Perform the following steps to see the white board in action: Make sure you have at least two HTML5 supported browsers installed. (I installed the latest version Google Chrome and Safari 5.1).

• Browse WhiteBoard.aspx using both browsers at the same time and log onto the white board using two different user names (No need to register).
• Send a message to the other user using the conversation/chat window from each of the browsers. You will see the message appear immediately into the other user’s conversation window.
• In one browser window, drag the mouse onto the white board to draw something. Once drawn, open the other browser window. Note that, whatever you just drew on the first browser’s white board, gets drawn automatically onto the second browser’s white board (without any page refresh). Do the same onto the second browser’s white board and see the same drawing is shown automatically onto the first browser’s white board.
• Click onto the Pencil icon (just below the white board) to switch to eraser mode and erase some drawing onto the white board by dragging the mouse onto the white board. Note that the same portion of the drawing is automatically erased from the other browser’s white board.
• Clear the white board by clicking onto the “Clear board” link just below the white board. Note that, the white board at the other browser window is being cleared automatically.

Technology (And a Bit Programming) Made it Possible

HTML5 is the core technology which made it possible to draw something on a web page (The Canvas object) and broadcast the drawing information or a messages to other participants (via WebSocket). Along with the HTML5 stuff, a bit of JQuery is used to do some quick client side programming, and, a WebSocket server (A slightly modified version of SuperWebSocket server, thanks to its author) is used to implement the server side functionality of WebSocket. Let's not forget that an ASP.NET web application is used to host the entire functionality, which I love the most. :)

WebSocket

Have you ever thought how chatting is implemented in Gmail?

You might already know about the fact that normally a web server is not able to send response to the browser without an HttpRequest initiated from that browser. Now, how do you think a chat message arrives at your chat window from your chatting partner? Are the messages being polled from the server in small time interval? Or, does the server application broadcast the messages to you in some particular mechanism?

Polling may not be a good choice for the following reasons:

• Polling at a regular interval would put huge and continuous loads on the server as each user session would initiate frequent poll requests at small time interval.
• The conversations would not seem natural and real time as the next message from the partner would not arrive until the next poll occurs, even if partner may already posted his or her message. So, your partner would also not get a timely response from you.

So, it has to be a server application broadcasting messages to the connected participants (Browsers) on a particular chat session. Interestingly, this is possible to implement using some really smart programming, and, this technique is often referred as HTTP Push or Comet.

Unfortunately, implementing a stable and full-proof HTTP Push or Comet engine is not that easy. There are challenges like firewalls and proxy server related issues, and, JavaScript is not able to communicate via Sockets. Also, HTTP connections are limited (Only two connections per domain in some browsers), and, one connection has to be remained open for accepting broadcasted messages from server, which is not an easy thing to deal with.

So, there was a need for something new, something simple which could help implement the server broadcasting needs in an easy manner. Thanks to HTML 5. It offers WebSocket, which is a new protocol that runs over TCP, supports fully duplex communication (Unlike HTTP, which is a simplex protocol) and which made it extremely easy to send or receive messages to or from a WebSocket server. The WebSocket API is being standardized by the W3C.

Unfortunately, a full fledged WebSocket server is not yet available, and hence, I had to rely on a WebSocket server which is built upon an open source contribution SuperWebSocket (http://superwebsocket.codeplex.com/) to build the white board.

Following is the basic WebSocket functionality:

//Initialize WebSocket
function connect()
{
   try
   {
       var socket;
       var host = "ws://localhost:8000/WebSocket/Default.aspx";

       var socket = new WebSocket(host);

       message('Socket Status: '+socket.readyState);

       socket.onopen = function(){
           message('Socket Status: '+socket.readyState+' (open)');
       }

       socket.onmessage = function(msg){
           message('Received: '+msg.data);
       }

       socket.onclose = function(){
           message('Socket Status: '+socket.readyState+' (Closed)');
       }         
    } 
    catch(exception)
    {
       message('Error'+exception);
    }
} 

//Send Message to WebSocket Server
function send(message)
{
    try
    {
        socket.send(message);
    }
    catch(exception)
    {
       message('Error:' + exception);
    }
}

HTML5 Canvas

The HTML5 canvas element could be used by JavaScript to draw graphics on a web page. A canvas is a rectangular area, and you can control every pixel of it.

You could draw different shapes on a canvas (Like Path, Box, Circle, etc.) by using it’s Javascript API.

Following is the basic use of HTML Canvas:

<!DOCTYPE HTML>
<html>
<body>
    <canvas id="myCanvas" width="200" height="100" style="border: 1px solid #c3c3c3;">
Your browser does not support the canvas element.
</canvas>
    <script type="text/javascript">
        var c = document.getElementById("myCanvas");
        var cxt = c.getContext("2d");
        //Go to x=20, y=20 position

        cxt.moveTo(20, 20);
 
        //Draw a line from point1 (x=20,y=20) to point2 (x=150,y=50)  
        cxt.lineTo(150, 50);
        //Renders the actual line
        cxt.stroke();
    </script>
</body>
</html>

It renders the following output in the browser:

Pretty nice and simple, right?

Drawing Continuous Lines using Mouse

Drawing on a white board requires drawing continuous and irregular lines on the canvas. So, while user drags the mouse on the canvas, the continuous line has to be drawn by drawing smaller lines between the points on the mouse trail. Basically, the following logic has to be used (utilizing the help of JQuery)::

$("#drawing-canvus").mousedown(function () {
    startDraw = start;
    context.beginPath();
});

$("#drawing-canvus").mousemove(function (e) {
    context.lineTo((e.clientX - position.left), (e.clientY - position.top));
    context.stroke();                 
});

$("#drawing-canvus").mouseup(function () {
    startDraw = false;
});

See \scripts\jqdraw.js to learn how the drawing functionality is implemented.

Sending Drawing Information to WebSocket Server

When user draws something on the white board, the drawing occurs simultaneously in all of the browsers which are currently connected to the same white board session. This operation is carried out in the following sequence of operations:

• While user draws with the mouse (by dragging), the co-ordinates (x,y) of the points in the mouse trail are collected in a JavaScript variable and once user releases the mouse, the co-ordinate information is send to the socket server.
• The socket server collects the co-ordinates and broadcasts the same co-ordinate information to the connected browsers.
• The scripts at the browsers collect the message (Co-ordinates) from the socket server and draw the points on their corresponding white board (canvas).

The logic is captured in the following code:

$("#drawing-canvus").mousedown(function () {
    startDraw = true;
    //Start building the co-ordinate string

    coOordinates = "" + isInk;
    context.beginPath();
});

$("#drawing-canvus").mousemove(function (e) {
    if (e.target.getAttribute("id") == "drawing-canvus") {
        if (startDraw) {
        context.lineTo((e.clientX - position.left), (e.clientY - position.top));

        //Add points to the co-ordinate string as long as user drags the mouse

        coOordinates += "#" + (e.clientX - position.left) + "," + 
					(e.clientY - position.top);
                               context.stroke();
    }         
 }); 

$("#drawing-canvus").mouseup(function () {
    startDraw = false;
    $("#drawing-canvus").trigger("drawmultiple", coOordinates );
});

Overcoming a Limitation of WebSocket

When user drags the mouse over the canvas, the mouse trail co-ordinate gets produced like the following:

1#274,162#277,164#280,167#285,171#286,173#288,173#290,175#292,176#298,176#305,183#309,189
 #315,197#317,201#319,201#323,206#328,211#331,214#332,215#334,216#338,223#342,229#346,233
 #351,238#354,241#354,244#357,248#359,252#362,256#363,257#365,258#367,261#372,266#375,268
 #377,268#380,274#383,277#386,281#386,282#386,283#388,284#390,284#390,285#392,287#393,289
 #394,290#396,293#397,296#399,299#402,307#403,307#404,310#406,313#406,314#406,318#407,323
 #407,325#408,326#408,327#410,331#412,336#414,345#416,350#418,358#419,360#420,366#422,374
 #422,376#423,380#424,382#424,384#424,385#424,386#424,388#422,391#420,395#419,397#418,397
 #417,399#416,399#415,401#413,401#410,403#407,406#404,411#397,415#395,419#390,420#386,421
 #383,422#382,423#381,423#381,424#379,425#369,425#361,428#357,429#350,431#345,434#334,435
 #325,436#318,436#305,436#296,432#289,431#284,430#279,429#272,427#263,422#258,420#247,416
 #235,410#228,404#221,400#213,395#207,391#199,384#189,377#183,371#180,369#175,366#174,365
 #170,358#166,348#160,342#157,334#154,327#154,320#154,315#155,306#153,298#152,290#153,273
 #156,261#161,244#161,243#163,238#164,233#167,231#169,222#171,214#173,211#175,209#178,199
 #179,198#185,191#187,185#189,183#189,182#190,180#191,180

The first number (1 in the above string) indicates the current mode of operations, which could either be 0 or 1. 0 indicates Eraser mode and 1 indicates Pencil mode.

The later numbers (say, 274,162) separated by hashes (#) indicates the co-ordinate points on the drawing.

The co-ordinate message usually gets large when the user draws a long line or draws the line slowly and in such cases, the white board fails to send the co-ordinate message to the Socket server due to the size limitation of the WebSocket protocol (I am not sure exactly how much size restriction is there for the WebSocket message, but, there is a size limitation for sure). As a result, if user draws something long which could results in generating a long co-ordinate message, white board at the other connected browsers are not updated with what the user draws at a particular browser’s white board .

To get around this, the co-ordinate message is being chunked in pieces internally and each chunk is sent to the websocket server and the server broadcasts each chunk of the co-ordinate messages to each connected browser to draw each particular piece of drawing on their corresponding canvas to render the overall drawing.

Implementing Some Other Utility Functionality

Just below the white board, there are a few controls which let a user to switch between the Pencil and Eraser mode and to clear the entire white board. The white board also shows user activity when user does something (draws on the white board or types something in the conversation/chat window) and this is shown simultaneously across all browsers connected to the same session.

Erasing Drawing

Like the actual white board, the user can erase the drawing or part of a drawing from the white board by selecting the Eraser mode. By default, the Pencil mode is selected (the currently selected mode is shown using the Pencil and Eraser icon) and hence when user drags the mouse on the white board, something is drawn on it. If user wants to erase some portion of his/her drawing, he/she needs to select the Eraser and it is as easy as clicking on the Pencil icon to switch to the Eraser mode. Once Eraser mode is selected, dragging onto the white board would result in erasing the portion of the drawing on the mouse pointer’s trail, and, all of the browsers participating at the same white board session erase the same portion of the drawing from their white board.

Erasing is done using the same technique which is being used to draw something on the white board, except the fact that, when Eraser mode is selected, the drawing color is switched to white (which is by default set to black when Pencil is selected).

Clearing the White Board

User can clear the entire white board by clicking on the “Clear Board” link. Clicking on this link executes the following script to clear the entire white board.

function clearCanvas() {
    canvas=document.getElementById("drawing-canvus");
    c=canvas.getContext("2d");
    c.clearRect(0,0,canvas.width,canvas.height);
}

Like the drawing and erasing, when user clears the white board in one particular browser window, the white board is cleared simultaneously in all browser windows participating in the same session.

Showing User Activity Info

Whenever user draws something onto the white board , or, types something onto the conversation/chat window, an info message is being shown just below the white board. Following messages are shown:

“User is drawing something...”
“User is erasing something...”
“User cleared the white board ”

Similar to the above functionality, the user activity info is being displayed in all browsers simultaneously. So, when User1 draws something, the following message is displayed in all browsers participating in the same session:

“User1 is drawing something...”

Sending and Receiving Chat Message

Like Gmail web chat, users participating in the same session can send or receive text messages using the conversation window, which is residing just beside the white board. When a user sends a message using the conversation window, the message is broadcasted to all participating users and they immediately see the message in their conversation window at the same time.

Whatever a user does onto the white board, the corresponding activity is immediately reflected to all browser’s white board. This is implemented using the following logic:

• Whatever the user does, a corresponding JSON string is built containing the necessary information and the JSON message is sent to the WebSocket server.
• The WebSocket server receives the JSON message and broadcasts it to all connected browsers.
• Browsers receives the JSON message, parses it to retrieve data and renders the corresponding output in the white board page.

The following method is used to build the JSON message formats for each different activity:

function getJsonMessage(action, message) {
    var jsonMessage = '{"Action":"' + action + '","Message":"' + message + '"}';
    return jsonMessage;
}

And, following is how the corresponding JSON messages are sent to the WebSocket server for each different user activity on white board:

function sendPixelMessage(message) {
    var jsonMessage = getJsonMessage("Pixel", message);
    ws.send(jsonMessage);
}

function sendChatMessage(textArea) {
    var message = $(textArea).val();
    if (message == "") return false;

    $(textArea).val("");
    var jsonMessage = getJsonMessage("Chat", message);
    ws.send(jsonMessage);
}

function sendInfoMessage(message) {
    var jsonMessage = getJsonMessage("Info", message);
     ws.send(jsonMessage);
}

function sendClearMessage() {
    var jsonMessage = getJsonMessage("Clear", "");
     ws.send(jsonMessage);
}

Showing the Latest White Board Status for a New User

Whenever a new user joins the session, he/she should be presented with the latest status of the white board. There may be scenarios when one user may join the white board later and in such case, he/she should see what has already been drawn or erased onto the white board, and also what conversation messages have been exchanged between the other participants. This is accomplished using the following logic:

• All of the JSON messages (which are sent from the browsers via WebSocket) at the SocketServer are stored into a global variable, before broadcasting the message to the connected browsers. This is accomplished using the following method at the WebSocket server:

  void socketServer_CommandHandler(WebSocketSession session, 
  	WebSocketCommandInfo commandInfo)
  {
      lock (m_SessionSyncRoot)
      {
          string messageType = GetTypeFromMessage(commandInfo.Data);
  
          string message = buildJSONMessage
  	(GetDataFromMessage(commandInfo.Data), messageType, session);
  
          List<string> messages = ApplicationData.Data as List<string>;
  
          if (messages == null)
          {
              messages = new List<string>();
  
              ApplicationData.Data = messages;
          }
  
          messages.Add(message);
  
          ApplicationData.Data = messages;
  
          SendToAll(message);
      }
  }  

• When a new user joins the white board, he/she is presented with the latest situation of the white board by rendering the drawing onto the white board and displaying the user activity (chat messages) onto the conversation window.

  This is accomplished using the following piece of codes in the WhiteBoard.aspx:

  function LoadPreviousActivity()
  {
      <%
      if(SuperWebSocket.ApplicationData.Data != null)
      {
          List<string> history = SuperWebSocket.ApplicationData.Data as List<string>;
  
          if (history != null)
          {
              foreach (string activity in history)
              {
                  %>
                      AppendActivity('<%=activity %>');
                  <%
              }
          }
      }
      %>
  } 

  One limitation of the WebSocket server is, unlike the ASP.NET, it doesn’t have any state management system like the ApplicationState or SessionState. So, there could be two alternatives to solve this issue.

• While sending a message to WebSocket, send the same message to ASP.NET using Ajax and store the message in an Application state or Cache and retrieve those message later from ASP.NET.
• Implement some custom storage mechanism at the WebSocket server and store messages there. I followed the later approach just because of simplicity. I used a static property of a public class (SuperWebSocket.ApplicationData.Data) to store all user messages and parse these all messages to display the latest status of the white board to a user who joins the white board session later.

Note: This server side storage mechanism implemented here is not really the best approach and it is done just for demonstration purposes. In reality, a full-fledged server-side storage mechanism has to be developed and I expect to see this important thing to be implemented sooner or later in the WebSocket servers.

Summary

Well, an obvious question that may be asked is “Was this white board fun? or is this something real that could be used for real production systems?”

I think, WebSocket is still in its early days, and, to be true, not all browsers support WebSocket (Or Html5) yet. So yes, this is actually a fun implementation, but, this demonstrates the capability and the potentials of the WebSocket and indicates its bright future.

HTML5 rocks (So does WhiteBoard)! 

History

• 12^th March, 2011: Initial post