Programming for Multi Touch

• Download source code - 4.2 KB
• Download executable file - 11.6 KB
 

 

Watch this YouTube video to see what this program does...

http://youtu.be/p-kPh_W6Hhc 

Introduction  

I have been working on a custom control for ages now - one which, for a long time now, I have known could benefit from having a multi touch interface. Experimenting with multi touch, however, seemed like a dream for the distant future until about a week ago when I was able to get my first touch screen laptop computer (a Toshiba C55T-10K - on a substantial discount!).

So - I have started looking at implementing multi touch for my custom control and searching for ways I could do this and the first real progress I have made is based on the MSDN article at...

http://msdn.microsoft.com/en-us/library/windows/desktop/dd744775%28v=vs.85%29.aspx

I guess there is educational value in these articles being a bit rough around the edges. Guys like me come and want to find out how they work. We open up a project and copy and paste the code in. The code works and demonstrates what it is supposed to but there are a few more things to do to it before it is ready to used in another project. So we play with it and worry at it until we find out what can be changed to make it more satisfying and/or usable.

It was quite hard for me going through the article and applying the fixes Tom1omT and duggulous had given and then finding one or two other things that still needed fixing. My aim, therefore, in writing up this commentary (that is quite closely based on the MSDN article) is to hopefully provide a smoother start for other people wanting  to get going with multi touch.  

Once I got the MSDN article code to run I made changes to it to improve its stability by removing memory leaks and making changes to the way touch point information is stored. Of particular interest to me, since I am hoping to implement multi touch functionality in my own custom control at some point in the future, was to find a clear and reliable way to achieve statefulness of the touch points. In the first edition of this article, the sample application would change the colour of a circle held on the screen depending on what was happening at other touch points. For the second edition of the article I have changed the code so that touch points being held on the screen will keep their state no matter what is happening at other touch points.

Using the code  

To get going make a Win 32 project. The MSDN article explains how to do this using the Visual Studio project wizard tool. The next task is to add some global variables and a global function to main .cpp file and add some code to the InitInstance and WndProc functions which will have been generated already by the wizard.

The first step described in the article is to put some lines into the project's targetver.h file. These check that things are all tickety boo regarding the software environment... 

#ifndef WINVER                  // Specifies that the minimum required platform is Windows 7.
#define WINVER 0x0601           // Change this to the appropriate value to target other versions of Windows.
#endif
 
#ifndef _WIN32_WINNT            // Specifies that the minimum required platform is Windows 7.
#define _WIN32_WINNT 0x0601     // Change this to the appropriate value to target other versions of Windows.
#endif     
 
#ifndef _WIN32_WINDOWS          // Specifies that the minimum required platform is Windows 98.
#define _WIN32_WINDOWS 0x0410   // Change this to the appropriate value to target Windows Me or later.
#endif
 
#ifndef _WIN32_IE               // Specifies that the minimum required platform is Internet Explorer 7.0.
#define _WIN32_IE 0x0700        // Change this to the appropriate value to target other versions of IE.
#endif      

 
I inserted these includes and declarations near the top of the main .cpp file... 

#include <windows.h>    // included for Windows Touch
#include <windowsx.h>   // included for point conversion

#define MAXPOINTS 10

// You can make the touch points larger
// by changing this radius value
static int radius      = 150;

//State information (ie colour of the circle)
// can be stored and retreived for the touch points.
struct circle{
	COLORREF colour;
	int sysID;
	int pointX;
	int pointY;
};

circle circlesArray[MAXPOINTS];

// For status reporting
int touchCount = 0;
int cycleCount = 0;

I've added in some global variables for status reporting purposes - touchCount and cycleCount. 

Other differences between my code and the MSDN code are:-

• Instead of using separate arrays for storing the system touch point ID idLookup and the two dimensional points arrays I have put all of this information plus the colour (that's British spelling in case your wondering) information into an array (circlesArray) of type circle. It seemed tidier to me to do it this way.  For the raw MSDN code (or as near as you can get to it) the statefulness stops once ten touch points have been placed on the screen. I give more details about how the statefulness is preserved later.
• Instead of prescribing the colours to be used in a array with ten fixed values I wrote a function to supply random colour values for the touch points. I plonked this function in near the top of the Multitouch.cpp file. One advantage of this is that it is that unlike the MSDN code there won't be a problem if the value of MAXPOINTS is increased. It was easy to do and the code is nice and concise:  

// This function makes a random colour value for the circle
COLORREF MakeColour(){
	return RGB(rand()%(255),rand()%(255),rand()%(255));
}

The original MSDN code has a function for returning a usable index for system identifiers for each of the touch points on the screen. This function (called GetContactIndex)  is able to return the appropriate index to the identifier storage array if the system identifier is stored in it or to allocate it to an empty member if both an empty member exists and the system identifier is not present in the array.

The problem with GetContactIndex is that it can't store new touch points after MAXPOINTS touch points have been allocated. Also, since I ultimately want to incorporate multi touch interfacing into a custom control I have been developing it was very important to me to be able to reliably retrieve state information associated with touch points that are re-used over and over again.  

To reliably retrieve the state information associated with the touch points (in this example that state information is the colour of the circles drawn on the screen) I ...

• replaced the data structures used in the original code with circlesArray (as described above).
• replaced function GetContactIndex with GetCircleIndex.
• enabled re-use of application memory which in turn facilitated infinite touch point re-use by writing a function to free up circlesArray memory - ReleaseCircleIndex.

Here are the new functions - GetCircleIndex and  ReleaseCircleIndex. Again, I placed them near the top of the .cpp file...  

// This function is used to return an index given an ID
int GetCircleIndex(int dwID){	
	for (int i=0; i < MAXPOINTS; i++){
	if (circlesArray[i].sysID == dwID){
			return i;
		}
	}

	for (int i=0; i < MAXPOINTS; i++){
		if (circlesArray[i].sysID == -1){
				circlesArray[i].sysID = dwID;
				circlesArray[i].colour = MakeColour();
				return i;
		}
	}	
	
	// Out of contacts
	return -1;

}


// This function is used to release an array member given an ID
void ReleaseCircleIndex(int dwID){

	for (int i=0; i < MAXPOINTS; i++){
		if (circlesArray[i].sysID == dwID){
			circlesArray[i].sysID = -1;
			circlesArray[i].pointX = -1;
			circlesArray[i].pointY = -1;
		}
	}

	//For aesthetics, these next lines will shuffle the vacant
	//array slot to the end of the array. This means that, for 
	//overlapping circles on the screen, the last touch point
	//made will cause a circle to be drawn on top of circles
	//representing already existing touch points.
	//It might be important to you to know the order in which
	//your touch points were placed. If not, this block can be
	//taken out.
	for (int i=0; i < MAXPOINTS; i++){
		if (i<MAXPOINTS-1 && circlesArray[i].sysID == -1){
			circlesArray[i] = circlesArray[i+1];
			circlesArray[i+1].sysID = -1;
			circlesArray[i+1].pointX = -1;
			circlesArray[i+1].pointY = -1;
			}
	}

}

 

Initialising the application

The InitInstance function in my code looks like this...  

BOOL InitInstance(HINSTANCE hInstance, int nCmdShow)
{
   HWND hWnd;
 
   hInst = hInstance; // Store instance handle in our global variable

   hWnd = CreateWindow(szWindowClass, szTitle, WS_OVERLAPPEDWINDOW,
      CW_USEDEFAULT, 0, CW_USEDEFAULT, 0, NULL, NULL, hInstance, NULL);
 
   if (!hWnd) {
      return FALSE;
   }
 
   // register the window for touch instead of gestures
   RegisterTouchWindow(hWnd, 0);  
 
   for (int i=0; i < MAXPOINTS; i++){
	circlesArray[i].sysID = -1;
	circlesArray[i].colour = RGB(0,0,0);
	circlesArray[i].pointX = -1;
	circlesArray[i].pointY = -1;
   }

   ShowWindow(hWnd, nCmdShow);
   UpdateWindow(hWnd);
 
   return TRUE;
} 

The salient parts being the call to RegisterTouchWindow and  block initialising all values in circlesArray.

All the other code we need to add goes into the WndProc function. This callback function acts on various windows messages that are passed to it. The sections we are going to add code to are those dealing with WM_TOUCH and WM_PAINT.

 

Preparing the WndProc function 

I put these declarations in at the start of the WndProc function... 

    // For double buffering
    static HDC memDC       = 0;
    static HBITMAP hMemBmp = 0;
    HBITMAP hOldBmp        = 0;


    // For tracking dwId to points
    int index;


    // For dealing with WM_TOUCH in WndProc
    int i, x, y;
    UINT cInputs;
    PTOUCHINPUT pInputs;
    POINT ptInput;   

 

Adding a handler for WM_TOUCH:   

You will need to add the section for WM_TOUCH. Add this block of code to the switch (message) block ...

		
	case WM_TOUCH:        
		cInputs = LOWORD(wParam);
		pInputs = new TOUCHINPUT[cInputs];

		cycleCount++;
		if (pInputs){
			if (GetTouchInputInfo((HTOUCHINPUT)lParam, cInputs, pInputs, sizeof(TOUCHINPUT))){
			  for (int i=0; i < static_cast<int>(cInputs); i++){
				
				TOUCHINPUT ti = pInputs[i];
      
				if (ti.dwID != 0){                            
					// Do something with your touch input handle
					ptInput.x = TOUCH_COORD_TO_PIXEL(ti.x);
					ptInput.y = TOUCH_COORD_TO_PIXEL(ti.y);
					ScreenToClient(hWnd, &ptInput);
					  
					if (ti.dwFlags & TOUCHEVENTF_UP){      

						//ReleaseContactIndex(ti.dwID);     
						ReleaseCircleIndex(ti.dwID);
						
						touchCount++;
					}else{ 
						//If the touch point already exists
                                                // - ie. someone has held their finger
						//on the screen or dragged it without 
                                                //lifting it off then GetCircleIndex
						//will return a value by which we can 
                                                //retrieve information pertaining to 
						//that touch point from circlesArray.
						//If the touch point doesn't exist then an 
                                                //array location will be allocated
						//to it, a colour created and stored along 
                                                //with its position on the screen
						//and that array location is returned.
						index = GetCircleIndex(ti.dwID); 

						circlesArray[index].pointX = ptInput.x;
						circlesArray[index].pointY = ptInput.y;
					}
				}
			
				if (!CloseTouchInputHandle((HTOUCHINPUT)lParam))
				{
					/*
					// error handling
					MessageBox(
						NULL,
						(LPCWSTR)L"!CloseTouchInputHandle",
						(LPCWSTR)L"Error",
						MB_ICONWARNING | MB_DEFBUTTON2
					);
					*/
				}
				
			  }

			}
			// If you handled the message and don't want anything else done with it, you can close it
			CloseTouchInputHandle((HTOUCHINPUT)lParam);


			delete [] pInputs;
		}else{
			// Handle the error here 
		} 
	
		
		
		InvalidateRect(hWnd, NULL, FALSE);
		break;

Note the last line but one where InvalidateRect is called. This appears to be essential for enabling the drawing area  to work in the way that this application requires it to work. The line is missing from the code in the original article. Thanks to duggulous for his comment on the MSDN article for pointing out that this is required. 

 

Adding a handler for WM_PAINT:   

Next add code to WndProc for handling WM_PAINT for drawing the circles. The section should end up looking something like this:

	

	case WM_PAINT:

		hdc = BeginPaint(hWnd, &ps);
			// TODO: Add any drawing code here...

		RECT client;
		GetClientRect(hWnd, &client);        
  
		// start double buffering
		if (!memDC){
		  memDC = CreateCompatibleDC(hdc);
		}
		hMemBmp = CreateCompatibleBitmap(hdc, client.right, client.bottom);
		hOldBmp = (HBITMAP)SelectObject(memDC, hMemBmp);          

		//This conditional provides a convenient block
		//within which backgroundBrush can be created and destroyed
		if (memDC){

			//A brush to create background is generated once
			//and destroyed once every time this function is called
			HBRUSH backgroundBrush = CreateSolidBrush(RGB(0,0,0));

			FillRect(memDC, &client, backgroundBrush);
     
			//Draw Touched Points                
			for (i=0; i < MAXPOINTS; i++){        
		
			
				//I added this block to monitor the touch point IDs on screen
				TCHAR buffer[180];
				_stprintf_s(buffer, 180, _T(
					"cc %d tc %d idl: %d %d %d %d %d %d %d %d %d %d                                                                       "),
					cycleCount, touchCount,
					circlesArray[0].sysID,
					circlesArray[1].sysID,
					circlesArray[2].sysID,
					circlesArray[3].sysID,
					circlesArray[4].sysID,
					circlesArray[5].sysID,
					circlesArray[6].sysID,
					circlesArray[7].sysID,
					circlesArray[8].sysID,
					circlesArray[9].sysID
				);
				RECT rect = {0,0,800,20};
				DrawText(memDC,buffer,100,(LPRECT)&rect,DT_TOP);		
			
				HBRUSH circleBrush = CreateSolidBrush(circlesArray[i].colour);
				SelectObject( memDC, circleBrush);    

				x = circlesArray[i].pointX;
				y = circlesArray[i].pointY;

				if  (x >0 && y>0){              
					Ellipse(memDC, x - radius, y - radius, x+ radius, y + radius);
				}
				ReleaseDC(hWnd, memDC);
				DeleteObject(circleBrush);
			}
  
			BitBlt(hdc, 0,0, client.right, client.bottom, memDC, 0,0, SRCCOPY);      
			DeleteObject(backgroundBrush);
		}

		EndPaint(hWnd, &ps);
	
		ReleaseDC(hWnd, hdc);
		DeleteObject(hMemBmp);
		DeleteObject(hOldBmp);
    break;

 

Notes on memory handling   

Note the CreateSolidBrush call and the assignment of the resulting HBRUSH to backgroundBrush. In my version of the code there are two places where I use this function and allocate the returned objects to variables (backgroundBrush and circleBrush).  Because I allocated the returned objects to variables I was able to delete those objects later in the code and prevent memory leaks.  This is how the background was drawn in the original MSDN code...

//This call to CreateSolidBrush causes a memory leak
//because there is no where where the created brush 
//object can be deleted!
FillRect(memDC, &client, CreateSolidBrush(RGB(255,255,255)));
   

In a comment on the MSDN article duggulous showed how memory leaks could be prevented by saving the returned object to a variable and deleting it later. Perhaps there is a way of tidying up after sending the object directly to FillRect as per the MSDN code but the article didn't say what it was. Duggulous' method seems to work so that is what I have used.  

As mentioned above, I have prevented memory leaks by assigning objects created by CreateSolidBrush to variable locations from where they can be deleted later. circleBrush is created and destroyed in every cycle of the for block which iterates through each item in circlesArray and backgroundBrush is created and destroyed just once each time WM_PAINT is handled. My compiler complained about the declaration of backgroundBrush being exposed within a case segment of a switch block. To get round it I contrived a cosy if(memDC){} block for it to live and die in and the compiler was perfectly happy with that.  

After reading the comments by duggulous and Tom1omT on the MSDN code and studying the article at http://www.winprog.org/tutorial/bitmaps.html it seemed wise to include the following lines.

	ReleaseDC(hWnd, hdc);
	DeleteObject(hMemBmp);
	DeleteObject(hOldBmp); 

The article describes common ways of creating a HDC and appropriate ways to then destroy them. Apparently, the appropriate way to delete memDC (again, a detail not dealt with in the MSDN code) is to call DeleteDC on it. I tried this in various places but couldn't get the code to behave with it so eventually I just left one call to when WM_DESTROY is handled.

	case WM_DESTROY:
		//next line gleaned from http://www.winprog.org/tutorial/bitmaps.html
		DeleteDC(memDC);
		PostQuitMessage(0);
		break;

Points of Interest   

The MSDN code is meant as a proof of concept. Ie. it is purely meant to show you how the multitouch interface works and doesn't worry too much about memory handling or long term stability of the code.  It has been interesting and fun getting the code going. The multi touch interface is going to be very useful and thanks to this article I will be able to make good of use of it. Thanks also to Tom1omT and duggulous for their vital comments!

I  have tried to leave a comment on the MSDN article to explain what I have done with the code but have been unsuccessful to date.

History  

1st edition 14th October, 2013.

2nd edition 28th October, 2013.  

• Repaired one more memory leak (by using and deleting backgroundBrush instead of sending CreateSolidBrush output straight to FillRect.
• Consolodated touch point data into circle structure and circlesArray.
• Ensured prolonged statefulness of information associated with the touch points (ie. that correct information was being returned from circlesArray for system touch point IDs.
• Assigned random colours to the circles.