The Windows Mobile 6.5 Gestures API: An Introduction

• Download source - 5.01 KB
• Download demo - 5.03 KB

Introduction

This article describes a simple application for Windows Mobile 6.5 Professional Edition which is a useful exercise for familiarizing ourselves with the Gestures API that is newly available in Windows Mobile 6.5.

Background

Readers of this article should be somewhat familiar with the basics of Win32 GUI development as well as have a beginner to intermediate familiarity with C/C++. If you do not know what HWND, Window Procedure, or MSG are, then I highly recommend reading through "A Generic Sample Application" found at MSDN under Windows API or better yet, if you can obtain a copy of Programming Windows by Charles Petzold, a good understanding of the first section, labeled "Section I: The Basics" should be sufficient for this exercise. Additionally, you will need Visual Studio 2005 or 2008 Professional Edition, and the Windows Mobile 6.0 and 6.5 SDKs (Professional Edition) installed on your machine in order to build and execute the code. Some familiarity with the Visual Studio environment (creating projects, navigating, adding files, etc...) is also assumed.

Using the Code

First we need to set up our environment. We will be building a simple application and will need to include some files from the Windows Mobile SDK. Open Visual Studio (I am using Visual Studio 2008 Professional, but 2005 Professional should also be fine) and create a new project. Select Visual C++ for the language type, and then select the "Smart Device" project type and the "Win32 Smart Device Project" template.

Give your project a name such as "GesturesTest" or whatever makes you happy.

Click "OK and the Win32 Smart Device Project Wizard will launch. Hit "next" on the "Overview" page. On the platforms page, we need to select the proper SDK that we will be using to develop our application. The Gestures API is only available in Windows Mobile 6.5 Professional, which requires the Windows Mobile 6 Professional SDK (note that the Windows Mobile 6.5 Professional SDK must be installed on your machine, however when selecting our platform, we must use the Windows Mobile 6 Professional SDK). Hit "Next" to continue.

Finally on the "Application Settings" page, for Application type, select "Windows Application". Also, for the sake of keeping this project simple, check the box that reads "Empty project" and make sure the ATL box is unchecked. Click "Finish" to complete the Smart Device Project Wizard.

Now that we have a ready project, we need to add a code file so we can begin writing our application. Right-click on "Source Files" and add a new .cpp file to our project. We also need to add some critical library references to our project. Right click on the project icon and select "Properties". Under "Configuration Properties", "C/C++", "General" add the "Include" directory from the Windows Mobile 6 Professional SDK to the "Additional Include Directories" property. The default path to this is typically "%Program Files%\Windows Mobile 6 SDK\PocketPC\Include\Armv4i".

Now, under "Linker", we need to add the necessary lib files. First, under "Linker", "General", "Additional Library Directories", we need to add the Windows Mobile 6 SDK Lib files directory. The default path to this is "%Program Files%\Windows Mobile 6 SDK\PocketPC\Lib\Armv4i".

Finally, we need to explicitly add two .lib files to the project. Under "Linker", "Input", "Additional Dependencies", add references to AygShell.lib and TouchGestureCore.lib. AygShell.lib contains some GUI API code we will be calling in this application. TouchGestureCore.lib contains the code required to execute the Gestures API.

Since we are building a native Windows App, we will need to start with our WinMain function. This article assumes some familiarity with topics such as creating a Windows GUI application using the Win32 API, so we will breeze through this part pretty quickly to get to the meat of this article, which is the Gestures API.

First, add our macro definitions and includes:

#define WIN32_LEAN_AND_MEAN

#include <windows.h>
#include <gesture.h>	// The header for the gestures API
#include <aygshell.h>

#define LABEL_WIDTH	240
#define LABEL_HEIGHT	125

Now add our function prototypes.

LRESULT CALLBACK WndProc(HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam);
LRESULT WINAPI OnCreate(HWND hwnd, WPARAM wParam, LPARAM lParam);
LRESULT WINAPI OnGesture(HWND hwnd, WPARAM wParam, LPARAM lParam);
LRESULT WINAPI OnColorStatic(HWND hwnd, WPARAM wParam, LPARAM lParam);
VOID WINAPI GetGestureDirection(WORD wDirection, OUT PTSTR szDirection, IN DWORD dwSize);
LRESULT WINAPI OnSize(HWND hwnd, WPARAM wParam, LPARAM lParam);

We also have a few global variables that we will be using in this application.

HINSTANCE g_hinst = NULL;
HWND g_hwndLabel = NULL;

And now our WinMain function:

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, 
	LPTSTR lpCmdLine, int nShowCmd)
{
	static TCHAR szAppName[] = TEXT("GesturesTest");

	HWND hwnd = NULL;
	MSG msg;
	WNDCLASS wndClass;

	ZeroMemory(&wndClass, sizeof(WNDCLASS));

	g_hinst = hInstance;

	wndClass.style = CS_HREDRAW | CS_VREDRAW;
	wndClass.lpfnWndProc = WndProc;
	wndClass.hInstance = hInstance;
	wndClass.hCursor = LoadCursor(NULL, IDC_ARROW);
	wndClass.hbrBackground = CreateSolidBrush(RGB(0, 0, 0));
	wndClass.lpszClassName = szAppName;

	hwnd = CreateWindow(szAppName,		// Window Class Name
			TEXT("Gestures Test"),	// Window Caption
			WS_VISIBLE,		// Window Style
			0,			// Initial X position
			0,			// Initial Y position
			CW_USEDEFAULT,		// Initial Width
			CW_USEDEFAULT,		// Initial Height
			NULL,			// Parent Window Handle
			NULL,			// Menu Handle
			g_hinst,			// Program Instance Handle
			NULL);			// Creation Parameters

	if (hwnd)
	{
		ShowWindow(hwnd, nShowCmd);
		UpdateWindow(hwnd);

		while (GetMessage(&msg, NULL, 0, 0))
		{
			TranslateMessage(&msg);
			DispatchMessage(&msg);
		}

		if (hwnd)
		{
			CloseHandle(hwnd);
		}

		UnregisterClass(szAppName, hInstance);

		return msg.wParam;
	}

	return GetLastError();
}

Next we need to create our Window Procedure. This is the function that matches the prototype:

LRESULT CALLBACK WndProc(HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam)
{
	switch(msg)
	{
	case WM_CREATE:
		return OnCreate(hwnd, wParam, lParam);
	case WM_GESTURE:
		return OnGesture(hwnd, wParam, lParam);
	case WM_SIZE:
		return OnSize(hwnd, wParam, lParam);
	case WM_CTLCOLORSTATIC:
		return OnColorStatic(hwnd, wParam, lParam);
	case WM_ACTIVATE:
		if (WA_INACTIVE == (DWORD)wParam)
		{
			return SendMessage(hwnd, WM_DESTROY, 0,0);
		}
		return DefWindowProc(hwnd, msg, wParam, lParam);
	case WM_DESTROY:
		if (g_hwndLabel)
		{
			CloseHandle(g_hwndLabel);
		}
		PostQuitMessage(0);
		return 0;
	default:
		return DefWindowProc(hwnd, msg, wParam, lParam);
	}
}

This should be fairly straightforward. We are capturing the various Window Messages that we are interested in for the purposes of this application. Note the WM_GESTURE case. When the application screen is touched, a WM_GESTURE message is generated and sent to the message loop. We'll talk about this more in a bit. You may also wonder what we are doing in the WM_ACTIVATE case. The default behavior for a Windows Mobile application is to remain live but inactive when the user hits the "close" button. We don't require that type of behavior for our little test app, so we are checking for WM_ACTIVATE messages with the WA_INACTIVE flag and simply closing our program when this message is received.

Our next function is the OnCreate function. This will handle any initial construction required for our application. In this case, we will be building a label to display our gesture data, and a (blank) menu bar to replace the thumb buttons.

LRESULT WINAPI OnCreate(HWND hwnd, WPARAM wParam, LPARAM lParam)
{
	if (!hwnd)
	{
		return ERROR_INVALID_PARAMETER;
	}

	RECT rect;
	GetClientRect(hwnd, &rect);

	int iLabelLeft = (rect.right >> 1) - (LABEL_WIDTH >> 1);
	int iLabelTop = (rect.bottom >> 1) - (LABEL_HEIGHT >> 1);

	// Create a Text Label and center it on screen
	g_hwndLabel = CreateWindowEx(0,
				 TEXT("STATIC"),
				 0,
				 WS_CHILD | WS_VISIBLE | SS_CENTER,
				 iLabelLeft,
				 iLabelTop,
				 LABEL_WIDTH,
				 LABEL_HEIGHT,
				 hwnd,
				 NULL,
				 g_hinst,
				 NULL);

	// Create an empty menu bar for our app
	SHMENUBARINFO mbi;
	ZeroMemory(&mbi, sizeof(SHMENUBARINFO));
	mbi.cbSize = sizeof(SHMENUBARINFO);
	mbi.hwndParent = hwnd;
	SHCreateMenuBar(&mbi);
	DrawMenuBar(hwnd);

	// Set the font
	LOGFONT lf;
	ZeroMemory(&lf, sizeof(LOGFONT));
	StringCchCopy(lf.lfFaceName, 32, TEXT("Courier New"));

	HFONT hFont = CreateFontIndirect(&lf);
	SendMessage(g_hwndLabel, WM_SETFONT, (WPARAM)hFont, 0);
	CloseHandle(hFont);

	return ERROR_SUCCESS;
}

Now let's build the OnColorStatic function. We'll be using this to set the colors for our label. Note that you can use any colors you like, but for my app I have chosen to use a plain black background and green text.

LRESULT WINAPI OnColorStatic(HWND hwnd, WPARAM wParam, LPARAM lParam)
{
	static HBRUSH hBrush = CreateSolidBrush(RGBA(0,0,0,0));

	HWND hwndStatic = (HWND)lParam;
	HDC dc = (HDC)wParam;

	SetTextColor(dc, RGB(100, 255, 100));
	SetBkColor(dc, RGBA(0, 0, 0, 255));
	SetBkMode(dc, TRANSPARENT);

	return (LRESULT)hBrush;
}

We include the OnSize function in order to handle changes in screen orientation. We want our label to remain vertically and horizontally centered on the screen.

LRESULT WINAPI OnSize(HWND hwnd, WPARAM wParam, LPARAM lParam)
{
	if (g_hwndLabel)
	{
		RECT rect;
		GetClientRect(hwnd, &rect);

		int iLabelLeft = (rect.right >> 1) - (LABEL_WIDTH >> 1);
		int iLabelTop = (rect.bottom >> 1) - (LABEL_HEIGHT >> 1);

		MoveWindow(g_hwndLabel,
			   iLabelLeft,
			   iLabelTop,
			   LABEL_WIDTH,
			   LABEL_HEIGHT,
			   TRUE);
	}

	return DefWindowProc(hwnd, WM_SIZE, wParam, lParam);
}

We have two functions left. First let's look at the simpler of the two, which is GetGestureDirection. This function evaluates whether or not a DWORD value is equivalent to a number of constants which are defined in the header gesture.h. If the DWORD is equal to one of the predefined direction constants, an OUT parameter will contain a string that corresponds to the direction. If the DWORD cannot be matched to an existing constant, the OUT parameter will read "NONE" (note that there is also a defined constant that corresponds to this as well).

VOID WINAPI GetGestureDirection(WORD wDirection, OUT TCHAR* szDirection, IN DWORD dwSize)
{
	if (dwSize <= 0)
	{
		dwSize = 16;
	}

	switch (wDirection)
	{
	case ARG_SCROLL_RIGHT:
		StringCchCopy(szDirection, dwSize, TEXT("RIGHT"));
		break;
	case ARG_SCROLL_UP:
		StringCchCopy(szDirection, dwSize, TEXT("UP"));
		break;
	case ARG_SCROLL_LEFT:
		StringCchCopy(szDirection, dwSize, TEXT("LEFT"));
		break;
	case ARG_SCROLL_DOWN:
		StringCchCopy(szDirection, dwSize, TEXT("DOWN"));
		break;
	case ARG_SCROLL_NONE:
	default:
		StringCchCopy(szDirection, dwSize, TEXT("NONE"));
		break;
	}

	return;
}

All that is left is to write the OnGesture function. This is the more complicated (but not terribly so) of our functions and will require more explaining. It contains all of the actions that take place when a user touches the screen. First let's look at the function prototype:

LRESULT WINAPI OnGesture(HWND hwnd, WPARAM wParam, LPARAM lParam);

Recall that we are calling the OnGesture function from our Window Procedure whenever we receive a WM_GESTURE message. According to MSDN, when WndProc receives a WM_GESTURE message, wParam contains a DWORD that corresponds to the Gesture ID, and lParam contains a handle to a GESTUREINFO structure (which also contains the Gesture ID). The GESTUREINFO structure contains data which we are interested in such as coordinates of the gesture, direction, and velocity. Let's take a look at the function:

LRESULT WINAPI OnGesture(HWND hwnd, WPARAM wParam, LPARAM lParam)
{
	if (!hwnd || !wParam || !lParam)
	{
		return ERROR_INVALID_PARAMETER;
	}

	GESTUREINFO gi;
	ZeroMemory(&gi, sizeof(GESTUREINFO));

	gi.cbSize = sizeof(GESTUREINFO);

	if (TKGetGestureInfo((HGESTUREINFO)lParam, &gi))
	{
		int iBufferSize = 128;
		PTSTR szGestureText = new TCHAR[iBufferSize];
		ZeroMemory(szGestureText, sizeof(TCHAR) * iBufferSize);

		switch(wParam)
		{

		/* No action on GID_BEGIN or GID_END events.  Otherwise
		   all we would ever see on the screen is GID_END :-)
		*/
		case GID_BEGIN:
		case GID_END:
			return DefWindowProc(hwnd, WM_GESTURE, wParam, lParam);
			break;

		/* A pan event occurs when a user drags their finger(or stylus) across
		   the screen.
		*/
		case GID_PAN:
		{
			/* According to MSDN, a GID_PAN event is 
			   supposed to produce direction,
			   angle, and velocity data when the 
			   GF_INERTIA flag is flipped on.  I
			   have yet to be able to reproduce this behavior.
			*/
			if (gi.dwFlags & GF_INERTIA)
			{
				WORD wDirection = 
					GID_SCROLL_DIRECTION(gi.ullArguments);
				WORD wAngle = GID_SCROLL_ANGLE(gi.ullArguments);
				WORD wVelocity = 
					GID_SCROLL_VELOCITY(gi.ullArguments);
				PTSTR szDirection = new TCHAR[16];
				ZeroMemory(szDirection, sizeof(TCHAR) * 16);

				GetGestureDirection(wDirection, szDirection, 16);

				StringCchPrintf(szGestureText, iBufferSize,
					TEXT("GID_PAN\nX: %d, Y: 
					%d\nDirection: %s\nAngle: 
					%d\nVelocity: %d"),
					gi.ptsLocation.x, gi.ptsLocation.y, 
					szDirection, wAngle, wVelocity);

				SetWindowText(g_hwndLabel, szGestureText);
				delete[] szGestureText;

				if (szDirection)
				{
					delete[] szDirection;
				}
			}
			else
			{
				StringCchPrintf(szGestureText, iBufferSize, 
					TEXT("GID_PAN\nX: %d, Y: %d"),
					gi.ptsLocation.x, gi.ptsLocation.y);

				SetWindowText(g_hwndLabel, szGestureText);
				delete[] szGestureText;
			}
			break;
		}

		/* A Scroll event occurs after a user quickly flicks 
		   their finger (or stylus) across the screen.  
		   This is distinguishable from a Pan because a Pan 
		   -always- occurs when a finger or stylus is dragged, 
		   while a Scroll -only- occurs after the user releases
		   their finger or stylus from the screen after a Pan event, 
		   and only if that Pan event occurs fast enough to 
		   register as a scroll.  
		   You'll see what I mean if you build and run the code on a touch 
		   enabled 6.5 device.  The angle values are in a raw argument
		   form and can be converted to radians using the 
	            GID_ROTATE_ANGLE_FROM_ARGUMENT macro.
		   I elected not to do that here.
		*/
		case GID_SCROLL:
		{
			WORD wDirection = GID_SCROLL_DIRECTION(gi.ullArguments);
			WORD wAngle = GID_SCROLL_ANGLE(gi.ullArguments);
			WORD wVelocity = GID_SCROLL_VELOCITY(gi.ullArguments);
			PTSTR szDirection = new TCHAR[16];
			ZeroMemory(szDirection, sizeof(TCHAR) * 16);

			GetGestureDirection(wDirection, szDirection, 16);

			StringCchPrintf(szGestureText, iBufferSize,
				TEXT("GID_SCROLL\nX: %d, Y: %d\nDirection: 
				%s\nAngle: %d\nVelocity: %d"),
				gi.ptsLocation.x, gi.ptsLocation.y, szDirection, 
				wAngle, wVelocity);

			SetWindowText(g_hwndLabel, szGestureText);
			delete[] szGestureText;

			if (szDirection)
			{
				delete[] szDirection;
			}
			break;
		}

		/* A Hold event occurs when the user touches the screen and 
		   does not move their finger (or the stylus) and also does not 
		   release the screen.  It usually takes about two seconds for a touch
		   event to register as a GID_HOLD message on my phone.  
	            MSDN states that the amount of time required is defined arbitrarily
		   at some lower level, possibly by the OEM.  
		   Therefore the amount of time it
		   takes for a GID_HOLD message to appear may vary by device.
		*/
		case GID_HOLD:
			StringCchPrintf(szGestureText, iBufferSize, 
				TEXT("GID_HOLD\nX: %d, Y: %d"),
				gi.ptsLocation.x, gi.ptsLocation.y);

			SetWindowText(g_hwndLabel, szGestureText);
			delete[] szGestureText;
			break;

		/* A Select event occurs when the user touches the screen.  
		   It is comparable to left clicking a mouse.
		*/
		case GID_SELECT:
			StringCchPrintf(szGestureText, iBufferSize, 
				TEXT("GID_SELECT\nX: %d, Y: %d"),
				gi.ptsLocation.x, gi.ptsLocation.y);

			SetWindowText(g_hwndLabel, szGestureText);
			delete[] szGestureText;
			break;

		/* A Double Select event occurs when the user double taps the screen.
		   That is, quickly taps the screen twice.  
		   It is comparable to double clicking the left
		   button of a mouse.
		*/
		case GID_DOUBLESELECT:
			StringCchPrintf(szGestureText, iBufferSize, 
				TEXT("GID_DOUBLESELECT\nX: %d, Y: %d"),
				gi.ptsLocation.x, gi.ptsLocation.y);

			SetWindowText(g_hwndLabel, szGestureText);
			delete[] szGestureText;
			break;
		default:
			break;
		}

		return ERROR_SUCCESS;
	}
	else
	{
		return ERROR_INVALID_PARAMETER;
	}
}

Points of Interest

I used MSDN pretty heavily when getting an initial grip on using WM_GESTURE messages. One frustrating point was that the MSDN documentation made no mention of the TouchGestureCore.lib file being required as a project dependency, or even that it exists at all. Fortunately it wasn't so cryptically named that I couldn't find it for myself.

One other item of note was that after playing with the app for a bit, it was easy to see that the Gestures API in Windows Mobile 6.5 supports multi-touch (at least on my HTC Imagio), so it's curious that I have yet to see an app out there that takes advantage of this feature.

History

• 29^th October, 2009: Initial version