Kinect Cursor for Hand Tracking

Navigating through a Natural User Interface using your palm is quite common – after all, it’s the primary navigation mechanism XBOX uses. Many Windows Kinect applications implement hand tracking for similar purposes. Today, I would like to share a Kinect hand cursor control I developed and you can use for your own apps. This hand cursor control will save you tons of time and you’ll be able to integrate it right into your existing WPF code!

• Download from GitHub

Here is the final result of this handful user control:

Using the control in your project is fairly easy. Read on!

Prerequisites

• Kinect for Windows or Kinect for XBOX sensor
• Microsoft Kinect SDK

The Code

OK, let’s type some quick code now.

Step 1: Download the Project from GitHub

Download the source code and build it using Visual Studio. Locate the assembly named KinectControls.dll.

Step 2: Import the Assembly to Your Project

Create a new WPF project and add a reference to the assembly you built previously.

Step 3: Import the Assembly to your XAML Code

Type the following line of code in your Window, view or user control definition:

xmlns:Controls="clr-namespace:KinectControls;assembly=KinectControls"

You can now drag a Canvas and place the hand cursor inside it:

<Canvas Name="canvas" Width="640" Height="480">
    <Controls:KinectCursor x:Name="cursor" Width="100" Height="100" />
</Canvas>

Step 4: Move the Cursor using C#

The KinectCursor class contains a method named Update. You need to call that method whenever you want to move the cursor. The Update method gets the X and Y coordinates of the hand in the 2D color or depth space. Alternatively, you can provide a ColorSpacePoint or DepthSpacePoint that contains the coordinates. I placed this code inside SkeletonFrameReady event handler. You can specify the “active” hand by comparing the Z values of the left and right hands.

// Select the hand that is closer to the sensor.
var activeHand = handRight.Position.Z <= handLeft.Position.Z ? handRight : handLeft;
var position = _sensor.CoordinateMapper.MapSkeletonPointToColorPoint(
                                        activeHand.Position,
                                        ColorImageFormat.RgbResolution640x480Fps30);

cursor.Flip(activeHand);                            
cursor.Update(position);

Also, notice the Flip method? The Flip method mirrors the cursor visual, so to properly match the active hand. That's it! You can check the complete source code on GitHub.

The hand image is a scalable vector shape I designed. You can change its color, dimensions or shadow effects easily:

<Controls:KinectCursor x:Name="cursor" 
Width="400" Height="400" Fill="Blue" />

Copyrights

You are free to use the user control as you wish for your personal and commercial projects, just by making a simple attribution in your project.