Click here to Skip to main content
65,938 articles
CodeProject is changing. Read more.
Articles
(untagged)

Kinect and WPF: Painting with Kinect using OpenNI

0.00/5 (No votes)
3 Dec 2013 1  
Painting with Kinect using OpenNI

It's time for the second blog post considering Kinect and WPF. In the previous one, I showed you how to obtain the raw and depth images from the Kinect cameras. This time, I developed a simple painting application in Windows Presentation Foundation (I implemented a "punch" gesture, too). Here is a video demonstrating the way it is used:

[Video Preview]

How It Works

Wave on camera and Kinect will start tracking the waving hand. While the hand is moving, color is drawn in the corresponding screen points. Punch forward to stop painting. Punch again to restart!

Prerequisites

Implementing the Demo in C# and WPF

Step 0

Ensure that OpenNI is properly installed in your Windows operating system.

Step 1

  • Open Visual Studio and create a new WPF application (I named it "KinectPaint") and select .NET 4 as the target framework (you can use either version 4 or 3.5).
  • Add a reference to OpenNI.net.dll. OpenNI.net is found under C:\Program Files\OpenNI\Bin.
  • Add an existing item and load SamplesConfig.xml to your project. Remember to copy this file in your output folder (Debug / Release). SamplesConfig.xml is found under C:\Program Files\OpenNI\Data. You need to have the default XML file replaced with something like the one I provided in my "how-to" post.

Step 2

Download my NuiHandTracker & NuiPositionEventArgs classes. These classes use OpenNI internally to track a hand. The first one also implements a custom "punch" gesture!

Step 3

Navigate to MainWindow.xaml and add a canvas in your XAML code (yes, it is both a painting and a WPF one):

<Canvas Name="canvasPaint" Width="640" Height="480" />  

Step 4

Navigate to MainWindow.xaml.cs and create a new instance of NuiHandTracker providing the SamplesConfig.xml file path as a parameter. Also handle its Hover and Push events.

_handTracker = new NuiHandTracker("SamplesConfig.xml");
_handTracker.Hover += new NuiHandTracker.HoverHandler(Hands_Hover);
_handTracker.Push += new NuiHandTracker.PushHandler(Hands_Push);

Step 5

We need a boolean member indicating whether the user is painting or not. Declare one and initialize it to true:

bool _isPainting = true;

Step 6

It's time to paint! Fill-in the Hands_Hover and Hands_Push event handlers:

void Hands_Hover(object sender, NuiPositionEventArgs e)
{
    // Hover event.
    if (_isPainting)
    {
        // Coordinates range from -320 to 320 and -240 to 240.
        DrawPixels(e.X + (IMAGE_WIDTH / 2), e.Y + (IMAGE_HEIGHT / 2));
    }
}
void Hands_Push(object sender, NuiPositionEventArgs e)
{
    // Push event!
    _isPainting = !_isPainting;
}

IMAGE_WIDTH and IMAGE_HEIGHT represent Kinect camera's resolution (640x480 respectively). DrawPixels method simply creates a new Ellipse object and adds it to the canvas element. X and Y coordinates, provided by OpenNI, range from -320 to +320 and -240 to +240. Canvas' coordinates start from 0, so we needed to convert the negative coordinates into positive ones properly. Here is the DrawPixels method:

private void DrawPixels(float x, float y)
{
    Dispatcher.BeginInvoke((Action)delegate
    {
        Ellipse ellipse = new Ellipse
        {
            Fill = new SolidColorBrush(Colors.Blue),
            Width = 25,
            Height = 25
        };

        Canvas.SetLeft(ellipse, x);
        Canvas.SetBottom(ellipse, y);
        
        canvasPaint.Children.Add(ellipse);
    });
}

You are done! KinectPaint is ready! I have added extra functionality in the demo I provide (such as random color generation and the ability to view the raw image from the camera).

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here