Introduction
With the release of Windows 7, we now have native support for multi-touch in our applications. As long as the hardware can register multiple, simultaneous touches, Windows 7 can handle it. In addition, .NET 4.0 and WPF 4.0 now have built-in support for multi-touch coding. This article will show you how to utilize these features. I wrote a multi-touch robot control app to demonstrate the new features. This was written using Visual Studio 2010 beta 2 and WPF 4.0.
Background
When Visual Studio 2010 beta 2 was first made available, I downloaded it right away and installed it. Then I wondered, what's new that I can do with this? Of course, multi-touch immediately came to mind. I'm excited about the new support for multi-touch. We are just scratching the surface (heh heh) for new app interfaces using multi-touch. I think there are going to be many cool, new user experiences created by developers like you.
Slider Control
For starters, I needed a slider control, so I created a WPF user control called SliderControl.XAML. I gave it a nice, rounded border and a linear gradient fill. Then, I added an ellipse as the control knob for the user to interact with.
TouchDown, TouchUp, and TouchMove Events
The TouchDown event is fired each time your finger initially touches the screen. In TouchEventArgs, you will get a TouchDevice object that contains an ID that the multi-touch hardware assigns to the current touch. You can use this ID in subsequent code to track the touch. However, there are other methods that can track this for you by capturing the touch.
The TouchUp event is fired each time you lift your finger off of the screen. You will get another TouchDevice object so that you can release any capture you have on the current touch.
The TouchMove event is fired many times as you drag your fingers across the screen. A separate event will fire for each finger movement on the screen. You cannot guarantee any order in which the events will arrive. This is the reason that TouchDevice.ID is important.
To detect and capture touch events, we need to first hook the event handlers in the constructor of SliderControl. You will notice that we hook the TouchDown and TouchUp events of the user control, but hook the TouchMove event of the ellipse. This is because the ellipse will eventually capture the TouchMove events and process them.
this.TouchDown += new EventHandler<TouchEventArgs>(SliderCtrl_TouchDown);
this.TouchUp += new EventHandler<TouchEventArgs>(SliderCtrl_TouchUp);
this.elSlider.TouchMove += new EventHandler<toucheventargs>(elSlider_TouchMove);
Once a TouchDown event fires, we want to determine if it occurred over the control knob, in this case, the ellipse. If the TouchDevice.DirectlyOver object returned is the ellipse, then we will capture the touch as long as the ellipse hasn't already captured another touch.
private void SliderCtrl_TouchDown(object sender, TouchEventArgs touchEventArgs)
{
if (touchEventArgs.TouchDevice.DirectlyOver == this.elSlider)
{
if (this.elSlider.TouchesCaptured.Count() == 0)
{
this.elSlider.CaptureTouch(touchEventArgs.TouchDevice);
touchEventArgs.Handled = true;
}
}
}
After a TouchDevice has been captured by the ellipse, the TouchMove event handler will receive all the moved data from that touch. So, we simply get the current position, do whatever calculations we want, and wait for the next TouchMove event. I use the position data to set the Top.Property of the ellipse. I also added code to limit how far it can travel on the screen, calculate a normalized value based on the position, and fire an event called ValueChanged.
private void elSlider_TouchMove(object sender, TouchEventArgs touchEventArgs)
{
double position = touchEventArgs.GetTouchPoint(this).Position.Y - 30;
if (position > MIN_VAL && position < MAX_VAL)
{
this.elSlider.SetValue(Canvas.TopProperty, position);
CalcCurrentValue(position);
this.txtVal.Text = string.Format("{0:F2}", this._curValue);
this.OnValueChanged(this, this._emptyArgs);
}
touchEventArgs.Handled = true;
}
After we lift one of our fingers off the screen, we will get a TouchUp event. Here, we will determine if that TouchDevice is captured by the ellipse. If so, then release it, center the ellipse on the slider, and send another ValueChanged event.
private void SliderCtrl_TouchUp(object sender, TouchEventArgs touchEventArgs)
{
if (touchEventArgs.TouchDevice.Captured == this.elSlider)
{
this.elSlider.ReleaseTouchCapture(touchEventArgs.TouchDevice);
CenterSlider();
touchEventArgs.Handled = true;
this.OnValueChanged(this, this._emptyArgs);
}
}
Using the Code
The whole reason for this app was to control a Lego NXT robot using multi-touch. I wanted two slider controls on the screen that could simultaneously control each motor independently. So, each slider control is wired up to fire a ValueChanged event, and the main app will send serial commands to the NXT robot to move each motor forward or backward depending on the position of the slider control.
If you don't want to control a robot, but you still want multi-touch slider controls for your application, then it's easy enough to pull out the SliderControl.XAML user control and drop it in your own app. You can hook the ValueChanged event and away you go. I did add a throttling mechanism in the SliderControl so that it doesn't send back values too quickly for the NXT robot to process. That's easy enough to remove if you don't want it.
I've uploaded a video to YouTube showing the app in action controlling the robot. You can find it at: Multi-touch Robot Control using WPF 4.0.
Devices in Windows 7
Windows 7 has made it very easy to connect up devices to your laptop. In this case, I connected a Lego NXT brick using Bluetooth. It was very simple. First, turn on the NXT brick and set it to be visible over Bluetooth. Then, open Devices and Printers in Windows 7. It's located below the Control Panel in the Start menu.
You should see your NXT available in Devices once you type in the correct pass-code. The default for NXT is '1234'.
You can double-click on the NXT icon and see the services that it makes available. In this case, it shows up as a serial port. The app uses the SerialPort class to talk with the NXT robot. You can find all the documentation and direct commands on the Lego NXT website. For this application, the serial port for the specific NXT you want to control is set in the App.Config file. Simply change the COM value to whatever your NXT usesm and run the app.
Conclusion
This article showed you how to easily access the multi-touch capabilities of multi-touch enabled laptops, using Visual Studio 2010 beta 2 and WPF 4.0. I hope that in the future, hardware manufacturers will ship laptops with more simultaneous touch points. I believe that there are still many great multi-touch .NET apps waiting to be written. We need the hardware to keep up with our imaginations.
In the future, I plan on adding more features to this app for even more multi-touch robot control fun.
History
- Feb. 2, 2010 - Initial release of article and source code.
