Ultrabook™ Device and Tablet Windows* Touch Developer Guide

• Download Ultrabook™ Device and Tablet Windows Touch Developer Guide - [PDF 935KB]

Contents

• Abstract
• Introduction
• Hardware Requirements
• Operating Environments for Ultrabook Devices
• Touch Interactions: Pointer, Gesture, and Manipulation
  • Pointer Interactions
  • Manipulation and Gesture Interactions
  • Custom Gesture Recognition
• Touch Support in Web Browsers
  • Internet Explorer 10 and its Compatibility with Windows 7
• Identifying Touch Capability
• UI design for Touch-Enabled Ultrabook Devices
• Resources for Developing Touch Applications
  • Related Articles on Intel Developer Zone
  • Related Articles on MSDN
• Summary
• About the Author

Abstract

This Guide contains information about the APIs that application developers need to use when they are developing apps targeted for Ultrabook™ devices. It covers information for developing apps that run on Windows* 7, Windows* 8 Desktop, Windows* Modern UI, as well as apps that run in web browsers.

This guide also provides descriptions of common user expectations for gestures and provides some guidelines for developing satisfying touch interfaces.

Introduction

Now that there are many touch-capable Ultrabook™ devices available, software developers want to create software that is designed for touch input. For an app to be fully touch-enabled, it must meet three criteria:

• The most frequently used controls are at least 40X40 pixels, big enough to be uniquely indicated by a fingertip.
• Relevant gestures are supported (panning, zoom, rotate, two-finger tap, press, and tap), and the effect occurs at the point of contact.
• The app provides smooth, responsive visual feedback while panning, zooming, and rotating so that it feels highly interactive.

Whether developers are writing a new app or want to update an existing app to include touch support, this guide will help them in their journey.

For more information on writing touch-enabled apps, refer to the MSDN article, Touch Interaction Design.

Hardware Requirements

Consumers have a variety of touch-capable devices to choose from. Developers have a challenging task when attempting to develop apps that feel natural on each device. In general, touch-enabled apps should be designed to run on any of their targeted devices while taking full advantage of the touch capabilities of each. Ultrabook devices represent the high-end of touch-capability and should be the design point for an app’s touch interface.

Ultrabook devices with touchscreens have been available since 2011:

• 2nd generation Intel® Core™ Processor Family (codenamed Sandy Bridge). These systems launched with Windows 7 and some have touchscreens.
• 3rd generation Intel® Core™ Processor Family (codenamed Ivy Bridge). These systems also launched with Windows 7 first, but are now being sold with Windows 8. We will see more of these having touch capability in 2013 and beyond, especially since many of the newer designs are convertibles.

The rest of this Developer Guide assumes that the target platform is a touch-capable Ultrabook system. Software designed for this device can be adapted to other touch-enabled devices that run the same OS.

Operating Environments for Ultrabook Devices

Designing apps today requires careful considerations of what environment customers most often use as well as which environment an app is best suited for. Whether an app is targeted for Windows 8 Desktop or as a Windows Store app, the developer needs to understand which interfaces are applicable.

If an app is targeted for the Windows 8 UI (also known as a Windows Store app), then it must use the WinRT APIs. If an app is to run in the Windows 8 Desktop environment, there are more choices: the legacy APIs from Windows 7 and the new Windows 8 APIs for touch. These interfaces will be discussed further in the sections below. Other options exist for developing web apps. Touch interfaces available for web browsers are also discussed below.

Touch Interactions: Pointer, Gesture, and Manipulation

There are varying levels of interpretation of touch input. Pointer events are the most basic because they represent individual points of touch contact. Gesture and Manipulation events are built upon that foundation. Gesture events provide an easy way to capture simple tap-and-hold gestures. Manipulation events are for touch interactions that use physical gestures to emulate physical manipulation of UI elements. Manipulation events provide a more natural experience when the user interacts with UI elements on the screen. The available touch interfaces have varying levels of support for these three levels of interpretation.

Read more about Gestures, Manipulations and Interactions on MSDN

Pointer Interactions

A pointer event is a single, unique input or “contact” from an input device such as a mouse, stylus, single finger, or multiple fingers. When a contact is made, the system creates a pointer when it is first detected and then it is destroyed when the pointer leaves the detection range or is canceled. In the case of multi-touch input, each contact is a unique pointer. Table 1 shows the interfaces for retrieving basic pointer events that are available to Ultrabook devices running Windows 7 and Windows 8.

Table 1. Pointer APIs available for Ultrabook Devices

Touch Interfaces available for Windows 7 and Windows 8:

• WM_TOUCH (messages)
• WM_POINTER (messages)
• PointerPoint (class)
• RegisterTouchWindow (function)

Refer to Guidelines for common user interactions on MSDN.

Windows 7 and Windows 8 Desktop Touch Interface: WM_TOUCH

The WM_TOUCH message can be used to indicate that one or more pointers, such as a finger or pen, have made contact on the screen.

Sample Code:

• Intel Developer Zone: Touch for Windows Desktop (This sample includes custom Gesture Recognition.)

Guidance:

• MSDN: Touch Interaction Design
• MSDN: Windows 7 Touch Input Programming Guide

Windows 8 Desktop Touch Interface: WM_POINTER

The WM_POINTER messages are part of the Direct Manipulation APIs and are specific to the Windows 8 Desktop. This interface can be used to capture individual touch pointers as well as Gestures and Manipulations. The WM_POINTER messages will be discussed further in the section on Manipulation and Gesture Interactions.

Reference on MSDN: Direct Manipulation APIs

Windows Modern UI Touch Interface: PointerPoint

The PointerPoint class is part of the Windows Runtime environment and is compatible only with Windows Store apps. It provides basic properties for the input pointer associated with a single mouse, stylus, or touch contact. MSDN has sample code that can help developers get started working with the PointerPoint interface.

Sample Code on MSDN: Input: XAML user input events sample

Manipulation and Gesture Interactions

Gesture events are used to handle static-finger interactions such as tapping and press-and-hold. Double-tap and right-tap are derived from these basic gestures:

• Gestures: the physical act or motion performed on or by the input device that can be one or more fingers, a stylus, or a mouse.
• Manipulation: the immediate, ongoing response an object has to a gesture. For example, the slide gesture causes an object to move in some way.
• Interactions: how a manipulation is interpreted and the command or action that results from the manipulation. For example, both the slide and swipe gestures are similar but the results vary according to whether a distance threshold is exceeded.

Table 2. Basic gestures used for manipulation interactions

Table 3. Gesture Interfaces available for Ultrabook™ Devices

References on MSDN:

• WM_TOUCH + IManipulationProcessor
• WM_GESTURE + GESTUREINFO
• WM_POINTER + Direct Manipulation APIS
• PointerPoint + GestureRecognizer

Table 4. Standard expected interactions and consequences for touch interactions

Interpreting Manipulation and Gesture Interactions for Windows 7 Desktop

The IManipulationProcessor interface can be used in conjunction with the WM_TOUCH API to provide a way to add translation, rotation, scaling, and inertia to UI objects. This combination provides functionality similar to the gesture recognizing features of WM_POINTER. Once the Manipulation Processor is enabled, manipulation starts as soon as a touch gesture is initiated.

Sample Code:

• MSDN: Manipulation and Inertia Sample

WM_GESTURE messages have a structure called GESTUREINFO that is available for the interpretation of gestures and manipulations. The MSDN web page for WM_GESTURE shows an example of how to obtain gesture-specific information using the GESTUREINFO structure

Note that WM_GESTURE has limitations, such as the maximum number of simultaneous touch inputs is only two and it does not support simultaneous gestures. For apps that require more capability but still need to support Windows 7 desktop, use the WM_TOUCH interface and either write a custom gesture recognizer, as detailed in the section Custom Gesture Recognition below, or use the Manipulation Processor interface with WM_TOUCH.

Sample Code on Intel Developer Zone (WM_GESTURE API + GESTUREINFO: Sample Application: Touch for Desktop

For more information on writing touch-enabled apps, refer to the MSDN article: Touch Interaction Design.

Handling Manipulation and Gesture Interactions for Windows 8 Desktop Apps

Applications targeted only for the Windows 8 Desktop can use the Direct Manipulation APIs (WM_POINTER messages). The pointer messages are passed to an internal Interaction Context object that performs recognition on the manipulation without the need to implement a custom gesture recognizer. There is a callback infrastructure where all interactions involving tracked contacts are managed.

Direct Manipulation is designed to handle both manipulation and gesture interactions and supports two models for processing input:

1. Automatic/Independent: Window messages are automatically intercepted by Direct Manipulation on the delegate thread and handled without running application code making it independent of the application.
2. Manual/Dependent: Window messages are received by the window procedure running in the UI thread, which then calls Direct Manipulation to process the message making it dependent on the application.

Gestures can be captured by initializing Direct Manipulation and preparing the system for input processing.

Refer to the Quickstart: Direct Manipulation on MSDN for an outline of the API calls required to accomplish typical tasks when working with Direct Manipulation.

Handling Manipulation and Gesture Interactions in Windows 8 Modern UI

The GestureRecognizer API is used to handle pointer input to process manipulation and gesture events. Each object returned by the PointerPoint method is used to feed pointer data to the GestureRecognizer. The gesture recognizer listens for and handles the pointer input and processes the static gesture events. For an example of how to create a GestureRecognizer object and then enable manipulation gesture events on that object see the MSDN GestureRecognizer web page (referenced below.)

References on MSDN:

• GestureRecognizer APIs
• PointerPoint method

Sample Code on Intel Developer Zone: Sample Application: Touch for Windows* Store.

Custom Gesture Recognition

When possible, use the built-in gesture recognizers (see Table 3). If the provided gesture and manipulation interfaces do not provide the functionality that is needed, or maybe the app needs to disambiguate between taps and gestures more rapidly, it may be necessary to write custom gesture recognition software. If this is the case, customers expect an intuitive experience involving direct interaction with the UI elements in the app. It is best to base custom interactions on the standard controls to keep user actions consistent and discoverable. Custom interactions should only be used if there is a clear, well-defined requirement and basic interactions don't support the app’s desired functionality. See Table 4 for the list of common and expected interactions and consequences for touch interactions.

Code Sample on Intel Developer Zone (WM_TOUCH with custom gesture recognition): Touch for Windows Desktop

Touch Support in Web Browsers

Touch input is also available to apps running in web browsers, with varying degrees of support depending on the browser. Since web browser capabilities change rapidly, it is generally better to detect supported features instead of specific browsers. Feature detection has proven to be a more effective technique once the determination has been made if it is Internet Explorer (IE) 10, a browser built on Webkit*, or a different browser that requires support. Feature detection requires less maintenance for the following reasons:

• New browsers get released and existing browsers are often updated. Existing code may not factor in the new browser versions. Updated browsers may support standards and features that were not supported when the browser detection code was designed.
• New devices frequently include new versions of browsers, and so browser detection code must be reviewed continually to support the new browsers. Creating customized implementations for each browser can become extremely complicated.
• Many browsers support the ability to modify the user-agent string making browser detection difficult to accurately identify.

WebKit powers Apple Safari* and Google Chrome*, and soon Opera will move their browser over to use it. Internet Explorer 10 does not use WebKit; however, both WebKit and IE 10 are built on top of the Document Object Model (DOM) Level 3 Core Specification. To review the standards associated with touch events, refer to the standard’s Touch Events Version 1, dated January 2013.

References:

• Webkit
• Document Object Model (DOM) Level 3 Core Specification
• Touch Events Version 1
• The navigator object
• Navigator userAgent property

IE 10 has its own touch interfaces that must be called for processing touch events. Use the navigator object with the userAgent property to determine if the browser supports the desired features. The following example indicates that the browser is Internet Explorer.

Usage:

<script type="text/JavaScript">
If (navigator.userAgent.indexOf(“MSIE”)>0)
    { 
         // Run custom code for Internet Explorer.
    }
</script>

Use the hasFeature method to determine if specific features are supported in the browser. For example, here is how to determine if a browser supports touch events (this works for IE 10 as well):

var touchSupported = document.implementation.hasFeature("touch-events","3.0");

Where “touch-events” is the feature that we are checking for and “3.0” is the DOM specification level that we are interested in. An app can then listen for the following touch events: touchstart, touchend, touchmove, and touchcancel. 

Reference: hasFeature method

To process touch events using a WebKit-based browser (Chrome, Safari, etc.), simply set up the following three events to cover the main input states:

canvas.addEventListener( ‘touchstart’, onTouchStart, false );
canvas.addEventListener( ‘touchmove’, onTouchMove, false);
canvas.addEventListener( ‘touchend’, onTouchEnd, false);

For Internet Explorer, reference the MSPointer event instead:

canvas.addEventListener( ‘MSPointerDown’, onTouchStart, false );
canvas.addEventListener( ‘MSPointerMove’, onTouchMove, false);
canvas.addEventListener( ‘MSPointerUp’, onTouchEnd, false);

Similarly, there are gesture event listeners as well. They are gestureStart, gestureChange, and gestureEnd for the non IE 10 browsers.

Download sample code handling DOM pointer events on MSDN: Input DOM pointer event handling sample.

Internet Explorer 10 and its Compatibility with Windows 7

While IE 10 does not use WebKit, it is built on top of the DOM Level 3 Events, HTML5, and Progress Event standards. This section provides information about IE 10 and how it interacts on Windows 7.

Standards:

• DOM Level 3 Events
• HTML5
• Progress Event [shouldn’t this have an “s”?]

Internet Explorer 10 on Windows 7 handles touch and pen input as simulated mouse input for the following Document Object Model (DOM) events:

• MSPointerCancel
• MSPointerDown
• MSPointerMove
• MSPointerOver
• MSPointerUp

IE 10 on Windows 7 will not fire any of the following DOM Gesture events:

• MSGestureChange
• MSGestureEnd
• MSGestureHold
• MSGestureStart
• MSGestureTap
• MSManipulationStateChanged

Table 5. Touch Interfaces for Internet Explorer* 10

For more information on developing touch-enabled web apps for IE 10 (MSDN): Internet Explorer 10 Guide for Developers

Sample code on MSDN: Input: Manipulations and gestures (JavaScript)

Identifying Touch Capability

Whether an app is a native app or a web app, the developer will want to add a check for hardware touch capability so that the app can configure its UI appropriately. Use the following methods to test for touch capability.

Windows 7 and Windows 8 Desktop

Apps targeting Windows 7 or for Windows 8 Desktop can call GetSystemMetrics with SM_DIGITIZER as the argument. The following code snippet is part of a Touch sample that can be downloaded from the Intel Developer Zone: Touch for Windows Desktop

void SDKSample::DeviceCaps::Touch::TouchGetSettings_Click(Platform::Object^ sender, Windows::UI::Xaml::RoutedEventArgs^ e)
{
    Button^ b = safe_cast<Button^>(sender);
    if (b != nullptr)
    {
        TouchCapabilities^ pTouchCapabilities = ref new TouchCapabilities();
        Platform::String^ Buffer;

        Buffer = "There is " + (pTouchCapabilities->TouchPresent != 0 ? "a" : "no") + " 
              digitizer presentn";
        Buffer += "The digitizer supports " + pTouchCapabilities->Contacts.ToString() + " 
              contactsn";

        TouchOutputTextBlock->Text = Buffer;
    }
}

Web apps

For Internet Explorer, use the msMaxTouchPoints property described as follows:

Test for touch capable hardware

Test for multi-touch capable hardware

Get the maximum number of touch points the hardware supports

For Chrome and Safari, use the following (same as above but replace msMaxTouchPoints with maxTouchPoints):

var result = navigator.maxTouchPoints;

It can be somewhat tricky to test for touch devices generically from web apps. While some functions work well on some browsers, others indicate that touch is present when it is not, i.e., if the browser itself supports touch, it may report that touch is available even if the device is not touch-capable.

Note that MaxTouchPoints will return 0 in IE 10 (Desktop) running on Windows 7.

References:

• msMaxTouchPoints
• maxTouchPoints

UI design for Touch-Enabled Ultrabook Devices

Apps designed for Ultrabook devices may need to process gestures such as taps, pans, zooms, etc. Apps that are touch-enabled may do little with the raw pointer data except to pass it to gesture detection.

New applications should be designed with the expectation that touch will be the primary input method. Mouse and stylus support require no additional work; however, software developers should consider several other factors when designing touch-optimized apps.

Table 6. Considerations for Touch-Enabled Apps

    // Check for Touch support
    // Get the Touch Capabilities by calling GetSystemMetrics
    BYTE digitizerStatus = (BYTE)GetSystemMetrics(SM_DIGITIZER);
    // Hardware touch capability (0x80); readiness (0x40)
    if ((digitizerStatus & (0x80 + 0x40)) != 0) //Stack Ready + MultiTouch
    {
        RegisterTouchWindow(m_pWindow->GetHWnd(), TWF_WANTPALM);
    }

	BYTE nInputs = (BYTE)GetSystemMetrics(SM_MAXIMUMTOUCHES);

Software developers should supply appropriate visual feedback during interactions so that users can recognize, learn, and adapt to how their interactions are interpreted by both the app and by the OS. Visual feedback is important for users to let them know if their interactions are successful, so they can improve their sense of control. It can help reduce errors and help users understand the system and input device.

Resources for Developing Touch Applications

Related Articles on Intel Developer Zone:

1. Comparing Touch Coding Techniques – Windows 8 Desktop Touch Sample
2. Exploring Touch Samples for Windows* 8 apps
3. Touch Code Sample for Windows* 8 Store
4. Touch Code Sample for Windows* 8 Desktop
5. Porting Win32 Apps to Windows 8 Desktop
6. Real-Time Strategy Game with Touch Screen

Related Articles on MSDN

1. Windows 7 Touch Input Programming Guide
2. Architectural Overview (Windows 7)
3. Troubleshooting Applications
4. Adding Manipulation Support in Unmanaged Code
5. Windows Touch Samples
6. Build Advanced Touch Apps in Windows 8* (Video)
7. Windows 8 SDK
8. Input: Touch hit testing sample
9. Desktop App Development Documentation (Windows)
10. Windows Touch Gestures Overview (Windows)
11. Getting Started with Windows Touch Messages (Windows)
12. Get PointerTouchInfo function (Windows)
13. (MSDN) Internet Explorer 10 Guide for Developers

Summary

App developers, who want to develop touch-enabled apps, whether their apps will run natively or are targeted as web apps, need to have a clear understanding of which APIs are available to them. This guide covered the interfaces available for the following environments: Windows 7, Windows 8 Desktop, Windows Modern UI, as well as apps running in web browsers. While Gestures and Manipulations are possible in Windows 7, app developers may find that the Windows 8 APIs (those targeted for the Desktop and/or for the Modern UI) offer the best options for automatic gesture recognition.

App developers who are writing touch-enabled web apps need to check their code for IE 10 since IE 10 has its own interface that must be used to process touch, gestures, and manipulation. Other Webkit-based browsers are also based on the DOM Level 3 standards and have touch and gesture event support.

This guide also covered descriptions of common Gesture and Manipulation interactions and provided some guidelines for developing touch-enabled apps.

This article was written by Gael Hofemeier. Gael is an Evangelist Application Engineer at Intel Corporation. Her focus is providing technical content that developers writing software for Intel® Architecture need. In addition to writing content, she also moderates the Business Client Forum on the Intel Developer Zone.

See Gael's Blog Author Page

Intel, the Intel logo, Ultrabook, and Core are trademarks of Intel Corporation in the US and/or other countries.

Copyright © 2013 Intel Corporation. All rights reserved.

*Other names and brands may be claimed as the property of others.

• Download ultrabook-touch-developer-guide.pdf - 935.56 KB