Automation Enabler PlugIn - Exposes Inaccessible Elements to UI Automation Framework

• Download demo - 164.24 KB

Introduction

Microsoft UI Automation is the new accessibility framework for Microsoft Windows, available on all operating systems that support the Windows Presentation Foundation (WPF). UI Automation provides programmatic access to most user interface (UI) elements on desktop.

Apart from demonstrating the usage of this framework to control an application, this article also shows how to create a plug-in that can be configured to reuse across all controls with minimal code change.

Background

The user Interface elements in commercial Windows based applications are created using custom user controls for the following reasons:

1. Give a uniform look and feel across the product and across product suites
2. Faster application development by adopting software re-use

These applications have to undergo extensive testing to pass the quality acceptance criteria before it can be released to the market. Automation testing in addition to manual testing is employed to exercise the application in various test scenarios. Automation testing has the advantage that it can be run for long hours over several days without interruptions to test the system functionality and stability - something which is difficult with only manual testing.

Problem

Automation Test frameworks available in the market which test User Interface based application rely on the accessibility of the windows controls used for creating the UI. However the same default accessibility may not available if the User Interface is created using painted UI element (as against window controls). In such cases, the automation framework shall not be able to access such elements by default. To be able to make such elements accessible via automation testing framework, the parent control encapsulating the painted UI elements would need to implement some prescribed interfaces (specified by Microsoft .NET UI Automation framework). Since this accessibility is not available by default and the author of the control has to specifically provide the implementation, there is significant additional overhead involved in this. The result is that either the accessibility remains unimplemented or an effort is spent in implementing this accessibility. The former means that the testability of the certain areas of the system is compromised. The latter means the developers have to plan additionally for learning and implementing the automation enabler feature for his UI Element.

Goal of This Article

This article is targeted towards the software developers and testers.

The goal of this article is to enable control developers to incorporate automation accessibility in a way which is:

1. Fast, easy in terms of effort and time
2. Able to be integrated with minimal technical issues
3. Able to be integrated with minimal knowledge of Automation framework
4. Does not impose limitation on how the control is designed and developed
5. Can be re-used for controls

Using the Code

We discuss the run-time environment of the Microsoft .NET UI Automation (the accessibility framework for Microsoft Windows) as an example. UI Automation enables assistive technology products such as screen readers to provide information about the UI to end users and to manipulate the UI by means other than standard input. UI Automation also allows automated test scripts to interact with the UI to enable UI driven testing of the application. All the UI Elements to be made accessible have to implement specific interfaces. During automation testing, these interfaces are then accessed programmatically through these standard interfaces by the Testing Engine which is running the test scripts.

The article proposes to design an Automation-Enabler Plug-in library whose reference can be plugged into the target control (which needs to be automation enabled). Proposed Automation-Enabler Plug-in library will implement the mandated interfaces on UI Element's behalf. Automation-Enabler Plug-in library will have separate classes available in the framework to provide the access of control and its child UI elements by implementing the required UI Automation framework interfaces.

There will be a separate XML configuration file for each user control to tell the plug-in framework about interfaces required by control and its child UI elements according to their behavior. Automation-Enabler Plug-in will expose only those behaviors to automation testing framework.

Plug-in library will use the APIs of control and its child UI elements to expose the behavior. Since the APIs defined in each control could be different, the XML configuration file will also contain the API mapping details for Control and its child UI elements. This API mapping details will contain the entries against plug-in callable APIs in control. For example, if the control/object has its Size defined by a field 'Bounds' and plug-in reads Size by a field 'BoundingRectangle', XML mapping file will alias these two entries so that internally plug-in framework will use 'Bounds' field to get the Size of object.

While Automation testing once the Automation-Enabler Plug-in will get the reference of user control, it will read the corresponding XML configuration file and provide the specified behavior for that user control to automation testing framework.

Schematic representation of the plug-in framework and interaction with user control and automation test framework is shown below:

In the example, I have created a customized control which involves a user control and some class objects as its child UI elements. So there are two main classes to construct the user control named – MeasurementControl and MeasurementItem. MeasurementControl class is derived from UserControl and acts as host/parent to display the UI elements inside it. MeasurementItem class is used to represent each UI element.

In Microsoft UI Automation Framework, AutomationElement class represents a UI Automation element in the UI Automation Tree, and contains values used as identifiers by UI Automation client applications.

When the MeasurementControl class instance is passed to the plug-in framework, plug-in implements IRawElementProviderSimple interface behalf of these classes; which is used to basically identify an AutomationElement (usually only control is identified as an AutomationElement). MeasurementControl is derived from UserControl so it is recognized as AutomationElement automatically but MeasurementItem instance cannot be recognized as AutomationElement by default.

Since MeasurementControl class acts as the root and MeasurementItem class represent each UI element inside the control, internally plug-in have implemented IRawElementProviderFragmentRoot interface for MeasurementControl class and IRawElementProviderFragment interface for MeasurementItem class. These two interfaces are required if our control behaves as a container control (which contains child UI elements inside it). These interfaces make each MeasurementItem also as a separate AutomationElement so that it appears distinct - visually as well as programmatically.

In the plug-in framework, ControlProvider class is responsible for exposing the MeasurementControl class and ObjectProvider class is responsible for exposing MeasurementItem class. IValueProvider and IExpandCollapseProvider are the UI Automation interfaces required to achieve certain behavior. These interfaces are implemented on the classes ControlProvider and ObjectProvider according to the XML configuration file. A typical XML configuration file is shown below:

<?xml version="1.0" encoding="utf-8" ?>
<ApiMapping>
  <Control>
    <ClassName>MeasurementControl</ClassName>
    <HelpTextProperty>This is MeasurementControl user control</HelpTextProperty>

    <!-- API mapping.-->
    <Name>Label</Name>
    <BoundingRectangle>Bounds</BoundingRectangle>
    <Items>Children</Items>

    <!-- Automation Framework Interface related.-->
    <IValueProvider available="false">
      <IsReadOnly>true</IsReadOnly>
      <SetValue>Label</SetValue>
      <Value>Label</Value>
    </IValueProvider>
  </Control>

  <Object>
    <!-- Similar entries for UI element class will be here.-->
  </Object>
</ApiMapping> 

For more details, please refer the attached source files or MSDN for UI Automation tutorial.

Disclaimer

Now very limited functionality is implemented in this plug-in framework. Currently, it supports only IExpandCollapseProvider and IValueProvider interfaces. I will gradually update and enhance its capabilities.

This work is patented by SIEMENS on my behalf, so read their online patent disclosure carefully before using this code (partially/fully) in your product.

Points of Interest

1. There is no need to implement any interface on user control classes. It also makes developers' life easy to incorporate the required changes in existing user controls for such functionality.
2. Since there will be a specific XML configuration for each type of user control, Automation-Enabler Plug-in Framework can be reused to expose all types of user controls in the system to the automation testing framework.
3. The risk involved in product stability is also less since minimal code change is required in user control class. Changes are required only to pass the reference to Automation-Enabler plug-in and those will be called only during automation testing, not during normal execution.
4. Use of Reflection to access the APIs of the user control classes.

History

• 28/12/2010 - Updated with screenshots
• 27/12/2010 - Initial post