Using Workflow Foundation and Visual Studio 2008 for Testing Automation

• Download source code - 127 KB

Introduction

In this article, I will show how to employ Windows Workflow Foundation (WF) for testing. Originally, Microsoft aimed WF on describing business processes inside an application. The idea is that we can use this technology to describe the process of testing as well. The tester designs a test using a visual diagram and, what is more important, he can execute this test automatically.

After reading the article, you will learn:

• How to automate your tests using Workflow Foundation
• How to automatically check results of the tests
• How to reuse the tests in other tests you are developing
• How to extend Workflow Foundation to satisfy your specific testing needs

To demonstrate all the "How-Tos", I created TestflowFramework in C# and attached it to the article. I expect you have enough knowledge of WF and C# to say "hello world". The rest of the stuff I will explain below.

Using the TestflowFramework tester, we can design tests on diagrams in Visual Studio 2008 (for brevity I named “Testflow” a diagram with a testing workflow). Blocks on the Testflow diagram correspond to actions of the user, e.g., ‘type a text in the field’ or ‘click the button’. After the diagram is done, the tester can run it and all actions will be automatically simulated against the application.

The solution accompanying this article contains a sample application and its testflows. The sample application manages users list. It supports viewing/adding/editing/removing users. The sample testflow performs the sequence of following actions: clicks ‘Add User’, enters ‘Login’, ‘Full Name’, ‘Department’ fields, and so on. In the next chapters, I will explain how to reuse this test and run it with different parameters multiple times.

This article concentrates on testing Windows Forms applications. Similarly, this approach can be extended in ASP.NET. One could even adopt it to Compact Framework and XNA platforms but with some additional efforts.

How it works

Well, Windows Workflow Foundation is a very powerful technology, and blending with Visual Studio 2008 makes it utterly explosive. Workflow Foundation has a perfectly extensible architecture both at runtime and design time.

At runtime, we need to emulate user actions and apply it to Windows Forms. The current version of TestflowFramework implements several activities: ClickButtonActivity, InputTextActivity, and CloseFormActivity. Anyone can extend this set and add some advanced activities to manipulate grids, viewers, etc.

For design time, I developed designer classes that tune the style of WF components, captions, and enforces some restrictions. Particularly, any testing action is allowed to be added only into a FormScope activity.

FormScope activity is a container element that hosts any testing activities and supplies context for them. In other words, FormScope contains actions that the user can do on a given form. After putting FormScope on the diagram, tester selects a form existing in the application. See picture below:

The picture below shows ClickButtonAction on the FormScope bound to AddUserForm. Note that the clicked button is being chosen from a strongly typed list that is read from AddUserForm through Reflection.

It’s also worth mentioning that the action emulation is done through Reflection. This approach has pros and cons. First of all, if you change the resolution, coordinates, and layout of your form, the tests will work well despite everything because the tests are bound to programmatic names of forms and controls. And this gives a second advantage: when you rename/delete a control, you will be notified by the compiler (yep, at compile time!). The drawback is that you will probably need to develop custom testing activities for some controls. But reward is a high reliability of tests. Notwithstanding, this is not the sole approach. Some cases may require another one (for example, if you want to test Win32 applications).

Writing tests for the application

The sample contains the application managing a simple user list. It has two forms:

As I have told above, our test is a sequence of user actions nested inside a FormScope activity. Every user action is described by a dedicated activity type. In the given version of Testflowframework, I have included four types:

1. ClickButtonActivity - this activity clicks a specific button on the form
2. InputTextActivity - this activity types a specified test into a TextBox control
3. CloseFormActivity - this activity closes the current form; it is equivalent to clicking the Close button at the top right corner of the window
4. ReadProperty - actually, it does not correspond to a user action directly; instead of doing something, it allows to read any control property and save it into the internal property of a workflow for successive checking (it is detailed below in the "Validating Test Results" chapter)

The diagram below shows the sample testflow for adding a user:

In this test button, "Add User" is clicked. Then the Login, Full Name, and Department fields are filled with specified test data. And clicking on the "OK" button commits it eventually.

Now, if you have grasped these basic principles, let's move further.

Reusing a test

As soon as you create a test, you want to reuse it with different parameters. Let’s say we have the “AddUser” test from above. How can we create three users with different Login, Full Name, and Department? We will use a technique of binding values with workflow properties to accomplish this task. See the two steps below:

1. Create a Test Part with parameters.

Create the “AddUser” activity. Open its diagram and add an InputTextActivity. For the TextBox property of this activity, select txtLogin and bind InputText to the workflow property “Login”. See picture below:



Do the same thing for FullName and Department. Finally, you have to have three activities bound to three properties.

2. Reuse the test part and set its parameters.

Create a new workflow and drag-n-drop the “AddUser” test from the toolbox. Then set the values of properties Login, FullName, Department. You may add this test as many times as you need to create any number of users.

Validating test results

At this point, our test has created three users. At least it is due... How can we verify this fact and alert if the results are wrong?

Well, in the given case, we will simply validate the number of users created: we will access the list of users (listUsers control at Main Form) and check the number of items. If it is not equal to 3, we abort the test.

I have developed a custom activity ReadProperty that can be reused to read any property from any control and make its value available for further checking in any of your tests. In the given case, we will read the Items.Count property from the listUsers control:

As you can see, reading data from the application is simple. We should select Control Name located on our form, and then we set the Property Name of this control. Finally, we specify the binding for the Destination value. So the value of listUsers.Items.Count will be saved into the UsersCount property of the current workflow class.

In the next step, we will validate this UsersCount property against a constraint:

Thus we check the condition. In case of failure, we can even specify the reason and it will be written into the log file. On the contrary, if all goes well, the form is gracefully closed after a short delay.

Running the test

The current version uses a very simple approach for test starting. To start the test, you need to start the application with several command line parameters, specifying the testing assembly containing the test and test name itself.

For example:

WinApp.exe -test "TestflowsForWinApp.dll" AddUser_TestFlow TestLog.txt

After the test is done, its result is written into a log:

31.12.2007 2:01 |         Success | AddUsersBatch
31.12.2007 4:18 |         Failure | AddUsersBatch. Reason: User Count is Wrong

The log file is named TestLog.txt and stored in the application folder (e.g., \WinApp\bin\Debug\ for the sample application).

No doubt this simplified approach can be elaborated for particular requirements. Logs could be saved into an XML file. The test launching engine can be implemented in a more sophisticated manner. Notwithstanding, I leave it as simple as possible for the sake of highlighting the primary idea.

Hints

1. Do not use the standard InvokeWorkflow activity to invoke one test from another. This activity starts the specified workflow asynchronously. In most cases, this is undesired behavior.
2. Put your reusable test parts into a separate assembly apart from the assembly with the main tests. The assembly with main tests should refer the assembly with the test parts. In this case, you will see test parts in the Toolbox and can easily drag-n-drop them into the main tests. If test parts would be in the same assembly, Visual Studio does not show them in the Toolbox.
3. You may use standard activities to enrich your tests. For example, use DelayActivity to emulate user thinking.

Extending Workflow Foundation and creating a custom test activity

Now you know how to create a test using TestflowFramework. It is time to learn how to design your own activity type and enrich your tests with new user actions. Below we will go through the process of creating a InputTextActivity that allows to enter test text into a TextBox control:

1. Create an Activity class and inherit it from WindowsControlActivity.
C#

public partial class InputTextActivity : WindowsControlActivity
{
}

We could inherit from the most general Activity class but WindowsControlActivity provides us with some basic functionality required for a testing activity.

2. Declare TextBox property.

According to the Workflow Foundation paradigm, properties should be declared in a special manner as shown below:

C#

public static readonly DependencyProperty TextBoxProperty = 
    DependencyProperty.Register(
    "TextBox", 
    typeof(string), 
    typeof(InputTextActivity), 
    new PropertyMetadata(
        "", 
        DependencyPropertyOptions.Metadata, 
        new Attribute[] 
        { new ValidationOptionAttribute(ValidationOption.Required) }
    )
);

[DefaultValue(""),
Description("TextBox where the text will be typed"),
RefreshProperties(RefreshProperties.All),
MergableProperty(false),
TypeConverter(typeof(ControlTypeConverter)),
Category("Testing")]
public string TextBox
{
    get
    {
        return (base.GetValue(TextBoxProperty) as string);
    }
    set
    {
        base.SetValue(TextBoxProperty, value);
    }
}

Every property should be accompanied by a static field of DependencyProperty type. This field plays the role of a metadata container and helps Windows Workflow do many useful services for us.

You could also note the TypeConverter(typeof(ControlTypeConverter)) attribute. This is a very important point for extending design time behavior. ControlTypeConverter begins work when our Activity is selected on a diagram and the user tries to set the value of the TextBox property in the Properties window.

3. Implement type converter.

A type converter provides the designer with a list of values and the user may select one of them from a dropdown list. ControlTypeConverter is a really simple class that you can see as in the following code snippet:

C#

protected class ControlTypeConverter : TypeConverter
{
    // Methods
    public ControlTypeConverter()
    { }
    public override TypeConverter.StandardValuesCollection 
           GetStandardValues(ITypeDescriptorContext context)
    {
        IControlFieldProvider provider = null;
        object[] instance = context.Instance as object[];
        if ((instance != null) && (instance.Length > 0))
        {
            provider = instance[0] as IControlFieldProvider;
        }
        else
        {
            provider = context.Instance as IControlFieldProvider;
        }
        ICollection values = new object[0];
        if (provider != null)
        {
            values = provider.GetPropertyValues(context);
        }
        return new TypeConverter.StandardValuesCollection(values);

    }
    public override bool GetStandardValuesExclusive(ITypeDescriptorContext context)
    { return true; }
    public override bool GetStandardValuesSupported(ITypeDescriptorContext context)
    { return true; }
}

The interface IControlFieldProvider used in the type converter is implemented in the WindowsControlActivity class. It reads all form fields through Reflection. Then it puts a field name into the value list if it has a type of Control:

C#

protected Type filterType = typeof(Control);
ICollection IControlFieldProvider.GetPropertyValues(ITypeDescriptorContext context)
{
    StringCollection strings = new StringCollection();
    if (this.FormType != null)
    {
        BindingFlags flags =     
            BindingFlags.NonPublic | 
            BindingFlags.Instance | 
            BindingFlags.GetField;

        foreach (FieldInfo info in this.FormType.GetFields(flags))
        {
            if (!info.IsSpecialName)
            {
                if (info.FieldType == filterType || info.FieldType.IsSubclassOf(filterType))
                    strings.Add(info.Name);
            }
        }
    }
    return strings;
}

Sticking to this pattern, you can declare any property with any predefined value.

4. Declare the InputText property.

Now it is time to move to the second property InputText. This property does not have predefined values and may accept any string value. But it has another useful feature: it is bindable. First of all, we have to prepare the infrastructure for bindable properties. To do that, we declare a special property:

C#

public static readonly DependencyProperty ParametersProperty = 
    DependencyProperty.Register(
        "Parameters", 
        typeof(WorkflowParameterBindingCollection), 
        typeof(InputTextActivity), 
        new PropertyMetadata(
            DependencyPropertyOptions.Metadata | 
            DependencyPropertyOptions.ReadOnly
        )
    );

public WorkflowParameterBindingCollection Parameters
{
    get
    {
        return ((WorkflowParameterBindingCollection)
                  (base.GetValue(ParametersProperty)));
    }
}

... and initialize it in the constructor with the following line:

C#

base.SetReadOnlyPropertyValue(ParametersProperty, 
        new WorkflowParameterBindingCollection(this));

The binding infrastructure is ready. Declare the property:

C#

public static DependencyProperty InputTextProperty = 
    DependencyProperty.Register(
        "InputText", 
        typeof(string), 
        typeof(InputTextActivity)
    );
    
[Description("Text entered into the text box")]
[Category("Testing")]
[DesignerSerializationVisibility(DesignerSerializationVisibility.Visible)]
public string InputText
{
    get
    {
        return ((string)(base.GetValue(InputTextProperty)));
    }
    set
    {
        base.SetValue(InputTextProperty, value);
    }
}

5. Implement runtime behavior.

Well, this point is very simple. In the general case, we would override the standard Execute method. But it has been done already in the parent class WindowsControlActivity. This class performs some preparation routines required for proper working with windows controls during the test. It also defines the DoControlActivity method that should be overridden by the inheritor. And we do that:

C#

protected override void DoControlActivity(Form form)
{
    TextBox box = GetControl(form, TextBox) as TextBox;
    box.Select();
    box.Text = InputText; 
}

Our testing activity selects the specified TextBox and "types" text there.

Bottom line

Congrats! We have extended Workflow Foundation and made our test activity work.

We declared a class inherited from WindowsControlActivity (or Activity). Then we declared two properties: first one had a predefined list of controls and the second one was a bindable string property. All declarations were made by a specific pattern required by WF. Finally, we defined runtime behavior through overriding the DoControlActivity method (or Execute method if we need to do something general).