A11Y and I18N Testing of Java based GUIs

Introduction

There are a lot of different approaches to software testing. One can use special verification tools based on Model Checking or Hoare Verification approach. It is also possible to use software that basically generates Unit tests or performs simple testing on the base of specification scripts, etc.

There is still one approach that is widely used – development of the test suites according to the software specifications.

I would not like to argue about applicability of the last approach because many companies still choose this way to test their software and there are plenty of developers who are writing tests right now!

That is why I decided to write this paper and share my experience in this area. The main theme here will be A11Y and I18N testing of Java based GUIs. I am not going to provide any source code examples, as the topic is quite simple.

Please note that in this article, I presume that the reader is aware about the following:

• Software testing
• GUI testing
• A11Y
• I18N
• Unit testing

Information

Anyone who is aware of Java based GUI testing knows about the following tools and libraries that are usually used in this process:

• XTest
• JUnit
• Jemmy

The only problem with A11Y and I18N testing is that it is hardly compatible with Unit testing and the main tool for running tests so called XTest supported only the JUnit based tests. That is why it was a disaster to implement multi-step GUI tests on the base of Unit testing. I am not going to discuss all the troubles that one may meet on this way here but I will mention them below.

Problems of Java GUI Testing

First, let me give the brief description of what A11Y, I18N and Unit testing are:

• A11Y testing is accessibility testing i.e. verification of the ability to access each and any GUI components without a mouse.
• I18N testing is the Internationalization testing, i.e. testing of the correctness of application labels and messages internationalization.
• Unit testing is an approach in which the atomic functionality of the system is tested. It is widely used in Extreme Programming and apart from that, in many other situations. For example, when you want to test a class by testing each of its methods.

Now it may become obvious that A11Y and I18N GUI testing are not compatible with Unit testing. The main reason for that is that any GUI testing is usually divided into the following steps:

1. Perform application start up. Set required application options, open windows required for testing
2. Perform test cases
   1. StartUp test case - common for all test cases
   2. StartUp test case – open some special windows or set test case specific options
   3. Perform test sequence
   4. TearDown test case – close specific windows and unset test case specific options
   5. TearDown test case – common for all test cases
3. Shut down application

On the other hand, Unit testing (to be more precise JUnit and thus Jemmy) presumed that there can be only one common set up for the set of test cases and then all of them are performed one by one with the common shut down.

To make things plainer, note that on each of GUI testing steps, the failure can occur. This may interfere with performing the rest of the steps and thus dramatically influence the testing results.

This was a short and brief introduction to the problems that the GUI test developers may face. In the next section, I am going to describe the approach that I and my colleague, Alexey V. Ivashchenko, had developed for productive A11Y and I18N Java based GUI testing.

Solution to the Problems Stated Above

The solution is quite simple. You need to extend JUnit (Jemmy) so that it will be possible to create common StartUp and TearDown methods. The developer should also be able to add customized StartUp and TearDown methods.

You should also keep in mind that if you use Jemmy (it was unstable and it still may be unstable) you must perform all the SturtUp and TearDown operations exclusively by the API functions, i.e. do not use mouse or keyboard event emulations. This will allow you to avoid test crashes during the StartUp and TearDown operations.

Also an important note should be made for the A11Y testing. Any test should be as atomic as possible, for example, the forward focus cycle (Tab) should be tested in one test case and backwards focus cycle in (Shift-Tab) another. Also pressing of each button and accessibility of each component should be checked in separate atomic test cases. This will increase the tests run-time but will seriously decrease the support needed by the test suite and will also increase its stability. By support, I mean the necessity to modify test cases in case any A11Y bug is found.

These three simple rules will help you to increase stability and maintainability of the test suite.

In the following chapter, I will describe how to extend Jemmy to rapidly implement A11Y and I18N tests.

An Approach to I18N and A11Y Testing of Java Based GUI

The approach described here is incredibly simple but still very effective. I call it “Checkers” approach.

The base idea here is that there is a finite set of atomic checks (AtomicChecker later) that are performed in A11Y and I18N testing. For example, in the A11Y testing usually you have to check for focus on this or that component or for opening of a window after pressing some combination of keys. For I18N testing, it is even more obvious. A main check here is the check for value of the text label that is placed on this or that component (this includes combo boxes and windows’ titles).

So all you have to do is to create a simple AtomicCheck interface with almost one check method that will allow performing the atomic check of the component (here I mean Java Component, i.e. button for example) and then all you have to do is create a set of classes that will implement it. For example, let us consider we have a button on the form.

For A11Y testing, one might need to check that after pushing (Tab), this button will obtain the focus. So you implement the AtomicCheck interface mentioned above by the class ButtonA11YAtomicCheck in the following way. This class should have a constructor that receives the keys combination to be pressed and the ID of the button in the container (our form). The check method derived from the AtomicCheck interface first emulates this keys combination pushing, then searches for the button with the given ID in the obtained Java Container (our form), and then checks that the button has the focus.

For I18N testing, everything will be quite similar except that the constructor should receive the expected label value instead of the combination of keys and in the check method, the corresponding button’s label should be compared with the one provided in the constructor.

Now, if you use your imagination, you will understand that this idea together with the polymorphism of Java can provide you with a relatively small but flexibly extendable set of classes (library later) that will allow you to perform standard checks for A11Y and I18N tests.

To perform test case, you have only to create an instance of AtomicChecker implementer to perform an atomic test. You can also create a list of different instances that will be activated one by one and will perform a complex testing sequence in the scope of one test case (focus cycle check for example).

Of course this is only the general idea and a “power reader” understands that there are a lot of details and nuances that stayed aside of this article, but all of them can be simply and naturally solved in the scope of the proposed paradigm.

The main advantage of the described approach is that if you want to create a set of test cases, you should care only about the StartUp and TearDown methods' implementation. The testing sequence itself will only consist of instantiation of AtomicChecker component(s) with the proper arguments and starting them in the loop that of course should be also unified and carried out to the level of the Checkers library as some kind of performance manager.

Also note that it will be very easy to implement the Checkers library on the base of the API provided by Jemmy.

Conclusion

In this article, I have briefly covered the different approaches to software testing and mainly concentrated on the A11Y and I18N testing of Java GUIs. I displayed several problems that any developer may face during the tests creation and the ways to avoid these problems.

The main aim of this article is to introduce the new approach of Checkers that will allow anyone to speed up A11Y and I18N test cases development and increase the stability and maintainability of the test suite.

References

• Jemmy
• XTest
• JUnit
• A11Y
• I18N

Acknowledgement

I would like to thank Alexey V. Ivashchenko who took part in the development of the Checkers approach.