As software developers, we all have our favorite tools for making us successful. Many are perfect for the job when getting started but are soon outgrown. Others require too much setup and training to “dip your toes in the water” just to simply find out if they are the right tool.
Cucumber JVM is a testing framework that enhances JUnit in order to provide an easier way to start doing Behavior-Driven Development (BDD). The Gherkin language (the language Cucumber understands) allows a software or quality engineer to more easily describe in words scenarios about expected behavior in a software application.
For me, Cucumber allows expression of what scenarios are automated without having to “look under the hood” and read Java code. This short blog about it describes why I use it and tips for how I use it that may be a little different than most.
Applications Evolve & Requirements Change
I believe that specification of software requirements is more of an art form. It would be great if we could ask business subject matter experts (SMEs) what they want the application to do and have them abstractly provide every detail necessary. Unfortunately, every project I have worked on in the past 30+ years that has tried to gather requirements up-front has encountered the inevitable situation of changing or misunderstood requirements halfway through the development process which makes suspect all of the requirements not yet implemented.
As a proponent of the Agile Methodology, I believe the best applications evolve. Requirements have to take shape over time. This concept drives rigid project managers and even stakeholders mad. “How can I plan three years out if I don’t know exactly what requirements are in each release?”
This topic can afford its own blog, but for now let’s simply expect that changes in requirements will happen at all phases of the project. Being able to react quickly and efficiently is what agile development is all about.
So, if five months into a twelve-month project, a requirement that was the premise for the five months of work is refined, how do I make sure changes to already written and tested code don’t cause regression problems?
Why Use Cucumber?
Cucumber and JUnit are tools that allow me automate the important capabilities and scenarios of use for my application. As an agile developer, I have to expect I will refactor code and design in reaction to requirement changes; the unit and integration tests I write give me the confidence that the scenarios tested in the past still work. This fact is why we unit test, but still there is the question of why Cucumber?
For me, Cucumber appeared on the scene when a project I was leading decided to move to JEE6, CDI, and Interceptors. These concepts all reuse services as well as use of Aspect-Oriented Programming to interject business rule behavior into service methods. That is, we decided to use a Java interceptor to ensure that business rules are met on every service method call before the call was ever started.
So how on earth do you test a notion like that? Not only do I need to test that my business logic properly, I need a test environment that will simulate my container behavior so that code a proper integration test.
At that time, there was really only one set of tools that would handle this environment: Cucumber and Arquillian. Arquillian is a test framework from Redhat that allows you to “shrink wrap” a deployable resource that can be used in a test container so that your integration test is really running in a container. How to set this up and make it work is more advanced topic than I will cover here, but starting with an Arquillian Cucumber test environment was not a case of dipping one’s toes in the water; it was more like bobbing for jellyfish.
Regardless, this test need drove me to investigate more about Cucumber as a test tool, which opened my eyes to the many possibilities of a BDD environment.
An example would be great right about now.
So far, this probably sounds like a sales pitch, but let me demonstrate what a Cucumber test looks like. Let us take for example a 1040EZ form.
If we were writing an application for this, we might need a scenario as follows:
Feature: Form 1040EZ
Scenario: Too much taxable interest
Given standard error messages
And Form line 1 contains $30,000.00
And Form line 2 contains $1,501.00
When Form is submitted
Then an error message with id F1040EZ-1 is shown
So let us dissect this scenario. The Feature and Scenario verbiage is simply text used to help describe what we are testing in general and what the specific scenario is about. Given, When, and Then syntax is a common test-by-specification jargon used to distinguish the different parts of a scenario.
- “
Given” is used to describe the setup of the test. - “
When” generally describes the Java method you are testing and the parameters provided to it. Our example has no parameters. - “
Then” is used to describe the outcome of executing the method. Sometimes there are messages; other times there is specific data that is expected; and other cases simply test that nothing broke or changed.
If you take this simple tax form, you might be able to find a number of other scenarios, like:
- Required values not specified
- Contradictory input
- Invalid data, like negative values or text where numbers were expected
- Valid scenarios like refunds and payments
There are a number of cases in this simple tax example. For a more complicated form, there may be hundreds of possible scenarios to test. The art of testing is finding the appropriate, manageable number of tests to write.
For our simple scenario, the Given-When-Then text would be stored in the resources of our Java code as a “.feature” file. All feature files are generally stored together in a set of folders that mimic a Java package structure. In fact, to make testing easy, you create the Java class for testing this feature in a package that matches this folder structure.
The Java class, referred to as a Steps file in Cucumber, defines what Java code to run for each Step, which simply is a Given, When, or Then. A step is matched with a feature step using annotations of @Given, @When, and @Then that have regular expressions arguments that are used to do the matching.
See the Cucumber documentation for better examples and more details on how this all fits together.
A Tip for Writing Cucumber Scripts With STANDARD Values
As noted in the example above, a “Given” step sets up the scenario. In our simple scenario, the first Given seems to hide some important data; specifically, what are the standard messages for the application? More complex scenarios may rely on large sets of pre-defined data.
Cucumber provides ways of listing out given data in tables so you could be very explicit in each “Given” step and show the dependent setup data. However this will make even the simplest of scenarios so verbose that no one would attempt to read them.
In my current project, we have this advanced our integration tests to the point that there are hundreds of lines of setup data. So most of our scenarios start with:
Given STANDARD values for …..
The “….” is replaced with keywords that list which classes of object should be provided with standard values. This makes each scenario succinct and uncluttered, but if a non-developer is looking at the scenario, how do they know what the standard values are?
Our solution to this is to provide a StandardObjects.feature file. This file starts like all the rest with “Given STANDARD values for …” but each scenario has a Then step that shows a table of expected values for that standard object.
In the example above, we had:
Given standard error messages
Using this STANDARD values feature approach, we would reuse that Given and supply a Then as follows:
Then standard error messages would include exactly the following:
| id | message |
| F1040EZ-1 |If taxable interest is greater than $1500, you cannot use 1040EZ |
| F1040EZ-2 |Line X is required |
| F1040EZ-3 |Value on line X is not valid |
Separating out standard value expectations to a separate feature declutters the functional scenarios, but still gives visibility into what the standard values are. However, please note that as standard values change or are added, the feature file must be updated. But I think this is a good thing.
Final Thoughts
CodeProjectBeing able to react quickly and efficiently is what agile development is all about. Cucumber is a tool that allows me to automate the important capabilities and scenarios of use for my application. This blog described why I use Cucumber to assist in that goal, and tips for how I use it. You can also check out another Keyhole blog on this topic here.
Changes in requirements will happen at all phases of a project, but when you have the right tooling in place, it can keep from being painful.
