TestRail: A Test-Case Management System

Introduction

A few days ago, I searched for a test case management tool that could be integrated with our bug-tracking system (we use Redmine). A new product’s quality is dependent upon how well it was tested before its release. Uncaught bugs can mean lost clients, so efficient testing is a top priority. There are numerous of test-case management systems on the market but only a few fits my requirements:

• Inexpensive
• Integrates with Redmine
• Web 2.0 Interface

I decided to share my experience with CodeProject customers.

Our Requirements

In our company, testers do the following:

1. Make up a test scenario
2. Make up test plans that include sets of test cases
3. Perform test cases according to test plans for each new product version
4. Correct test cases according to new product versions and functionality

Many people still use Excel to store test cases and test results, but that can lead to problems. So in addition to the test case management system, we use Redmine as a bug-tracking system.

On a daily basis, our testers perform hundreds of manual test cases; if the case fails, the tester adds a new bug to the project in Redmine. If the test fails again on the next program version, the tester might duplicate the bug, forgetting that this bug was already described.

To prevent this duplication, a test case management system must provide a tie between test cases and bugs. That way, should the test case fail, the tester will see that the problem has already been described, and will not duplicate the bug. This is one of the reasons why it is necessary to abandon test cases in Excel in favor of a good test case management system.

TestRail

We considered many test case systems, including some free ones: TestLink, Klaros, QaTraq, Rth, Testopia, etc., but after a long search, we chose TestRail.

The main reason for our choice, in addition to a simple and convenient interface is TestRail’s connection with Redmine. Some of the other programs also had links to Redmine, but were unacceptable due to issues like complicated interfaces, problematic installations, a higher price, bugs, or compatibility issues with different Redmine versions.

Using URLs, TestRail integrates with Redmine, simply and effectively. While this system is not 100% automated, it works. It’s easy to use, it’s quick to set up, and it usually remains unaffected by newer versions of Redmine.

TestRail’s system requirements:

• Windows or Linux Server
• ionCube PHP Loader extension (learn more)
• Sqlsrv or mysql PHP extension for database access
• cURL PHP extension 

Quick Start  

1. Сreate a demo using the above link
2. Configure TestRail to work with Redmine. Enter the URLs of your Redmine installation under Administration> Site Settings> Integration Tab. Use the following URLs to integrate TestRail with Redmine:
   
   
   
   • Defect View Url: http://your-server/issues/%id%
   • Defect Add Url: http://your-server/projects/[projectid]/issues/new
   
   
3. Create a project, test scenarios, and add 10-15 test cases
4. Click the Run tab and execute your test cases.
5. Let's assume that test case # 33 failed. On the “Add Test Result” window, click on the Add link.
6. A new browser page with a “New Issue” window in your Redmine project will appear
7. Describe the bug and save it as a number, for instance #750.
8. Go back in our window to the “Add Test Result” window and add #750 as Defect.
   

   You will have just created a relationship between a test case and a bug. Now, when the test-case identifies the bug again, you will avoid registering the bug a second time.

   
   

Conclusion

TestRail impresses with its simplicity and clarity, but it still has all the necessary functionality.

By obtaining TestRail, we have created comfortable conditions for our testers, giving them a very simple, convenient, and functional tool. The program also offers tools for analysis and protection against information loss.

Testers can now run more tests while exercising full control. They will have the ability to analyze the QA process, too. This means software designers can create a higher quality, bug-free product resulting in satisfied and happy customers.