Introduction
This unit testing framework consists of about 125 lines of code in a single header. It aims to be the simplest way to get started writing unit tests for C++ developers. Being simple, it is also easy to customize. Many unit testing frameworks require linking to a separate library or require jumping through several hoops just to get started. This can make it more difficult to begin writing tests.
One of the situations developers often face when they set out to write unit tests is that they're already working on a project. If the project isn't broken out into independent libraries, it can be difficult to write stand-alone unit tests. In addition, many of the core functions of a program simply can't be broken out into a separate test executable. Ideally, a developer should be able to write a set of unit tests, include it with a single function call within the program itself, and easily #define the test out later on.
This may not seem like a strong approach to software engineering. However, it is generally acknowledged that testing early and often is better than the alternative. By minimizing the effort required to get started writing tests, the goal of testing early can be more easily accomplished. As the test suite grows and the project moves forward, the tests can be factored into a separate library or executable as time permits.
Background
I started my quest by looking for existing solutions. There is a lot of interest and activity around the xUnit frameworks within the unit testing community. It originated with Smalltalk's SUnit framework, which inspired the developers of JUnit. This, in turn, inspired the creators of NUnit, CppUnit, and several other similar frameworks.
One of the things that most of these frameworks have in common is that they are built using languages that support some reflection capability. This makes it easier to assign attributes to a test function or a setup function and have it automatically included in a test run. C++ developers are not so fortunate. C++ developers must do things more manually (or resort to templates or macros). This isn't a big deal, though. It's what we expect.
Many articles have been written on unit testing in the xUnit community and the various libraries available. However, there is surprisingly little written about options for C++ developers. One such article at Games from Within offers a survey of some of the frameworks available.
For an introduction to unit testing in general, have a look at one of the comprehensive articles here at The Code Project.
Design
After evaluating several frameworks, I decided that none of them met my basic criteria of being extremely simple to use and modify. I decided to see how hard it would be to write a framework that would fit into a single header and would consist of as few lines of code as possible. Here is a list of my basic design criteria:
- Fits into a single header file (no source modules or libraries)
- No longer than a couple of hundred lines of code
- Easy to modify and extend
- Re-routable message output
- Optional macros
- No templates
- No dynamic memory allocation
- Usable in an embedded environment
- Usable on down-level C++ compilers (no fancy C++ features)
The design constraint that it should not dynamically allocate any memory would allow tests to be created on the stack. This would make it easy to write a simple main program entry point and just declare and run the tests all at once without worrying about cleanup or memory leaks.
A consequence of these constraints was that a class would be required for each test. The alternative of using a function pointer wouldn't allow for chaining tests within a suite without allocating memory. Also, it didn't seem in the spirit of C++ to use function pointers.
Using the Code
To get a test up and running, three things are necessary:
- A test case must be written
- A test suite may be written
- The test suite and test case must be added to a runner class and then called
We'll follow this sequence in the illustration. The sample included for download is different.
Writing a Test Case
In this example, we derive our test case from the TestCase base class. TestCase, like the other components of the framework, is a struct. This helps us avoid a lot of public access specifiers.
Test code is added to the single test method. The TestSuite class contains any data that is shared across test cases and it is passed to every test call. The name method is used to provide meaningful output in the event of a test case failure.
struct TestAccountWithdrawal : TestCase
{
const char* name() { return "Account withdrawal test"; }
void test(TestSuite* suite)
{
TestAccountSuite* data = (TestAccountSuite*)suite;
data->account->Deposit(10);
bool succeeded = data->account->Withdraw(11);
T_ASSERT(succeeded == false);
T_ASSERT(data->account->Balance() == 10);
}
};
Adding a Test Suite
The test suite contains a group of related tests. It serves the purpose of both the test suite and test fixture in some other unit testing frameworks.
The two key methods (both optional) are setup and teardown. Each call to a test case is framed with this call pair.
struct TestAccountSuite : TestSuite
{
const char* name() { return "Account suite"; }
void setup()
{
account = new Account();
}
void teardown()
{
delete account;
}
Account* account;
};
Putting It All Together
Once a test suite and at least one test case have been written, they may be added to a runner and executed.
#include <stdio.h>
#include "shortcut.h"
#include "tests/account.h"
int main(int argc, char* argv[])
{
TestRunner runner;
TestAccountSuite accountSuite;
TestAccountWithdrawal accountWithdrawalTest;
accountSuite.AddTest(&accountWithdrawalTest);
runner.AddSuite(&accountSuite);
runner.RunTests();
return 0;
}
One of the benefits of this lightweight system is that all test code may be kept in headers. This avoids some duplication between a separate class declaration and implementation. Because the unit test framework is implemented in a single header, only a single driver module (containing main, for example) is required.
This system also makes it easy to add tests to an existing application. For example, the tests could be called at program startup within a #ifdef DEBUG section. In release mode, no trace of the tests would exist in the application binary. This may not be the case when linking to other unit testing libraries.
Obviously, this is not a long-term solution. It is a good way to get started, though. Developers can start writing tests immediately and the tests can be factored into a separate executable when time permits.
Internals
This section may be skipped. It explains a little bit about how the (very few) moving pieces work.
Base Classes
All test cases derive from a very basic class, called TestCase.
struct TestCase
{
TestCase() : next(0) {}
virtual void test(TestSuite* suite) {}
virtual const char* name() { return "?"; }
TestCase* next;
};
The class has a name accessor method, which is used for logging errors. The test suite uses the next pointer to chain the test cases into a linked list. The test itself is implemented in the virtual test method.
The test suite has the same structure, except that it contains a list of tests and has different methods to override: setup and teardown.
struct TestSuite
{
TestSuite() : next(0), tests(0) {}
virtual void setup() {}
virtual void teardown() {}
virtual const char* name() { return "?"; }
void AddTest(TestCase* tc)
{
tc->next = tests;
tests = tc;
}
TestSuite* next;
TestCase* tests;
};
As stated earlier, the test suite class plays the same role as the test suite and test fixture classes in other frameworks. In such frameworks, suites often play the role of a test grouping construct while the fixture provides a setup/teardown mechanism. Since ShortCUT is such a simple framework, there was no need to create this additional level of complexity. If a development team needs this feature, it may be easily added as a customization.
The Runner
The test runner is the heart of the system. It, too, is very straightforward. The main routine, RunTests calls RunSuite for each suite.
struct TestRunner
{
...
void RunSuite(TestSuite* suite, int& testCount, int& passCount)
{
TestCase* test = suite->tests;
while (test)
{
try
{
suite->setup();
test->test(suite);
passCount++;
}
catch (TestException& te)
{
log->write("FAILED '%s': %s\n", test->name(), te.text());
}
catch (...)
{
log->write("FAILED '%s': unknown exception\n", test->name());
}
try
{
suite->teardown();
}
catch (...)
{
log->write("FAILED: teardown error in suite '%s'\n", suite->name());
}
test = test->next;
testCount++;
}
}
...
}
The key points to note here are that, first, the log class can be implemented and set outside of the framework. This makes it easy to display results to another output target, such as a window.
The second point, which can be an annoyance, is that the test suites and test cases are chained together in singly-linked lists. This means that they are traversed and executed in LIFO order. This is the reverse from the order in which they were added.
It would be a simple matter to customize the framework to fix annoyances like this. I chose not to, since the goal was to make the framework as simple as possible.
Customization and Conclusion
The main goal of the framework was to have the absolute simplest system possible, within the design requirements and constraints. Every line of code was scrutinized for its value. In some cases, such as with the TestLog class, a few lines were added because they helped to meet a design requirement. Even though the framework would have been simpler, it would have lost basic flexibility.
The header is about 200 lines of code. A quarter of the code is actually unnecessary. It was included as an example of how to implement custom assert functionality through exceptions and how to implement a couple of helper macros to avoid repetitive code.
The framework is useable in its basic form. It is hoped that it will form the basis of systems that are tailored to the needs of the developers who use them (instead of the other way around). It should provide enough utility to get going quickly, and its basic structure should make it easy to modify, customize, and extend going forward.
History
- 15th February, 2007: Original article
