Table of Contents
Introduction
Although developers have been unit testing their code for years, it was typically performed after the code was designed and written. As a great number of developers can attest, writing tests after the fact is difficult to do and often gets omitted when time runs out. Test-driven development (TDD) attempts to resolve this problem and produce higher quality, well-tested code by putting the cart before the horse and writing the tests
before we write the code. One of the core practices of Extreme Programming (XP), TDD is acquiring a strong following in the Java community, but very little has been written about doing it in .NET.
What Are Unit Tests?
According to Ron Jeffries, Unit Tests are "programs written to run in batches and test classes. Each typically sends a class a fixed message and verifies it returns the predicted answer." In practical terms this means that you write programs that test the public interfaces of all of the classes in your application. This is not requirements testing or acceptance testing. Rather it is testing to ensure the methods you write are doing what
you expect them to do. This can be very challenging to do well. First of all, you have to decide what tools you will use to build your tests. In the past we had large testing engines with complicated scripting languages that were great for dedicated QA teams, but weren't very good for unit testing. What journeyman programmers need is a toolkit that lets them develop tests using the same language and IDE that they are using to develop the application. Most modern Unit Testing frameworks are derived from the framework created by Kent Beck for the first XP project, the Chrysler C3 Project. It was written in Smalltalk and still exists today, although it has gone through many revisions. Later, Kent and Erich Gamma (of Patterns fame) ported it to Java and called it jUnit. Since then, it has been ported to many different languages, including C++, VB, Python, Perl and more.
The NUnit Testing Framwork
NUnit 2.0 is a radical departure from its ancestors. Those systems provided base classes from which you derived your test classes. There simply was no other way to do it. Unfortunately, they also imposed certain restrictions on the development of test code because many languages (like Java and C#) only allow single inheritance. This meant that refactoring test code was difficult without introducing complicated inheritance hierarchies. .NET introduced a new concept to programming that solves this problem: attributes. Attributes allow you to add
metadata to your code. They typically don't affect the running code itself, but instead provide extra information about the code you write. Attributes are most often used to document your code, but they can also be used to provide information about a .NET assembly to a program that has never seen the assembly before. This is exactly how NUnit 2.0 works. The Test Runner application scans your compiled code looking for attributes that tell it which classes and methods are tests. It then uses reflection to execute those methods. You don't have to derive your test classes from a common base class. You just have to use the right attributes. NUNit provides a variety of attributes that you use when creating unit tests. They are used to define test fixtures, test methods, setup and teardown methods. There are also attributes for indicating expected exceptions or to cause a test to be skipped.
TestFixture Attribute
The
TestFixture attribute is used to indicate that a class contains test methods. When you attach this attribute to a class in your project, the Test Runner application will scan it for test methods. The following code illustrates the usage of this attribute. (All of the code in this article is in C#, but NUnit will work with any .NET language, including VB.NET. See the NUnit documentation for additional information.)
namespace UnitTestingExamples
{
using System;
using NUnit.Framework;
[TestFixture]
public class SomeTests
{
}
}
The only restrictions on classes that use the
TestFixture attribute are that they must have a public default constructor (or no constructor which is the same thing).
Test Attribute
The
Test attribute is used to indicate that a method within a test fixture should be run by the Test Runner application. The method must be public, return void, and take no parameters or it will not be shown in the Test Runner GUI and will not be run when the Test Fixture is run. The following code illustrates the use of this attribute:
namespace UnitTestingExamples
{
using System;
using NUnit.Framework;
[TestFixture]
public class SomeTests
{
[Test]
public void TestOne()
{
}
}
}
SetUp & Teardown Attributes
Sometimes when you are putting together Unit Tests, you have to do a number of things, before or after each test. You could create a private method and call it from each and every test method, or you could just use the Setup and Teardown attributes. These attributes indicate that a method should be executed before (
SetUp) or after (
Teardown) every test method in the Test Fixture. The most common use for these attributes is when you need to create dependent objects (e.g., database connections, etc.). This example shows the usage of these attributes:
namespace UnitTestingExamples
{
using System;
using NUnit.Framework;
[TestFixture]
public class SomeTests
{
private int _someValue;
[SetUp]
public void Setup()
{
_someValue = 5;
}
[TearDown]
public void TearDown()
{
_someValue = 0;
}
[Test]
public void TestOne()
{
}
}
}
ExpectedException Attribute
It is also not uncommon to have a situation where you actually want to ensure that an exception occurs. You could, of course, create a big try..catch statement to set a boolean, but that is a bit hack-ish. Instead, you should use the
ExpectedException attribute, as shown in the following example:
namespace UnitTestingExamples
{
using System;
using NUnit.Framework;
[TestFixture]
public class SomeTests
{
[Test]
[ExpectedException(typeof(InvalidOperationException))]
public void TestOne()
{
}
}
}
When this code runs, the test will pass only if an exception of type InvalidOperationException is thrown. You can stack these attributes up if you need to expect more than one kinds of exception, but you probably should it when possible. A test should test only one thing. Also, be aware that this attribute is not aware of inheritance. In other words, if in the example above the code had thrown an exception that derived from InvalidOperationException, the test would have failed. You must be very explicit when you use this attribute.
Ignore Attribute
You probably won't use this attribute very often, but when you need it, you'll be glad it's there. If you need to indicate that a test should not be run, use the
Ignore attribute as follows:
namespace UnitTestingExamples
{
using System;
using NUnit.Framework;
[TestFixture]
public class SomeTests
{
[Test]
[Ignore("We're skipping this one for now.")]
public void TestOne()
{
}
}
}
If you feel the need to temporarily comment out a test, use this instead. It lets you keep the test in your arsenal and it will continually remind you in the test runner output.
The NUnit Assertion Class
In addition to the attributes used to identify the tests in your code, NUnit also provides you a very important class you need to know about. The
Assertion class provides a variety of static methods you can use in your test methods to actually test that what has happened is what you wanted to happen. The following sample shows what I mean:
namespace UnitTestingExamples
{
using System;
using NUnit.Framework;
[TestFixture]
public class SomeTests
{
[Test]
public void TestOne()
{
int i = 4;
Assertion.AssertEquals( 4, i );
}
}
}
(I know that isn't the most relevant bit of code, but it shows what I mean.)
Running Your Tests
Now that we have covered the basics of the code, lets talk about how to run your tests. It is really quite simple. NUnit comes with two different Test Runner applications: a Windows GUI app and a console XML app. Which one you use is a matter of personal preference. To use the GUI app, just run the application and tell it where your test assembly resides. The test assembly is the class library (or executable) that contains the Test Fixtures. The app will then show you a graphical view of each class and test that is in that assembly. To run the entire suite of tests, simple click the
Run button. If you want to run only one Test Fixture or even just a single test, you can double-click it in the tree. The following screenshot shows the GUI app:
There are situations, particularly when you want to have an automated build script run your tests, when the GUI app isn't appropriate. In these automated builds, you typically want to have the output posted to a website or another place where it can be publicly reviewed by the development team, management or the customer. The NUnit 2.0 console application takes the assembly as a command-line argument and produces XML output. You can then use XSLT or CSS to convert the XML into HTML or any other format. For more information about the console application, check out the NUnit documentation.
Doing Test-Driven Development
So now you know how to write unit tests, right? Unfortunately just like programming, knowing the syntax isn't enough. You need to have a toolbox of skills and techniques before you can build professional software systems. Here are a few techniques that you can use to get you started. Remember, however, that these tools are just a start. To really improve your unit testing skills, you must practice, practice, practice. If you are unfamiliar with TDD, what I'm about to say may sound a little strange to you. A lot of people have spent a lot of time telling us that we should carefully design our classes, code them up and then test them. What I'm going to suggest is a completely different approach. Instead of designing a module, then coding it and then testing it, you turn the process around and do the testing first. To put it another way, you don't write a single line of production code until you have a test that fails. The typical programming sequence is something like this:
- Write a test.
- Run the test. It fails to compile because the code you're trying to test doesn't even exist yet! (This is the same thing as failing.)
- Write a bare-bones stub to make the test compile.
- Run the test. It should fail. (If it doesn't, then the test wasn't very good.)
- Implement the code to make the test pass.
- Run the test. It should pass. (If it doesn't, back up one step and try again.)
- Start over with a new test!
While you are doing step #5, you create your code using a process called Coding by Intention. When you practice Coding by Intention you write your code top-down instead of bottom up. Instead of thinking, "I'm going to need this class with these methods," you just write the code that you want... before the class you need actually exists. If you try to compile your code, it fails because the compiler can't find the missing class. This is a good thing, because as I said above, failing to compile counts as a failing test. What you are doing here is expressing the intention of the code that you are writing. Not only does this help produce well-tested code, it also results in code that is easier to read, easier to debug and has a better design. In traditional software development, tests were thought to verify that an existing bit of code was written correctly. When you do TDD, however, your tests are used to define the behavior of a class before you write it. I won't suggest that this is easier than the old ways, but in my experience it is vastly better. If you have read about Extreme Programming, then this is primarity a review. However, if this is new to you, here is a sample. Suppose the application that I'm writing has to allow the user to make a deposit in a bank account. Before creating a
BankAccount class, I create a class in my testing library called
BankAccountTests. The first thing I need my bank account class to do is be able to take a deposit and show the correct balance. So I write the following code:
namespace UnitTestingExamples.Tests
{
using System;
using NUnit.Framework;
[TestFixture]
public class BankAccountTests
{
[Test]
public void TestDeposit()
{
BankAccount account = new BankAccount();
account.Deposit( 125.0 );
account.Deposit( 25.0 );
Assertion.AssertEquals( 150.0, account.Balance );
}
}
}
Once this is written, I compile my code. It fails of course because the BankAccount class doesn't exist. This illustrates the primary principle of Test-Driven Development: don't write any code unless you have a test that fails. Remember, when your test code won't compile, that counts as a failing test. Now I create my BankAccount class and I write just enough code to make the tests compile:
namespace UnitTestingExamples.Library
{
using System;
public class BankAccount
{
public void Deposit( double amount )
{
}
public double Balance
{
get { return 0.0; }
}
}
}
This time everything compiles just fine, so I go ahead and run the test. My test fails with the message "TestDeposit: expected: <150> but was <0>". So the next thing we do it write just enough code to make this test pass:
namespace UnitTestingExamples.Library
{
using System;
public class BankAccount
{
private double _balance = 0.0;
public void Deposit( double amount )
{
_balance += amount;
}
public double Balance
{
get { return _balance; }
}
}
}
This time our tests pass.
Using Mock Objects - DotNetMock
One of the biggest challenges you will face when writing units tests is to make sure that each test is only testing one thing. It is very common to have a situation where the method you are testing uses other objects to do its work. If you write a test for this method you end up testing not only the code in that method, but also code in the other classes. This is a problem. Instead we use mock objects to ensure that we are only testing the code we intend to test. A mock object emulates a real class and helps test expectations about how that class is used. Most importantly, mock objects are:
- Easy to make
- Easy to set up
- Fast
- Deterministic (produce predictable results)
- Allow easy verification the correct calls were made, perhaps in the right order
The following example shows a typical mock object usage scenario. Notice that the test code is clean, easy to understand and not dependent on anything except the code being tested.
namespace UnitTestingExamples.Tests
{
using DotNetMock;
using System;
[TestFixture]
public class ModelTests
{
[Test]
public void TestSave()
{
MockDatabase db = new MockDatabase();
db.SetExpectedUpdates(2);
ModelClass model = new ModelClass();
model.Save( db );
db.Verify();
}
}
}
As you can see, the MockDatabase was easy to setup and allowed us to confirm that the Save method made certain calls on it. Also notice that the mock object prevents us from having to worry about real databases. We know that when the ModelClass saves itself, it should call the database's Update method twice. So we tell the MockDatabase to expect two updates calls, call Save and the confirm that what we expected really happened. Because the MockDatabase doesn't really connect to a database, we don't have to worry about keeping "test data" around. We only test that the Save code causes two updates.
- "When Mock Objects are used, only the unit test and the target domain code are real." -- Endo-Testing: Unit Testing with Mock Objects by Tim Mackinnon, Steve Freeman and Philip Craig.
Testing the Business Layer
Testing the business layer is where most developers feel comfortable talking about unit testing. Well designed business layer classes are loosely coupled and highly cohesive. In practical terms, coupling described the level of dependence that classes have with one another. If a class is loosely coupled, then making a change to one class should not have an impact on another class. On the other hand, a highly cohesive class is a class that does the one thing is was designed to do and nothing else. If you create your business layer class library so that the classes are loosely coupled and highly cohesive, then creating useful unit tests is easy. You should be able to create a single test class for each business class. You should be able to test each of its public methods with a minimal amount of effort. By the way, if you are having a difficult time creating unit tests for your business layer classes, then you probably need to do some significant refactoring. Of course, if you have been writing your tests first, you shouldn't have this problem.
Testing the User Interface
When you start to write the user interface for your application, a number of different problems arise. Although you can create user interface classes that are loosely coupled with respect to other classes, a user interface class is
by definition highly coupled to the user! So how can we create a automated unit test to test this? The answer is that we separate the
logic of our user interface from the actual presentation of the view. Various patterns exist in the literature under a variety of different names: Model-View-Controller, Model-View-Presenter, Doc-View, etc. The creators of these patterns recognized that decoupling the logic of what the view does (i.e., controller) from the view is a
Good Thing. So how do we use this? The technique I use comes from Michael Feathers' paper
The Humble Dialog Box. The idea is to make the view class support a simple interface used for getting and setting the values displayed by the view. There is basically no code in the view except for code related to the painting of the screen. The event handlers in the view for each interactive user interface element (e.g., a button) contain nothing more than a pass-thru to a method in the controller. The best way to illustrate this concept is with an example. Assume our application needs a screen that asks the user for their name and social security number. Both fields are required, so we need to make sure that a name is entered and the SSN has the correct format. Since we are writing our unit tests
first, we write the following test:
[TestFixture]
public class VitalsControllerTests
{
[Test]
public void TestSuccessful()
{
MockVitalsView view = new MockVitalsView();
VitalsController controller = new VitalsController(view);
view.Name = "Peter Provost";
view.SSN = "123-45-6789";
Assertion.Assert( controller.OnOk() == true );
}
[Test]
public void TestFailed()
{
MockVitalsView view = new MockVitalsView();
VitalsController controller = new VitalsController(view);
view.Name = "";
view.SSN = "123-45-6789";
view.SetExpectedErrorMessage( controller.ERROR_MESSAGE_BAD_NAME );
Assertion.Assert( controller.OnOk() == false );
view.Verify();
view.Name = "Peter Provost";
view.SSN = "";
view.SetExpectedErrorMessage( controller.ERROR_MESSAGE_BAD_SSN );
Assertion.Assert( controller.OnOk() == false );
view.Verify();
}
}
When we build this we receive a lot of compiler errors because we don't have either a MockVitalsView or a VitalsController. So let's write skeletons of those classes. Remember, we only want to write enough to make this code compile.
public class MockVitalsView
{
public string Name
{
get { return null; }
set { }
}
public string SSN
{
get { return null; }
set { }
}
public void SetExpectedErrorMessage( string message )
{
}
public void Verify()
{
throw new NotImplementedException();
}
}
public class VitalsController
{
public const string ERROR_MESSAGE_BAD_SSN = "Bad SSN.";
public const string ERROR_MESSAGE_BAD_NAME = "Bad name.";
public VitalsController( MockVitalsView view )
{
}
public bool OnOk()
{
return false;
}
}
Now our test assembly compiles and when we run the tests, the test runner reports two failures. The first occurs when TestSuccessful calls controller.OnOk, because the result is false rather than the expected true value. The second failure occurs when TestFailed calls view.Verify. Continuing on with our test-first paradigm, we now need to make these tests pass. It is relatively simple to make TestSuccessful pass, but to make TestFailed pass, we actually have to write some real code, such as:
public class MockVitalsView : MockObject
{
public string Name
{
get { return _name; }
set { _name = value; }
}
public string SSN
{
get { return _ssn; }
set { _ssn = value; }
}
public string ErrorMessage
{
get { return _expectedErrorMessage.Actual; }
set { _expectedErrorMessage.Actual = value; }
}
public void SetExpectedErrorMessage( string message )
{
_expectedErrorMessage.Expected = message;
}
private string _name;
private string _ssn;
private ExpectationString _expectedErrorMessage =
new ExpectationString("expected error message");
}
public class VitalsController
{
public const string ERROR_MESSAGE_BAD_SSN = "Bad SSN.";
public const string ERROR_MESSAGE_BAD_NAME = "Bad name.";
public VitalsController( MockVitalsView view )
{
_view = view;
}
public bool OnOk()
{
if( IsValidName() == false )
{
_view.ErrorMessage = ERROR_MESSAGE_BAD_NAME;
return false;
}
if( IsValidSSN() == false )
{
_view.ErrorMessage = ERROR_MESSAGE_BAD_SSN;
return false;
}
return true;
}
private bool IsValidName()
{
return _view.Name.Length > 0;
}
private bool IsValidSSN()
{
string pattern = @"^\d{3}-\d{2}-\d{4}$";
return Regex.IsMatch( _view.SSN, pattern );
}
private MockVitalsView _view;
}
Let's briefly review this code before proceeding. The first thing to notice is that we haven't changed the tests at all (which is why I didn't even bother to show them). We did, however, make significant changes to both MockVitalsView and VitalsController. Let's begin with the MockVitalsView. In our previous example, MockVitalsView didn't derive from any base class. To make our lives easier, we changed it to derive from DotNetMock.MockObject. The MockObject class gives us a stock implementation of Verify that does all the work for us. It does this by using expectation classes through which we indicate what we expect to happen to our mock object. In this case our tests are expecting specific values for the ErrorMessage property. This property is a string, so we add an ExpectationString member to our mock object. Then we implement the SetExpectedErrorMessage method and the ErrorMessage property to use this object. When we call Verify in our test code, the MockObject base class will check this expectation and identify anything that doesn't happen as expected. Pretty cool, eh? The other class that changed was our VitalsController class. Because this is where all the working code resides, we expected there to be quite a few changes here. Basically, we implemented the core logic of the view in the OnOk method. We use the accessor methods defined in the view to read the input values, and if an error occurs, we use the ErrorMessage property to write out an appropriate message. So we're done, right? Not quite. At this point, all we have is a working test of a controller using a mock view. We don't have anything to show the customer! What we need to do is use this controller with a real implementation of a view. How do we do that? The first thing we need to do is extract an interface from MockVitalsView. A quick look at VitalsController and VitalsControllerTests shows us that the following interface will work.
public interface IVitalsView
{
string Name { get; set; }
string SSN { get; set; }
string ErrorMessage { get; set; }
}
After creating the new interface, we change all references to MockVitalsView to IVitalsView in the controller and we add IVitalsView to the inheritance chain of MockVitalsView. And, of course, after performing this refactoring job we run our tests again. Assuming everything is fine, we can create our new view. For this example I will be creating an ASP.NET page to act as the view, but you could just as easily create a Windows Form. Here is the .ASPX file:
<%@ Page language="c#" Codebehind="VitalsView.aspx.cs"
AutoEventWireup="false"
Inherits="UnitTestingExamples.VitalsView" %>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" >
<html>
<head>
<title>VitalsView</title>
<meta name="GENERATOR" Content="Microsoft Visual Studio 7.0">
<meta name="CODE_LANGUAGE" Content="C#">
<meta name=vs_defaultClientScript content="JavaScript">
<meta name=vs_targetSchema
content="http://schemas.microsoft.com/intellisense/ie5">
</head>
<body MS_POSITIONING="GridLayout">
<form id="VitalsView" method="post" runat="server">
<table border="0">
<tr>
<td>Name:</td>
<td><asp:Textbox runat="server" id=nameTextbox /></td>
</tr>
<tr>
<td>SSN:</td>
<td><asp:Textbox runat="server" id=ssnTextbox /></td>
</tr>
<tr>
<td> </td>
<td><asp:Label runat="server" id=errorMessageLabel /></td>
</tr>
<tr>
<td> </td>
<td><asp:Button runat="server" id=okButton Text="OK" /></td>
</tr>
</table>
</form>
</body>
</html>
And here is the code-behind file:
using System;
using System.Web.UI.WebControls;
using UnitTestingExamples.Library;
namespace UnitTestingExamples
{
public class VitalsView : System.Web.UI.Page, IVitalsView
{
protected TextBox nameTextbox;
protected TextBox ssnTextbox;
protected Label errorMessageLabel;
protected Button okButton;
private VitalsController _controller;
private void Page_Load(object sender, System.EventArgs e)
{
_controller = new VitalsController(this);
}
private void OkButton_Click( object sender, System.EventArgs e )
{
if( _controller.OnOk() == true )
Response.Redirect("ThankYou.aspx");
}
#region IVitalsView Implementation
public string Name
{
get { return nameTextbox.Text; }
set { nameTextbox.Text = value; }
}
public string SSN
{
get { return ssnTextbox.Text; }
set { ssnTextbox.Text = value; }
}
public string ErrorMessage
{
get { return errorMessageLabel.Text; }
set { errorMessageLabel.Text = value; }
}
#endregion
#region Web Form Designer generated code
override protected void OnInit(EventArgs e)
{
InitializeComponent();
base.OnInit(e);
}
private void InitializeComponent()
{
this.Load += new System.EventHandler(this.Page_Load);
okButton.Click += new System.EventHandler( this.OkButton_Click );
}
#endregion
}
}
As you can see, the only code in the view is code that ties the IVitalsView interface to the ASP.NET Web Controls and a couple of lines to create the controller and call its methods. Views like this are easy to implement. Also, because all of the real code is in the controller, we can feel confident that we are rigorously testing our code.
Conclusion
Test-driven development is a powerful technique that you can use today to improve the quality of your code. It forces you to think carefully about the design of your code, and is ensures that all of your code is tested. Without adequate unit tests, refactoring existing code is next to impossible. Very few people who take the plunge into TDD later decide to stop doing it. Try and you will see that it is a
Good Thing.
More Info on the Web
- NUnit
- http://www.nunit.org/
- csUnit
- http://www.csunit.org/ (an alternative to NUnit)
- DotNetMock
- http://sourceforge.net/projects/dotnetmock/
- MockObjects
- http://www.mockobjects.com/
- The Humble Dialog Box
- http://www.objectmentor.com/resources/articles/TheHumbleDialogBox.pdf
