Introduction
VSTest is a powerful unit testing framework that's tightly integrated into Visual Studio Team System. However, there's still room for improvement, and a great way to improve upon it is to add to the framework using some of the new C# language features.
Background
Test Driven Development (TDD) has been around for a while now. There are numerous Frameworks available in .NET that enable this development methodology, such as NUnit, MbUnit, and VSTest, introduced with Visual Studio Team System 2005.
Recently, TDD has been undergoing a metamorphosis into Behavior Driven Development (BDD). This newer development methodology is a subtle refinement on TDD. In BDD, the emphasis is on specifications, not testing. This is mostly a change in the point of view, not a radical change in the methodology. Instead of tests, you have specifications. Instead of test methods, you have behavior specifications.
The traditional TDD libraries can still be used when following BDD, though the trend is to change the libraries so that they follow the newer terminology. This helps the developer, especially the initiates, to focus on specifying behaviors, instead of focusing on testing. There are a few newer .NET libraries taking this approach, such as NSpec, NBehave, and NSpecifiy.
In an article I've been working on, I was using VSTest and attempting to do TDD. As someone new to this methodology, having always in the past created the code first, and then created unit tests, I was finding the whole concept to be difficult. While researching the topic to determine why, I started to realize I was writing my tests with a focus on the implementation details of the code that didn't yet exist, rather than on the behavior I desired. This is precisely what BDD, I believe, is all about. Thinking in the proper way started to make it easier (I won't yet claim that it's easy) to follow the "test first" way of developing software.
I considered switching to one of the other BDD specific testing frameworks at this point, but I really like the integration of VSTest. The only real differences between the BDD testing frameworks and the TDD frameworks is the vocabulary. Realizing this, I noticed that in VSTest, there's very little vocabulary forced on you. There's the TestClassAttribute, TestMethodAttribute and Assert, for the most part. All of these use vocabulary that BDD would prefer we not use. However, for me, the two attribute names are not that big of an issue, as long as you name your test methods in a manner more inline with specifications. It would be nice to change the attribute names, especially for BDD initiates, but it's not really necessary. The Assert class(es) on the other hand, did cause me some heartburn.
First, from a practical point of view, the Assert class(es) suffer from the simple fact that they are not complete. Though there are a lot of assertions you can make using VSTest, the other frameworks all provide more. Adding assertions to fill out the gaps would actually be awkward. A simple example of this would be when you realize that StringAssert does not have a DoesNotContain method. In order to add this assertion, you'd have to create your own assertion class, and therein lies the problem: what would you call it? Even in its own namespace, you couldn't use the name StringAssert without causing usage to be difficult, because of the reuse of the name. Any other name would be likely to be confusing or overly verbose.
Second, from a BDD perspective, the names of the assertions are already poor. BDD wants your specifications to be written in a "fluent" manner, as close to plain English as possible. Not only does this make the specifications easier to read, but tools can be used to strip out the specifications from the code and produce documentation that is readable by non-developers.
These are the factors that lead to my developing these extensions.
Using the Code
The extensions are provided as a framework that can be easily used alongside the VSTest framework. Simply add a reference to the CodeProject.VisualStudio.QualityTools.UnitTestFramework assembly.
The main class used from this framework is the Specify static class. Two static methods in this class (with logical overloads for specific types) provide the starting point for any specifications: That and ThatAction. These methods return a SpecificationValue, which provides a convenient type for writing extension methods that describe the actual specification.
Extension methods are a new concept in C# 3.5 (and other .NET languages). They are simply static methods that have their first parameter qualified with the this keyword. The compiler will allow you to call these extension methods using a syntax identical to calling a member of the object, even though these are not member functions. For example, there's a member of Specify named ShouldEqual.
public static void ShouldEqual<T>(this SpecificationValue<T> self,
object expected)
{
}
With this extension method, we can write a specification that some property on our class should have a specific value under some given condition by writing very fluent code.
[TestMethod]
public void SomeProperty_should_construct_with_the_value_xyzzy()
{
SomeObject target = new SomeObject();
Specify.That(target.SomeProperty).ShouldEqual("xyzzy");
}
The framework already includes most of the typical specifications found in the other unit testing libraries. However, if you find that you need a new specification, it's easy to create an extension method taking a SpecificationValue as the first parameter, that throws the normal AssertFailedException if the specification fails. SpecificationValue is a generic type, and there are a few rules you should follow when writing your extension methods because of this.
- If the specification should work on any type, make the extension method generic with a generic parameter that's used to specify the type of
SpecificationValue.
- If the specification should take a specific type, or a type derived from that type (or that implements the interface type), make the extension method generic with a generic parameter that's used to specify the type of
SpecificationValue, then use generic constraints to constrain the type.
- If the specification should take a specific type that is a sealed type or a built in type, do not make the extension method generic, but instead specify the exact
SpecificationValue type.
If you look at the code for the framework, you'll find numerous ways in which these rules were applied.
In addition to the Specify class, there are a few specialty classes included in the framework that I've found useful when creating data models. The PropertyChangedWatcher makes it easy to specify that your properties on objects that implement INotifyPropertyChanged should raise PropertyChanged events when modified.
[TestMethod]
public void SomeProperty_should_raise_PropertyChanged_when_modified()
{
SomeObject target = new SomeObject();
PropertyChangedWatcher watcher = new PropertyChangedWatcher(target);
target.SomeProperty = "xyzzy";
Specify.That(watcher).ShouldHaveSeen("SomeProperty");
}
CollectionChangedWatcher provides similar functionality for collections that implement INotifyCollectionChanged.
Finally, TestExtensions provides an extension method, SerializeClone, that helps in testing serializable objects.
[TestMethod]
public void SomeObject_should_be_serializable()
{
SomeObject target = new SomeObject();
string expectedValue = "xyzzy";
target.SomeProperty = expectedValue;
SomeObject clone = target.SerializeClone();
Specify.That(clone.SomeProperty).ShouldEqual(expectedValue);
}
Points of Interest
This framework provides a nice vocabulary for writing specifications when doing BDD, while continuing to use VSTest for its tight integration with the IDE. It does not, however, provide a complete BDD framework, since we're still relying on the VSTest framework for tests, which BDD purists may find distasteful. It would be nice if VSTest provided us more extensibility points, so that we could remove the "test" vocabulary entirely, but currently I do not believe there is any way to accomplish that.
In addition to providing a better specification vocabulary, the framework makes it easier to extend the specifications by using extensions methods, new in .NET 3.5. This is the key to the entire framework.
History
- 12-8-2007: Initial release
