Introduction
Following on from my previous article which gave an introduction to configuring a build using Team Foundation Services 2015, I'm going to continue and demonstrate how to add unit tests to your build process. If you haven't already read my first article then please feel free to do so before continuing with this one.
As with my previous article, I want to keep this article nice and straight forward. To do that I will use the unit testing framework that is shipped with Visual Studio 2015. However the concepts can be equally applied to other unit testing frameworks such as Nunit.
Background
It is assumed that the reader has some basic knowledge of Team Foundation Services and of the basic principles of building software and unit tests.
Creating our unit tests
In order to add unit tests to our build, we firstly need to create some unit tests. The beauty of unit testing is that the exact same tests that you write during the development process (whether you follow Test Driven Development or not) can be added to the build process. So there is no duplication of effort. All tests written during the development cycle are then executed as part of the build process.
In Visual Studio open your solution. Then right-click on the solution and select Add then New Project.... Select the language of your choice from the filters on the left hand side (Visual C# for example) and select Test. You should now be presented with a list of project templates for testing as in the screenshot below.
For the purposes of this article I will create a C# test project but feel free to select the language of your choice. Select Unit Test Project and enter a suitable name and location for the project and then click OK.
When the new project has finished being created, you will find a default Program.cs file in the project with a skeletal structure already in place for creating your unit tests. You can either continue to use this template or add a new file from scratch. It doesn't matter either way. You may find it easier to use the default class or you may prefer to create one from scratch.
The important parts of the class to note are as follows.
- The reference to the assembly Microsoft.VisualStudio.TestTools.UnitTesting (or other unit testing assembly if using another unit testing framework such as NUnit)
- The class is decorated with the
[TestClass] attribute - Each method needs to be public, have a return type of void (or equivalent if using a different language) and be decorated with the
[TestMethod] attribute - Each method needs to exercise the necessary assertions to provide adequate unit testing coverage
Other than the [TestMethod] decoration for your test methods, there are a couple of other method decorations that you may find useful.
[Ignore] - use this is prevent a test method from being executed by the test runner. This can be useful when you want to (possibly temporarily) disable a particular test without having to delete it.[ExpectedException] - use this to test for exceptions. Often in structured exception handling it can be difficult to test the exceptions i.e. where the code drops into the catch section of the code. With a unit testing framework it is simple. Using the [ExpectedException()] decoration you can test for any exception e.g. [ExpectedException(typeof(ArgumentException)))]
A full and detailed discussion on what constitutes a good set of unit tests is beyond the scope of this article. However I can fully recommend Roy Osherove's book The Art of Unit Testing. This is probably the only book on unit testing you will ever need (it's certainly the only book I own on the subject).
Before proceeding and adding the unit tests to the build, be sure that they all run and pass (you can force them to fail later if you want your build to take corrective action if a unit test fails during a build).
Adding our unit tests to our build
First of all you'll need to open your TFS 2015 home page in your browser of choice. The URL should be similar to http://tfsroot:8080/tfs and you ought to see something similar to the screenshot below. This is the TFS 2015 dashboard.
Select the desired project you want to add tests to and you will be presented with the TFS project dashboard similar to the screenshot below.
I am going to assume that there is already a build created and that we are only concerned with adding unit tests to it. For instructions on how to create a TFS 2015 build see my previous article.
So let's start by clicking on the Add build step... selecting the Test filter on the left hand side and selecting Visual Studio Test option and clicking the Add button to the right. We should now be presented with an empty Visual Studio Test step as in the screenshot below.
Add the necessary information to your build step as in the screenshot below. Continuing with the theme of keeping this article simple the only one you need to enter for the time being is Test Assembly. Using the elipsis select the project you created previously (which contains your unit tests).
You will notice in the screenshot that I have added a value for Run Settings File. This is a file that you can use to customise how your tests are executed. You can specify the Platform, Configuration and many other aspects of your unit test execution. I have found this to be incredibly useful and it is well worth having a look into these in more detail. Here is a useful article on configuring your unit tests via a .runsettings file.
In the example .runsettings file below I am simply telling the test runner where to place the output from the unit tests. This is so that the test results can be published (which I will cover in just a moment). In order to publish your test results you need to know where they have been saved, hence why I have configured this location using a .runsettings file. It is best to add your .runsettings file to your version control system (which in our case is TFS 2015).
="1.0"="utf-8"
<RunSettings>
<RunConfiguration>
<ResultsDirectory>.\TestResults</ResultsDirectory>
</RunConfiguration>
</RunSettings>
Publishing our test results
In the previous step we ran our unit tests and additionally configured this so that those results got published to a known location using a .runsettings file. We can now publish those results so that they are visible to our build process and can be displayed on the build results dashboard after our build has completed.
Click on the Add build step... select the Test filter on the left hand side and select Publish Test Results option and click the Add button to the right. We should now be presented with an empty Publish Test Results build step.
- For Test Result Format select VSTest
- In the Test Results Files textbox enter the name of the folder you added to your .runsettings file previously with the
*.trx option. This is required because each time your unit tests are executed they are published to a filename that contains the exact date/time they were created. So it's impossible to know the filename in advance. Using the *.trx option ensures your build step can locate the output from the test runner.
If you don't want to wait for a code check-in to trigger a build, you can simply force a build to the queue. To view the results from your test step, go to the list of completed builds and select the last build.
On the right hand side of the build results dashboard you will see a link to the Test Results. Click on the link in blue to open the VSTest Test run results.
The VSTest Test run results provides detailed coverage of the tests that were run and their results.
The VSTest Test run results are also summarised by a series of graphs that are easy to read and can be used by software managers, project managers etc who want to see a top level view of the tests.
Summary
That's all there is to it. There is a great deal more that you can do with TFS 2015 with regards to testing that has not been covered in this article. However, I hope this has given you sufficient information for you to create your own unit tests and add them to your build. Feel free to leave a comment if you would like me to further elaborate on anything within this article.
