MsTest 101: Using the MsTest Runner

Introduction

This article is an introduction in Testing and in particular, the inside workings of MsTest. This article is an entry level 101 with more detailed and indepth articles coming your way soon to make the most out of the test runner.

The article will show you how to set up a [TestClass] and [TestMethods] to kick start your testing efforts with MsTest. Microsoft's Test Runner which ships with .NET Framework and .NET Core.

Background

There are many test runners out in the marketplace, however, for now, this article will focus on MsTest, but you can apply the same principles and learnings to any other unit test runner as they all offer very similar experiences.

Let's Dive In!

There is a solution I've put together to support this article for everyone to use. It's currently hosted on GitHub and can be found at the below link. If you want to download the solution, or clone the solution, this will really help practically when reading the article. The source code is there for you to use and keep. If you have any questions or suggestions, please make a pull request or comment at the bottom of the article.

• Code can be downloaded or cloned: GitHub

At this point, I'm hoping you're excited to see some Tests using MsTest? And by now, hopefully you'll have the source code to follow along with the article. Your built solution should hopefully look similar to the below, 9 tests and a successful build.

Note: To see the Test Explorer in Visual Studio, remember this is under the menu structure: Test -> Windows -> Test Explorer.

What are we waiting on .... let's get testing!

Test Class vs Test Method

[TestClass] is an attribute which is added to a normal C# Class file. MsTest uses the [TestClass] attribute to find all tests within your solution and are available to be executed. Without this attribute of [TestClass], you can write 1000s of [TestMethod] but the Test Runner will never find any tests and therefore be able to execute your tests, we love testing so we want to ensure our test can be seen and run.

The Test Class Attribute is added to the top of a class when you create a unit test in Visual Studio, however if you add a plain C# Class file into a Solution, you'll need to always remember to add this if the intent of that class is a Test one.

[TestMethod] is an attribute which is required on a method to make it seen and executable as a test by MsTest. Every test method requires you to add the [TestMethod] attribute. Without this attribute, the compiler just reads this as a normal C# method and that is not what we want. We want test methods to be executable by the test runner, hence we add the attribute.

If you open up the solution, and the SimpleUnitTest.cs as shown below, notice that our class has the [TestClass] attribute on it, and each test method has the [TestMethod] decorated. Easy you say.... you'll never forget to do this I hear you say, one day you may as we all have. If you bring it back to basics, you'll always find the issue.

I once forgot to add [TestClass] to my Test Class, oops! When I built the solution, I expected a few hundred data driven tests to show to me, I ended up with ZERO showing as I forgot that little attribute. Once I added it on top of the class and rebuilt the solution, hey presto - hundreds of tests showed up!

Simplest Test

I'm always a big fan of keeping things simple and starting from the beginning, so our easiest test in this 101 article is named, you guessed it, the name the first test to look at is:

[TestMethod]
public void Simplest_Test()
{
     const int one = 1;
     Assert.AreEqual(1, one);
}

The above is the simplest test I could think of, does 1 = 1... if you run the Test in the solution, it should turn green, otherwise my maths skills are seriously damaged. This test demonstrates a few things to us so we can get used to the test runner:

• We use [TestMethod] as the header of the method, which in turn can be read as test by MsTest.
• The Test Method has no return type always just void. Test Methods do not need to be string, int, bool or anything else as we do not return from the this method, we Assert in the method.
• Assert is a keyword in MsTest for Microsoft's assertion framework. This can be replaced if required with another assertion framework such as Shouldly, however for this solution, we are focusing on MsTest in full.
• These sorts of tests take milliseconds to execute.

Now you've seen the simplest test you'll ever see for the reason of showing you how the attributes and assertions work, we will move to another few tests. The following tests and sections will build upon MsTest Runner attributes and ability, should you wish to use them.

Age and Name Tests

This is a test which creates a new person call "Bob" who is 25 yrs old. The test creates a new instance of Person and passes in the person's name and age. Then the person created is tested to ensure the properties for Age are being set correctly. If this constructor of Person was not set up correctly and had a bug whereby it always set the age to be 100 regardless of what you passed in, then the test would fail and that'd be a good failure as shows a bug.

[TestMethod]
public void Age_Test()
{
    var bob = new Person("Bob", 25);
    Assert.AreEqual(25, bob.Age);
}

The name test below exercises the same sort of test just checking Bob is called Bob. If we have more than 1 assertion in a test, let's say Age Assertion first, and then the Name Assertion, if the Age assertion fails, then the Name assertion won't be exercised. This is something we like to do - add many assertions per test as the first failure can hide other failures from view.

[TestMethod]
public void Name_Test()
{
    var bob = new Person("Bob", 25);
    Assert.AreEqual("Bob", bob.Name);
}

Data Driven Row Tests

The following tests are written once, but run multiple times. In MsTest, there is an attribute which can be added to the Test method called DataRow. Now this DataRow are arguments which we pass into our test, and data drive them. If we have 5 data rows on a Test Method, the test will execute 5 times with the inputs described in the Test Row.

Examples can be seen below and executed in the solution, they really are a great way for data driving simple tests to clearly see inputs and outputs of your test. I use them once in a while for my testing solution when a DataSource such as an Excel document, or database is a little overkill for the data I'm trying to exercise in the test.

[DataRow("Bob", 25)]
[DataRow("Bill", 35)]
[TestMethod]
public void Data_Row_Age_Test(string name, int age)
{
    var bob = new Person(name, age);
    Assert.AreEqual(age, bob.Age);
}

[DataRow(57)]
[DataRow(-100)]
[DataRow(21)]
[TestMethod]
public void Data_Row_Number_Test(int number)
{
    Assert.AreEqual(21, number);
}

The above tests are all data driven via the DataRow attribute. Notice that if I pass a string in as the first argument "Bob", my method needs to have a string parameter, likewise I pass an int of age therefore my method needs that parameter to accept the int. So you end up with your test method allowing parameters. This works on simple data structures and I'd say go explore and see if this works for you, your needs and if you need more complex structures, we'll discuss those as well in another topic.

Maybe next week's article will be the MsTest DataSource ones where we talk about Databases, Excel Docs, JSON Files and anything you want!

Exception Tests

So as Devs / QAs, we do not like exceptions, right? Well, in some cases, we want to throw exceptions and let something else consume these exceptions to handle for the user. In this instance, where we want to raise an exception, i.e., negative test we want to ensure our application / program throws the correct exception. How do we assert that an exception is thrown you might ask as it seems pretty hard to do?

MsTest makes it easy with an attribute of [ExpectedException]. What this does is override any Internal Assertions we may have and say, the tests assertion is to expect an exception of type X. Now if an Exception is not raised by the code under test, then the test method fails as it should do, if an exception is raised but is of the wrong type, then again the test fails as it should. Take a look at the below examples and see how useful this can be, we've all tried to handle exceptions at one stage or another but MsTest with this attribute allows us to test effectively wanting to throw exceptions... pretty nice for sure!

[TestMethod]
[ExpectedException(typeof(NotImplementedException))]
public void Expected_Exception_Test()
{
    // Do Something Arrange / Act

    //Expect an exception from the Act, no need to throw it like this usually as its a demo.
    throw new NotImplementedException("This is showing an exception can be a valid test result");
}

[TestMethod]
[ExpectedException(typeof(DivideByZeroException))]
public void Expected_Exception_Failure_Test()
{
    // Do Something Arrange / Act

    //Expect an exception from the Act, no need to throw it like this usually as its a demo.
    throw new NotImplementedException();
}

When you run the above tests in the solution, notice that the second test does not pass, it fails. The reason is it throws a NotImplementedException() however the test expects a DivideByZeroException to be thrown. Trying to write this test ourselves is messy and MsTest allows us to do this in a smooth in.

Timeout Tests

We've all be testing where everyone has an opinion on the speed of tests. Opinions though do not mean much without facts to me, if you say to me that feels slow, I'll ask why and please quantify slow to you? If an API test takes 10ms or 20ms, is that slow? Does a Selenium Test take 3 seconds or 60 seconds, is that slow? It's all about non-functional requirements.

QAs and Devs both suffer I feel from not having enough data to challenge or say actually it's within the Non-Functional Requirements. Let's say we have a Rating engine and we expect it to return a result quicker than 100 milliseconds, how do we tell our test to time this?

Add a stopwatch, get a stopwatch in hand, blink :-)... Thankfully with the below 101 item I'm running through today, you can add a timeout to your test in 2 seconds... or maybe 3... or 4.

Notice that MsTest exposes a Timeout attribute we can use and add to our test methods which is really cool. See below and notice the first test has a timeout and will fail if it doesn't execute within 1 second, and the next time we say actually you have infinite time. Notice test times are in milliseconds so if you want 4 second test max or fail, change 1 to be 4000 as this is milliseconds.

[TestMethod]
[Timeout(1)] // Milliseconds
public void Timeout_Test()
{
    var bob = new Person("Bob", 25);
    Assert.AreEqual(25, bob.Age);
}


[TestMethod]
[Timeout(TestTimeout.Infinite)]
public void Timeout_Infinite_Test()
{
    var bob = new Person("Bob", 25);
    Assert.AreEqual(25, bob.Age);
}

Points of Interest

Hopefully, this has been informative and shows you the first few steps in setting up tests using MsTest, all principles however are shared across other test runners, so if you visit their site:

• TestClass
• TestMethod
• Our First Test
• Data Driving Tests using Data Rows
• Expecting Exceptions and how to handle those
• Adding Timeouts so your tests fail if they are too long running.
• Note you can use a Timeout, Exception, and DataRow all on the same test, you don't need only 1 at a time. These can all be stacked meaning you have a really powerful test runner and testing framework to start getting involved.

I'd love to get any questions or feedback, and appreciate anyone who stuck me to this point! Happy dev / testing!!!