.NET Code Coverage with JetBrains dotCover

Introduction

How do we assess the quality of the code we are shipping? That's an interesting question, and one that has no simple answer as there are different angles to consider. To build confidence in the quality of the source code, we can use tools like ReSharper and leverage over 2,300 code inspections to ensure we are using our programming language in a good way.

Now, how do we build confidence in our logic? Typically, we solve that aspect by writing unit tests and integration tests. These tests then call into our business logic and verify that a given input results in a known output or behavior. But that leads us to another question: how do we know we have tested all code paths in our logic? How can we be confident our test suite has sufficient coverage of our code?

This is where a tool like JetBrains dotCover comes in handy. It helps calculate and visualize which parts of our code are covered by a certain test or application execution, for code that is targeting the full .NET Framework or .NET Core.

Getting started

We can use JetBrains dotCover with either Microsoft's Visual Studio, or with JetBrains Rider (check out my previous CodeProject article to learn more about this cross-platform .NET IDE).

Once installed, dotCover adds a unit test runner that can run, debug and analyze coverage of unit tests written with the MSTest, NUnit, xUnit, or MSpec frameworks. For ReSharper or Rider users, dotCover integrates directly into the existing test runner.

From the test runner, we can use the toolbar (or menu entry) to Cover All Tests From Solution. This will run our tests and reports back results in the Unit Test tool window:

Meanwhile, the Unit Test Coverage tool window will provide insights into code coverage:

From the Unit Test Coverage tool window, we can immediately see how much of our application code statements have been covered by our test run. Generally, a higher percentage is always better, as it means more branches of our code have been executed and thus, verified by a test.

Do we need 100% code coverage?

When reading about code coverage and test coverage, there are various opnions about whether we need 100% code coverage or not. While 100% code coverage is a noble goal to strive for, but does it really mean we have the perfect test suite? On the other hand, how can we be sure 90% code covevrage is enough? Maybe there are bugs hidden in the 10% we missed!

As with many things in IT, it depends. Without starting a holy war, it's probably a good idea to combine the code coverage metric with another metric such as cyclomatic complexity. In short, it represents the number of linearly independent paths through code. Even shorter: it tells us how complex our code is. When combining these two metrics, we can get a better understanding of areas where we may need to increase code coverage: if a complex piece of code has low coverage, we need to test more.

The Unit Test Coverage tool window has a Hot Spots view which does exactly this: it calculates cyclomatic complexity of our code and helps us detect potential risk areas. Hovering an entry shows us coverage and complexity.

The bigger a type or method is shown here, the more complex it is. Using the Hot Spots view helps us determine which areas of our code need more testing.

Unit test code coverage

Once our tests have been run, either from the Unit Test tool window or directly from the editor, we get insights into code coverage from the Unit Test Coverage tool window. From there, we can can order results by project or namespace, and drill down into a given method to look at how well it has been covered.

The Unit Test Coverage tool window is not the only place where coverage is reported. dotCover analyzes coverage at the statement level, and visualizes this in the editor. We can open the editor and look at our code. Colors in the left gutter will tell us whether a statement has been covered: green when the statement was covered, gray when it was not covered, and red when the statement was covered but a unit test failed. We can hover them to get more information.

dotCover not only keeps track of which statements have been covered or not. It also tracks which test(s) are responsible for covering a given statement. Again by clicking the gutter, we can display covering tests, explore results, and navigate to a specific test.

Tip: when using ReSharper, the Navigate To shortcut (Alt+`) lets us easily navigate from code to test and back.

Having coverage information is very handy. From the tool window, we can easily find parts of our codebase that have lower code coverage. From the editor, we can easily find code paths that have lower code coverage, and find out which areas of our code need more test cases.

Continuous testing

Having knowledge of which test covers which portion of code is also useful for something else: continuous testing. dotCover can figure out which unit tests are affected by our latest code changes, and can then automatically re-run them. Continuous testing in dotCover comes in handy not only when doing test-driven development, but also to help speed up our developer workflow.

When using continuous testing, there is no need to manually rebuild our project(s) and re-run all tests after making changes. Since dotCover knows which tests cover which code, it can build and run the impacted tests automatically in the background.

From any unit test session, we can enable continuous testing. For example, to have dotCover automatically run tests and get coverage information each time we save or build our solution, select Autostart Tests on Build: Cover New and Outdated Tests.

Once enabled, we can start working as usual: update our code, build, and see test results updated in real time.

Note that depending on our preferences, dotCover can run tests either on build or on save. This can be configured from the settings.

Code coverage for manual testing

While unit tests (and integration tests) are great tools to validate our business logic is correct, sometimes nothing is better than running the application and manually working with it, also known as manual testing.

Independently of having unit tests in our solution or not, dotCover allows us to analyze code coverage for executed applications. We can run an application (or our Visual Studio startup project) with dotCover attached, execute a specific scenario, and then get a coverage snapshot for that run.

Just like in earlier examples, we can then use the data in this snapshot and see which parts of our code were called during the executed scenarios, and which parts were not.

Note that attaching dotCover to a full application will not only help determine whether a given manual test case covers a specific portion of code. It also helps understand a new code base by knowing which features cover which areas of that code base.

Code coverage in continuous integration

While getting code coverage information in Visual Studio or Rider is useful, it may be good to also make it a part of our continuous integration (CI) process — especially when running unit tests is already part of that process.

We can use the dotCover command line tools to run our unit tests with code coverage enabled. For TeamCity, we can use the built-in coverage engine instead. After a build completes and tests have been run, our CI server will then display a code coverage summary.

Tip: check out this video tutorial on setting up dotCover in continuous integration for more details.

When making use of dotCover in a CI environment, reports can be exported to XML or HTML, so we can look a coverage details. Coverage snapshots can also be stored as build artifacts, and then imported in dotCover on our own machine. This lets us explore coverage information from the CI server in the same way we would explore it for a local test run.

Conclusion

As we've seen in this article, JetBrains dotCover helps determine to what extent our code is covered by unit tests. We can use it to verify whether we have tested all code paths in our logic, and how confident we can be in the amount of code that is covered by our test suite.

We also learned that 100% code coverage is often not required, because code coverage should be seen in relation to the complexity of our code. The dotCover Hot Spots view helps us detect areas of our code base that need more testing.

Many development teams already run unit tests as part of their continuous integration (CI) pipeline. We can make use of the dotCover command line tools or TeamCity integration to collect code coverage information from our CI server.

It's possible to install dotCover in Visual Studio, or use it as part of ReSharper Ultimate. If you are using JetBrains Rider, there's a dotCover plugin available as well.

Download dotCover now and give it a try!