The usual choices for getting code coverage metrics in Visual Studio is to buy the Enterprise version of Visual Studio or buy the dotCover third party tool, both of which are costly, especially if you are developing small applications yourself and want to check the coverage of your unit tests.
There is a free NuGet package for generating code coverage metrics called OpenCover, and together with another free NuGet package, ReportGenerator, can give you code coverage reports for free.
The problem I found, however, was the documentation available to get the tool working was less than adequate for my purposes and it took several hours of faff to get it working for me. To save you time and money, I have documented my experience here.
The following were all referenced in the creation of this document:
I have tested that this works in Visual Studio 2013 Professional and Visual Studio 2015 Community Edition, with both MSTest and NUnit.
MSTest unit tests can be run with either mstest.exe or the newer vstest.console.exe. As I primarily use NUnit, I have not done anything other than basic testing to ensure OpenCover works with both these test runners so cannot say how either might work on large projects.
I assume you already have a solution with application project that you are testing with a test project, and where you are using NUnit for that testing that you have already included the NUnit NuGet package.
You will need to open your solution in Visual Studio and get the following packages using Package Manager Console:
REM PM> Install-Package OpenCover
REM PM> Install-Package ReportGenerator
If you are using NUnit rather than MSTest, you will also need this package:
REM PM> Install-Package NUnit.ConsoleRunner
In the solution root folder, you will then need to create a batch file that you will use to produce the code coverage report. I have three batch files shown below, one for use where you are using NUnit, one where you are using MSTest and the mstest.exe test runner, and one for where you are using MSTest and the vstest.console.exe test runner.
You will need to modify the batch file to specify the DLL that contains your unit tests, i.e., change this line as appropriate (note that %~dp0 is a symbol that is replaced automatically with the folder path of the batch file):
SET DllContainingTests=%~dp0WebApp.Tests\bin\Debug\WebApp.Tests.dll
You may also want to change the filter applied to the results to exclude parts of your solution where you are not interested in the code coverage metrics, in my case, I have asked it to include everything (+[*]*), and then exclude my Tests project (-[*.Tests*]*) and all classes ending in Config (-[*]*.*Config), i.e.:
-filter:"+[*]* -[*.Tests*]* -[*]*.*Config"
If you are using MSTest, you may need to change the TestRunnerExe variable to point to the correct version of mstest.exe or vstest.runner.exe. The examples below are pointing to the ones in Visual Studio 2013, but if you are using 2015, you will need to change this to Microsoft Visual Studio 14.0 instead of Microsoft Visual Studio 12.0.
All that then remains is to execute the batch file to produce a report of code coverage that allows you to drill down to see the actual colour coded code being covered, or not.
NUnit Version
@ECHO OFF
REM OpenCover-NUnit.bat
REM Run opencover against NUnit tests in your test project and show report of code coverage
REM Derivative work based on work by:
REM Shaun Wilde - https://www.nuget.org/packages/OpenCover/
REM Daniel Palme - https://www.nuget.org/packages/ReportGenerator/
REM Charlie Poole - https://www.nuget.org/packages/NUnit.ConsoleRunner/
REM Allen Conway -
REM http://www.allenconway.net/2015/06/using-opencover-and-reportgenerator-to.html
REM Andrew Newton -
REM http://www.nootn.com.au/2014/01/code-coverage-with-opencover-example.html#.VxiNn_krLDc
SET DllContainingTests=%~dp0WebApp.Tests\bin\Debug\WebApp.Tests.dll
REM *** IMPORTANT - Change DllContainingTests variable (above) to point to the DLL
REM *** in your solution containing your NUnit tests
REM ***
REM *** You may also want to change the include/exclude filters (below)
REM *** for OpenCover
REM ***
REM *** This batch file should dbe placed in the root folder of your solution
REM *** Before being able to use this to generate coverage reports you
REM *** will need the following NuGet packages
REM PM> Install-Package OpenCover
REM PM> Install-Package ReportGenerator
REM PM> Install-Package NUnit.ConsoleRunner
REM
REM NUnit Test Runner (done this way so we dont have to change the code
REM when the version number changes)
for /R "%~dp0packages" %%a in (*) do if /I "%%~nxa"=="nunit3-console.exe" SET TestRunnerExe=%%~dpnxa
REM Get OpenCover Executable (done this way so we dont have to change the
REM code when the version number changes)
for /R "%~dp0packages" %%a in (*) do if /I "%%~nxa"=="OpenCover.Console.exe" SET OpenCoverExe=%%~dpnxa
REM Get Report Generator (done this way so we dont have to change the code
REM when the version number changes)
for /R "%~dp0packages" %%a in (*) do if /I "%%~nxa"=="ReportGenerator.exe"
SET ReportGeneratorExe=%%~dpnxa
REM Create a 'GeneratedReports' folder if it does not exist
if not exist "%~dp0GeneratedReports" mkdir "%~dp0GeneratedReports"
REM Run the tests against the targeted output
call :RunOpenCoverUnitTestMetrics
REM Generate the report output based on the test results
if %errorlevel% equ 0 (
call :RunReportGeneratorOutput
)
REM Launch the report
if %errorlevel% equ 0 (
call :RunLaunchReport
)
exit /b %errorlevel%
:RunOpenCoverUnitTestMetrics
REM *** Change the filter to include/exclude parts of the solution you want
REM *** to check for test coverage
"%OpenCoverExe%" ^
-target:"%TestRunnerExe%" ^
-targetargs:"\"%DllContainingTests%\"" ^
-filter:"+[*]* -[*.Tests*]* -[*]*.*Config" ^
-mergebyhash ^
-skipautoprops ^
-register:user ^
-output:"%~dp0GeneratedReports\CoverageReport.xml"
exit /b %errorlevel%
:RunReportGeneratorOutput
"%ReportGeneratorExe%" ^
-reports:"%~dp0\GeneratedReports\CoverageReport.xml" ^
-targetdir:"%~dp0\GeneratedReports\ReportGenerator Output"
exit /b %errorlevel%
:RunLaunchReport
start "report" "%~dp0\GeneratedReports\ReportGenerator Output\index.htm"
exit /b %errorlevel%
MSTest Version Using mstest.exe
@ECHO OFF
REM OpenCover-MSTest.bat
REM Run opencover against MSTest tests in your test project and show report of code coverage
REM Derivative work based on work by:
REM Shaun Wilde - https://www.nuget.org/packages/OpenCover/
REM Daniel Palme - https://www.nuget.org/packages/ReportGenerator/
REM Allen Conway -
REM http://www.allenconway.net/2015/06/using-opencover-and-reportgenerator-to.html
REM Andrew Newton -
REM http://www.nootn.com.au/2014/01/code-coverage-with-opencover-example.html#.VxiNn_krLDc
SET DllContainingTests=%~dp0WebApp.Tests\bin\Debug\WebApp.Tests.dll
REM *** IMPORTANT - Change DllContainingTests variable (above) to point to the DLL
REM *** in your solution containing your NUnit tests
REM ***
REM *** You may also want to change the include/exclude filters
REM *** (below) for OpenCover
REM ***
REM *** This batch file should dbe placed in the root folder of your solution
REM *** Before being able to use this to generate coverage reports you will
REM *** need the following NuGet packages
REM PM> Install-Package OpenCover
REM PM> Install-Package ReportGenerator
REM
REM *** MSTest Test Runner (VS2013, will need to change 12.0 to 14.0 for VS2015)
SET TestRunnerExe=%PROGRAMFILES(X86)%\Microsoft Visual Studio 12.0\Common7\IDE\MSTest.exe
REM Get OpenCover Executable (done this way so we dont have to change
REM the code when the version number changes)
for /R "%~dp0packages" %%a in (*) do if /I "%%~nxa"=="OpenCover.Console.exe" SET OpenCoverExe=%%~dpnxa
REM Get Report Generator (done this way so we dont have to change the code
REM when the version number changes)
for /R "%~dp0packages" %%a in (*) do if /I "%%~nxa"=="ReportGenerator.exe"
SET ReportGeneratorExe=%%~dpnxa
REM Create a 'GeneratedReports' folder if it does not exist
if not exist "%~dp0GeneratedReports" mkdir "%~dp0GeneratedReports"
REM Run the tests against the targeted output
call :RunOpenCoverUnitTestMetrics
REM Generate the report output based on the test results
if %errorlevel% equ 0 (
call :RunReportGeneratorOutput
)
REM Launch the report
if %errorlevel% equ 0 (
call :RunLaunchReport
)
exit /b %errorlevel%
:RunOpenCoverUnitTestMetrics
REM *** Change the filter to include/exclude parts of the solution you want to
REM *** check for test coverage
"%OpenCoverExe%" ^
-target:"%TestRunnerExe%" ^
-targetargs:"/noisolation /testcontainer:\"%DllContainingTests%\"" ^
-filter:"+[*]* -[*.Tests*]* -[*]*.Global -[*]*.RouteConfig -[*]*.WebApiConfig" ^
-mergebyhash ^
-skipautoprops ^
-register:user ^
-output:"%~dp0GeneratedReports\CoverageReport.xml"
exit /b %errorlevel%
:RunReportGeneratorOutput
"%ReportGeneratorExe%" ^
-reports:"%~dp0\GeneratedReports\CoverageReport.xml" ^
-targetdir:"%~dp0\GeneratedReports\ReportGenerator Output"
exit /b %errorlevel%
:RunLaunchReport
start "report" "%~dp0\GeneratedReports\ReportGenerator Output\index.htm"
exit /b %errorlevel%
MSTest Version Using vstest.console.exe
@ECHO OFF
REM OpenCover-VSTest.bat
REM Run opencover against MSTest tests in your test project and show report of code coverage
REM Derivative work based on work by:
REM Shaun Wilde - https://www.nuget.org/packages/OpenCover/
REM Daniel Palme - https://www.nuget.org/packages/ReportGenerator/
REM Allen Conway -
REM http://www.allenconway.net/2015/06/using-opencover-and-reportgenerator-to.html
REM Andrew Newton -
REM http://www.nootn.com.au/2014/01/code-coverage-with-opencover-example.html#.VxiNn_krLDc
SET DllContainingTests=%~dp0WebApp.Tests\bin\Debug\WebApp.Tests.dll
REM *** IMPORTANT - Change DllContainingTests variable (above) to point to the DLL
REM *** in your solution containing your NUnit tests
REM ***
REM *** You may also want to change the include/exclude filters
REM *** (below) for OpenCover
REM ***
REM *** This batch file should dbe placed in the root folder of your solution
REM *** Before being able to use this to generate coverage reports you will
REM *** need the following NuGet packages
REM PM> Install-Package OpenCover
REM PM> Install-Package ReportGenerator
REM
REM *** MSTest Test Runner (VS2013, will need to change 12.0 to 14.0 for VS2015)
SET TestRunnerExe=%PROGRAMFILES(X86)%\Microsoft Visual Studio 12.0\Common7\IDE\
CommonExtensions\Microsoft\TestWindow\vstest.console.exe
REM Get OpenCover Executable (done this way so we dont have to change
REM the code when the version number changes)
for /R "%~dp0packages" %%a in (*) do if /I "%%~nxa"=="OpenCover.Console.exe" SET OpenCoverExe=%%~dpnxa
REM Get Report Generator (done this way so we dont have to change the code
REM when the version number changes)
for /R "%~dp0packages" %%a in (*) do if /I "%%~nxa"=="ReportGenerator.exe"
SET ReportGeneratorExe=%%~dpnxa
REM Create a 'GeneratedReports' folder if it does not exist
if not exist "%~dp0GeneratedReports" mkdir "%~dp0GeneratedReports"
REM Run the tests against the targeted output
call :RunOpenCoverUnitTestMetrics
REM Generate the report output based on the test results
if %errorlevel% equ 0 (
call :RunReportGeneratorOutput
)
REM Launch the report
if %errorlevel% equ 0 (
call :RunLaunchReport
)
exit /b %errorlevel%
:RunOpenCoverUnitTestMetrics
REM *** Change the filter to include/exclude parts of the solution you want to
REM *** check for test coverage
"%OpenCoverExe%" ^
-target:"%TestRunnerExe%" ^
-targetargs:"\"%DllContainingTests%\"" ^
-filter:"+[*]* -[*.Tests*]* -[*]*.Global -[*]*.RouteConfig -[*]*.WebApiConfig" ^
-mergebyhash ^
-skipautoprops ^
-register:user ^
-output:"%~dp0GeneratedReports\CoverageReport.xml"
exit /b %errorlevel%
:RunReportGeneratorOutput
"%ReportGeneratorExe%" ^
-reports:"%~dp0\GeneratedReports\CoverageReport.xml" ^
-targetdir:"%~dp0\GeneratedReports\ReportGenerator Output"
exit /b %errorlevel%
:RunLaunchReport
start "report" "%~dp0\GeneratedReports\ReportGenerator Output\index.htm"
exit /b %errorlevel%
