Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New article that details unit test code coverage using coverlet #18955

Merged
merged 17 commits into from
Jun 17, 2020
Merged
2 changes: 1 addition & 1 deletion docs/core/testing/order-unit-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,4 +79,4 @@ To order tests explicitly, NUnit provides an [`OrderAttribute`](https://github.c
## Next Steps

> [!div class="nextstepaction"]
> [Unit testing best practices](unit-testing-best-practices.md)
> [Unit test code coverage](unit-testing-code-coverage.md)
6 changes: 6 additions & 0 deletions docs/core/testing/unit-testing-best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,12 @@ Writing tests for your code will naturally decouple your code, because it would
- **Self-Checking**. The test should be able to automatically detect if it passed or failed without any human interaction.
- **Timely**. A unit test should not take a disproportionately long time to write compared to the code being tested. If you find testing the code taking a large amount of time compared to writing the code, consider a design that is more testable.

## Code coverage

A high code coverage percentage is often associated with a higher quality of code, however; the measurement itself *cannot* determine the quality of code. Setting an ambiguous code coverage percentage goal is counterintuitive. Imagine a complex project with thousands of conditional branches, and imagine that it requires 95% code coverage. Currently the project maintains 90% code coverage. The amount of time it takes to account for all of the edge cases in the remaining 5% could be a massive undertaking and the value proposition quickly diminishes.
IEvangelist marked this conversation as resolved.
Show resolved Hide resolved

Using code coverage for what it truly represents is always best. It is not an indicator of success, nor does it imply a high code quality - instead it is simply the amount of code that is covered by unit tests. For more information, see [unit testing code coverage](unit-testing-code-coverage.md).
IEvangelist marked this conversation as resolved.
Show resolved Hide resolved

## Let's speak the same language
The term *mock* is unfortunately very misused when talking about testing. The following defines the most common types of *fakes* when writing unit tests:

Expand Down
78 changes: 78 additions & 0 deletions docs/core/testing/unit-testing-code-coverage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
---
title: Use code coverage for unit testing
description: Learn how to use the code coverage capabilities for .NET unit tests.
author: IEvangelist
ms.author: dapine
ms.date: 06/15/2020
---

# Use code coverage for unit testing

Unit tests help to ensure functionality, and provide a means of verification for refactoring efforts. Code coverage is a measurement of the amount of code that is ran as part of a unit test run - either lines, branches, or methods. As an example, if you have a simple application with only two conditional branches of code (_branch a_, and _branch b_), a unit test that verifies conditional _branch a_ will report branch code coverage of 50%.
IEvangelist marked this conversation as resolved.
Show resolved Hide resolved

This article discusses the usage of code coverage for unit testing with coverlet, and it is [open source project on GitHub](https://github.com/coverlet-coverage/coverlet). Coverlet is a cross platform code coverage framework for .NET. Additionally, [coverlet](https://dotnetfoundation.org/projects/coverlet) is part of the .NET foundation.
IEvangelist marked this conversation as resolved.
Show resolved Hide resolved

## Tooling

To use coverlet for code coverage, an existing unit test project must have the appropriate package dependencies, or alternatively rely on [.NET global tooling](../tools/global-tools.md).
IEvangelist marked this conversation as resolved.
Show resolved Hide resolved

### Integrate with .NET test
IEvangelist marked this conversation as resolved.
Show resolved Hide resolved

To integrate with [`dotnet test`](../tools/dotnet-test.md), add the appropriate package dependency. From the directory of the *.csproj* file, run the following [`dotnet add package`](../tools/dotnet-add-package.md) command:
IEvangelist marked this conversation as resolved.
Show resolved Hide resolved

```dotnetcli
dotnet add package coverlet.collector
```

The [coverlet.collector](https://www.nuget.org/packages/coverlet.collector) NuGet package is added to the project. Next, run the following command:

```dotnetcli
dotnet test --collect:"XPlat Code Coverage"
```

> [!IMPORTANT]
> The `"XPlat Code Coverage"` argument is a friendly name that corresponds to the data collectors from coverlet. This name is required, but is case insensitive.
IEvangelist marked this conversation as resolved.
Show resolved Hide resolved

As part of the `dotnet test` run, a resulting *TestResults/{test-run-guid}/coverage.cobertura.xml* file is output. The XML file contains the results.
IEvangelist marked this conversation as resolved.
Show resolved Hide resolved

#### MSBuild

As an alternative, you could use the MSBuild package. From the .NET Core CLI at the directory level of the *.csproj* file, run the following [`dotnet add package`](../tools/dotnet-add-package.md) command:
IEvangelist marked this conversation as resolved.
Show resolved Hide resolved

```dotnetcli
dotnet add package coverlet.msbuild
```

The [coverlet.msbuild](https://www.nuget.org/packages/coverlet.msbuild) NuGet package is added to the project. Next, run the following command:

```dotnetcli
dotnet test /p:CollectCoverage=true
```

> [!NOTE]
IEvangelist marked this conversation as resolved.
Show resolved Hide resolved
> There is a subtle differences in the output between the two `dotnet test` commands. The first output was an XML file, whereas the last is a JSON file.
IEvangelist marked this conversation as resolved.
Show resolved Hide resolved

### Use with .NET global tooling

To use coverlet globally, it can be installed as a [.NET global tool](../tools/global-tools.md). Run the following [`dotnet tool install`](../tools/dotnet-tool-install.md) command:

```dotnetcli
dotnet tool install --global coverlet.console
```

The [coverlet.console](https://www.nuget.org/packages/coverlet.console) NuGet package is added to the environment globally. Now, `coverlet` can be used.
IEvangelist marked this conversation as resolved.
Show resolved Hide resolved

```console
coverlet C:\Source\Directory\bin\Debug\netcoreapp3.1\Test.dll --target "dotnet" --targetargs "test . --no-build"
IEvangelist marked this conversation as resolved.
Show resolved Hide resolved
```

## See also

- [Visual Studio unit test cover coverage](/visualstudio/test/using-code-coverage-to-determine-how-much-code-is-being-tested)
- [GitHub - Coverlet repository](https://github.com/coverlet-coverage/coverlet)
- [.NET Core CLI test command](../tools/dotnet-test.md)

## Next Steps

> [!div class="nextstepaction"]
> [Unit testing best practices](unit-testing-best-practices.md)
2 changes: 2 additions & 0 deletions docs/core/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -431,6 +431,8 @@ items:
href: testing/selective-unit-tests.md
- name: Order unit tests
href: testing/order-unit-tests.md
- name: Unit test code coverage
href: testing/unit-testing-code-coverage.md
- name: Unit test published output
href: testing/unit-testing-published-output.md
- name: Live unit test .NET Core projects with Visual Studio
Expand Down