In this What QA Wolf means by “test coverage” post, the author John Gluck starts with a simple definition of test coverage, discusses what tests to write and how, and ultimately cautions against using coverage targets as an end goal.
The simple definition of coverage from Michael Bolton is “how thoroughly we have examined the product with respect to some model”.
Regardless of the definition, any given coverage metric can be interpreted objectively or subjectively. e.g. Unit-test code coverage can be
- Objective: the percentage of lines of code exercised when a set of tests executes.
- Subjective: Are those lines of code essential to the working of the application? Did the tests contain meaningful assertions?
What is more important is that a high level of coverage should equate to high confidence that the code is ready for release. For example, tests have been code-reviewed and actually detects defects.
How do you determine what tests to write?
Tests may be written based on
- how a user interacts with a UI
- Documented requirements
- Specific areas of risk in the application
- other approaches such as revenue, security, compliance
How do you write your tests?
One approach is to use the Arrange, Act, Assert (AAA) framework.
Arrange is the setup of things such as classes, mocks, and expectations.
Act is the invocation of the code (typically a method) being tested.
Assert is verifying the outcome, by checking whether the expectations were met.
The goal beyond coverage
However you decide what tests to write, how to write them, and what coverage levels to aim for, avoid using that target coverage as an end goal.
Instead, ensure the team is developing a deep understanding of the application-under-test, its usage patterns, and creating tests where a high coverage level will give a high confidence that the code is fit for purpose.
