Mutation testing
Quality assurance consists of many activities, but often our primary goal is to verify and validate software our team is working on. Through various means we try to establish whether it fulfills customer requirements and other criteria like code quality, documentation, security, and so on. But who ensures that our job was done correctly? Even when we have a colleague who can review our test artifacts, they probably don't have enough time or context to do this thoroughly due to their own commitments.
This is why I always liked mutation testing as it aims to evaluate the quality of tests themselves. The idea is that we introduce small changes in the software under test and see if our tests managed to catch unexpected behavior by failing. We can, for instance, modify an SQL query from this:
select * from shopping_carts where user_id = ?
To this:
select * from shopping_carts where user_id != ?
So even though an insert method somewhere else would correctly add items to a shopping cart, it'll appear to be empty to the user due to the inverted where clause. If we don't have a test to validate the shopping cart after adding some items, all our tests will pass, revealing a gap in coverage. Like any other form of testing, mutation testing isn't exhaustive and it cannot prove the absence of bugs. Still, it can be a useful tool.