What is the difference between Property Based Testing and Mutation testing?
These are very different beasts but both would improve the value and quality of your tests. Both tools contribute to and make the "My code coverage is N%" statement more meaningful.
Hypothesis would help you to generate all sorts of test inputs in the defined scope for a function under test.
Usually, when you need to test a function, you provide multiple example values trying to cover all the use cases and edge cases driven by the code coverage reports - this is so called "Example based testing". Hypothesis on the other hand implements a property-based testing generating a whole bunch of different inputs and input combinations helping to catch different common errors like division by zero, None, 0, off-by-one errors etc and helping to find hidden bugs.
Mutation testing is all about changing your code under test on the fly while executing your tests against a modified version of your code.
This really helps to see if your tests are actually testing what are they supposed to be testing, to understand the value of your tests. Mutation testing would really shine if you already have a rich test code base and a good code coverage.
What helped me to get ahold of these concepts were these Python Podcasts:
- Property-based Testing with Hypothesis
- Validating Python tests with mutation testing
- Hypothesis with David MacIver