🎯 Chapter Insight
Most tests begin with examples.
Given this input, we expect that output. Given another input, we expect something else.
This approach is useful, intuitive, and often necessary. But it has an important limitation: we choose the examples.
And we naturally choose examples we can think of.
That means our tests tend to reflect the scenarios we already expect. We test normal values, obvious boundaries, and perhaps a few special cases we encountered while developing the feature.
The bugs that surprise us often live somewhere else.
Property-based testing approaches the problem differently. Instead of describing individual examples, we describe properties that should remain true across a large range of possible inputs.
Then we allow the testing tool to generate those inputs for us.
The question changes from:
“Does the code produce the correct result for these five examples?”
to:
“What must always remain true, regardless of which valid input we provide?”
That shift can expose assumptions we did not even realise we were making.
💡 Developer Lens
Imagine you are testing a function that sorts a collection.
Traditional example-based tests might look conceptually like this:
Input:
[3, 1, 2]
Expected result:
[1, 2, 3]
Then perhaps you add another example for an empty list, another for duplicate values, and another for negative numbers.
Those are useful tests.
But there are countless possible lists you have not tested.
Property-based testing encourages you to step back and ask what must always be true after sorting.
For example:
- The result should contain the same number of elements as the input
- Every input value should still exist in the result
- The elements should be ordered
- Sorting an already sorted result should not change it
These are properties of the operation rather than examples of its behaviour.
A property-based testing framework can then generate many different collections and check whether those properties continue to hold.
Suddenly your test suite is exploring combinations you might never have written manually.
Empty collections. Very large collections. Duplicate values. Extreme numbers. Strange combinations of values.
The framework becomes an active participant in finding the scenarios your assumptions overlooked.
The Interesting Bugs Live Outside Our Examples
Developers are usually quite good at testing the situations they expect.
Unfortunately, software failures frequently happen in situations we did not expect.
Perhaps a parser works perfectly for every example in the test suite but fails when two unusual characters appear together.
Perhaps a calculation behaves correctly for positive values but produces an unexpected result around zero.
Perhaps an API transformation works for all realistic examples developers considered, yet breaks when optional fields appear in an unusual combination.
You cannot manually anticipate every possible input.
Property-based testing reduces that burden by exploring the input space automatically.
Instead of trying to predict every problematic example, you define the rule the system must respect and allow the testing framework to search for violations.
This changes the developer’s role.
Your job becomes less about inventing examples and more about identifying invariants.
What must always be true?
That is often a much more interesting question.
Properties Reveal the Real Contract
Thinking in properties can also improve your understanding of the software itself.
Suppose you cannot describe what must always remain true for a function.
That may indicate that its contract is unclear.
Perhaps it has too many responsibilities. Perhaps its behaviour depends on hidden state. Perhaps there are edge cases whose expected behaviour has never been defined.
Trying to write a property forces you to think more deeply about the guarantees your code actually provides.
For a serializer, you might ask whether serialising and then deserialising a value should return the original value.
For an encoder and decoder, you might expect decoding an encoded value to reproduce the input.
For a collection operation, you might expect the number of elements to remain constant.
For financial calculations, you might identify rules about totals, ranges, or precision that should never be violated.
These properties describe something fundamental about the domain.
And once those rules are expressed as executable tests, they become part of the safety net protecting the system.
Let the Framework Find the Weird Inputs
One of the most valuable aspects of property-based testing is that generated inputs are not influenced by the same expectations as the developer writing the test.
A human may test:
1
,
10
,
100
, and
1000
.
The generator may eventually discover that
0
, a negative value, a huge number, or an unexpected combination of inputs violates the property.
That failing input is valuable because it teaches you something about the system.
Perhaps the implementation contains a bug.
Perhaps the property was defined incorrectly.
Or perhaps the system’s expected behaviour for that case was never decided.
All three outcomes improve your understanding.
The goal is not merely to make a test fail. The goal is to uncover assumptions hiding inside the implementation.
Property-based testing gives those assumptions somewhere to surface.
Example-Based Tests Still Matter
Property-based testing does not mean abandoning normal tests.
Examples remain extremely valuable.
A carefully chosen example can document an important business case far more clearly than a generated property. Regression tests are excellent for preserving behaviour around previously discovered bugs. Specific scenarios can also communicate requirements in a way that is immediately understandable to other developers.
The two approaches solve different problems.
Example-based tests ask:
“Does this particular scenario behave correctly?”
Property-based tests ask:
“Does this fundamental rule survive many different scenarios?”
A strong test suite can use both.
Concrete examples document meaningful cases while properties explore the spaces between them.
Property-Based Testing and Developer Assumptions
Perhaps the most interesting benefit of property-based testing is not the number of inputs it generates.
It is what those inputs reveal about how we think.
Developers carry assumptions constantly.
We assume strings will not be empty. We assume identifiers will follow the expected format. We assume numbers will stay within reasonable ranges. We assume data arrives in a certain order. We assume two values that usually appear together will always appear together.
Most of the time, those assumptions remain invisible.
Until production disagrees.
Property-based testing creates opportunities for our assumptions to be challenged earlier.
Instead of waiting for a real user to discover the unexpected combination, we ask the testing framework to actively search for it.
That is a powerful shift.
Tests stop being only confirmation that the software works under expected circumstances.
They become experiments designed to discover where our understanding is incomplete.
🧭 Reflection
Look at the tests in your current codebase.
How many verify only a handful of carefully selected examples?
Could some of those examples be expressed as a broader rule?
What properties should always remain true regardless of the input?
And perhaps more importantly:
What assumptions are hidden in your code simply because nobody has generated an input that challenges them yet?
A passing collection of examples tells you that those examples work.
A property that survives thousands of generated cases provides a different kind of confidence.
⚙️ Practical Tip
Choose one function this week with a clear invariant.
Do not start with the most complicated part of your system. Pick something where the relationship between input and output is relatively easy to describe.
Then ask:
- What must always be true?
- What should never happen?
- Does an operation preserve something about the input?
- Can two operations reverse each other?
- Does performing the same operation twice produce the same result?
- Are there limits the output must never exceed?
Write one property expressing that rule.
Then use a property-based testing tool to generate many different inputs and challenge it.
When the framework finds a failure, resist the temptation to immediately add that input as another isolated example and move on.
Investigate why the property failed.
The most valuable discovery may not be the unusual input itself.
It may be the assumption you did not know your code was making.
🔢 #42 of 53 | The Pragmatic Programmer Series
This post is part of my 53-week series summarizing The Pragmatic Programmer, one timeless principle each week, translated into modern software practice and reflection.
