dcreager.net

Property-based testing

Property-based testing is a way of testing software by generating large numbers of random but structured inputs, and verifying that certain properties hold for each input. It is not an exhaustive search, since it presumably would take far too long to enumerate all possible inputs. But it's helpful for finding interesting classes of inputs, and corner cases, automatically.

Property-based testing libraries will typically “minimize” an input when it fails, trying to find the “smallest” input that also fails. You have some amount of control over what is considered “smaller” based on how you construct the inputs. This can help narrow down which part of the code is broken.

CSP would be a good language for property tests

Imperative property testing

This phrase was first coined by matklad, from what I can tell. It describes an API shape where you are given (only) a source of random data, and it's up to you to create instances of whatever datatypes you need by drawing bits from that random data source.

That contrasts with what you'd presumably call “declarative” property testing, where you provide e.g. a function signature of what data you need, and rely on the property testing library to provide you with a random instance of that data.

“Imperative” property testing and the arbtest crate

Libraries

Hypothesis [Python]

QuickCheck [Haskell]

QuickCheck [Rust]

arbtest [Rust]

proptest [Rust]

Rapid [Go]

Theft [C]

Papers

[Krook2023] QuickerCheck: Implementing and Evaluating a Parallel Run-Time for QuickCheck