Before this: Rules you can run twice
The idea
Tests turn rules into checks: a padded key matches after trimming, a missing key never matches, a duplicate keeps the right survivor.
Also test invariants on real runs: input = clean + exceptions, keys unique, no missing required fields.
Worked example
Input: tiny table with a padded key and a duplicate
- Visual
(concept) expected output for a tiny input
- SQL (DuckDB)
SELECT COUNT(*) = (SELECT COUNT(*) FROM clean) + (SELECT COUNT(*) FROM exceptions) FROM raw
- pandas
assert len(raw) == len(clean) + len(exceptions)
Result: Tests fail as soon as a rule change breaks a case
Common mistakes
- No tests.
- Only happy-path tests.
Check yourself
Name one invariant to test on every run.
Input rows equal clean rows plus exceptions.
Practice it
Pick one surviving record per account.