Before this: Profile before you change anything
The idea
A data dictionary is a promise about the data. Check it: required columns without blanks, codes within the allowed list, ids unique, units as stated.
Each conflict is either a rule (quarantine values outside the list) or a question (is the dictionary out of date?). Record which, with counts.
Worked example
Input: status allowed: shipped, pending, cancelled; data also has 'returned'
- Visual
Inspector → status → distinct values vs allowed list
- SQL (DuckDB)
SELECT status, COUNT(*) FROM orders WHERE lower(trim(status)) NOT IN ('shipped','pending','cancelled') GROUP BY ALL- pandas
df.loc[~df.status.str.strip().str.lower().isin(ALLOWED), "status"].value_counts()
Result: 3 rows 'returned': ask whether it is a new valid status
Common mistakes
- Trusting the dictionary blindly.
- Changing the dictionary to match the data silently.
Check yourself
What do you do when data and dictionary disagree?
Count the conflict, show an example and ask the owner; apply a documented rule meanwhile.
Practice it
Clean the export under its declared rules.