Before this: Missing is not empty (and neither is zero)
The idea
A missing value can mean the question wasn't asked, didn't apply, or was refused. Analyses treat these differently.
If the source encodes the reason (codes, a separate column), keep it in a reason column before turning the value into a real missing value.
Worked example
Input: income: 52000, 'refused', 'n/a'
- Visual
Add step → Map values → refused/n/a → (missing), reason column kept
- SQL (DuckDB)
CASE WHEN income IN ('refused','n/a') THEN income END AS income_missing_reason- pandas
df["income_missing_reason"] = df["income"].where(df["income"].isin(["refused", "n/a"]))
Result: income: 52000, missing, missing; reasons kept
Common mistakes
- Collapsing all reasons into one blank.
- Filling refused answers with averages.
Check yourself
Why keep a reason column?
Because 'refused' and 'not applicable' need different handling and reporting.
Practice it
Label unknown categories, keep missing numbers missing.
Next: Label unknown categories, keep missing numbers missing