Before this: Missing is not empty (and neither is zero)
The idea
Filling a missing value is a claim. For a category used to group a report, the label "unknown" is honest: it is visibly not a real region and keeps every row in a bucket.
For a measurement, any filled number is a fake observation. A missing duration replaced with 0 says the visit lasted zero minutes, and the average duration drops for no real reason.
Turn blank text into missing first. Otherwise a blank region stays as empty text, the fill skips it, and the report shows an unlabeled bucket next to "unknown".
Worked example
Input: region "", duration_min "" / region "West", duration_min "0"
- Visual
region: Trim → Blank to missing → Fill missing (unknown). duration_min: Trim → Blank to missing → Convert type to int64 (keep missing).
- SQL (DuckDB)
COALESCE(NULLIF(trim(region, chars), ''), 'unknown') AS region, CAST(NULLIF(trim(duration_min, chars), '') AS BIGINT) AS duration_min
- pandas
df["duration_min"] = duration.mask(duration == "").astype("Int64")
Result: (unknown, missing) and (West, 0): the reported zero stays a zero, the missing duration stays missing.
Common mistakes
- Filling numeric gaps with 0 or the mean without a flag.
- Filling before turning blanks into missing.
- Labels that look like real categories.
Check yourself
Is filling a missing duration with 0 safe if you add a note?
No. The value becomes a fake measurement in every later calculation; keep it missing.
Why turn blank region text into missing before filling?
Fill only replaces missing values; blank text would stay as an empty, unlabeled category.
Practice it
Label missing regions as unknown and keep missing durations missing.
Next: Label unknown categories, keep missing numbers missing