Data Cleaning Exercises

Reference · Python (pandas)

Look up values from another table in Python (pandas)

Bring columns from a lookup table by key, keeping every row and never matching missing keys.

When to use it

Enriching rows with master data (customer region, account name).

How

merge(how="left", validate="many_to_one") after removing missing keys from the lookup side.

lookup = customers[customers["customer_key"].notna()]
df.merge(lookup, on="customer_key", how="left", validate="many_to_one")
  • pandas matches <NA> to <NA> unless you remove them.

Pitfalls

  • Fan-out from a non-unique lookup key.
  • Missing keys matching each other in pandas.
  • Inner joins that drop rows silently.

Lesson: Joins where missing keys never match

Practise it: Join orders to customers without matching blanks