Data Cleaning Exercises

Exercise · beginner · 15 min

Referential integrity with explicit exceptions

Payments must point at a known account. Quarantine missing and unknown references with distinct reasons so every row is accounted for.

The rules

These rules decide what a correct result is. Your answer is checked on this dataset and on hidden datasets that follow the same rules.

Inputs
  payments(id, account, amount_cents) — text. id is unique. amount_cents is always a valid whole number (refunds are negative).
  accounts(account, name) — text. Accounts are trimmed and unique, never missing.

Rules
  account: remove ASCII whitespace at both ends; an empty result is missing.
  Quarantine each invalid payment with the FIRST failing reason:
    1. missing_account   account is missing
    2. unknown_account   account has no matching row in accounts
  Valid payments get the account's name.

Outputs (row order does not matter)
  clean(id text, account text, name text, amount_cents int64)
  exceptions(id text, reason text)

The data

payments · 10 rows

idaccountamount_cents
pay59hn99951974
payij5o457923
payfhun777641
paym2n8missing72752
paywa3o999-1746
pay7u1w·7·-1489
pay77t1999-3521
payacbh4880852
payngmn787639
payfkj9999-461

accounts · 5 rows

accountname
12Bolt GmbH
7Delta Co
4Cedar SA
22Acme Ltd
48Echo LLC

What you practise

  • lookups
  • quarantine
  • reconciliation

Your answer is also checked on hidden datasets generated from the same rules.

Download this dataset (CSV) with its data dictionary

Sign in to bookmark this and keep private notes.