Data Cleaning Exercises

Task: the rules
Inputs
  orders(ingest_id, order_id, line_no, customer_code, order_ts, amount, status) — all text, any cell may be missing.
  customers(customer_code, region) — already validated: codes are trimmed and unique when present; a code may be missing.

Normalize
  customer_code: remove ASCII whitespace (space, tab, CR, LF) at both ends; an empty result is missing. Keep leading zeros.
  status: remove ASCII whitespace at both ends, then lowercase.
  amount: remove ASCII whitespace at both ends.
  line_no: whole number.

Quarantine each invalid row to exceptions with the FIRST failing reason, checked in this order:
  1. bad_timestamp     order_ts is missing, or is not exactly YYYY-MM-DD, "T" or one space, HH:MM[:SS[.f up to 6 digits]],
                       then an explicit offset (Z or ±HH:MM, at most ±14:00); hour 00–23, minute/second 00–59;
                       the date must exist on the calendar. A timestamp without an offset is invalid (never assume a zone).
  2. bad_amount        amount is missing or not: optional "-", 1–10 digits, optionally "." and 1–2 digits. Refunds (negative) are valid.
  3. bad_status        status is not one of: shipped, pending, cancelled.
  4. unknown_customer  customer_code is missing or has no matching customer. A missing code never matches a missing code.

Deduplicate the remaining rows
  The grain is (order_id, line_no). Keep the row with the latest order_ts instant; on a tie keep the smallest ingest_id
  (code-point order). Every other row goes to exceptions with reason "duplicate".

Outputs (row order does not matter)
  clean(order_id text, line_no int64, customer_code text, region text, order_ts timestamptz [UTC], amount decimal(12,2), status text)
  exceptions(ingest_id text, reason text)

Tab inserts spaces. Press Escape, then Tab, to move focus out of the editor.

Input: orders, rows 1 to 25 of 43
#ingest_idstringorder_idstringline_nostringcustomer_codestringorder_tsstringamountstringstatusstring
1ing-1010-rtgSO-0001110056422026-11-01T01:30:00-05:00·42.10· (raw value " 42.10 ")pending
2ing-1027-nwaSO-0001810056422026-02-30T10:00:00ZN/Apending
3ing-1006-eedSO-0001630030242026-01-15T10:00:00+01:00007.5shipped!
4ing-1014-uy3SO-0001120062812026-03-08T01:30:00-05:0012.50·Shipped (raw value " Shipped")
5ing-1037-di1SO-0001210056422026-13-01T00:00:00Z100pending
6ing-1033-rvdSO-000153·002518⇥ (raw value " 002518\t")2026-06-30T23:59:59.123456+05:300·Shipped (raw value " Shipped")
7ing-1011-jdmSO-0001310030242024-02-29T12:00:00Z0shipped
8ing-1008-qpfSO-000232·003024⇥ (raw value " 003024\t")2026-11-01T01:30:00-05:00007.5pending
9ing-1039-cr5SO-000101·008814⇥ (raw value " 008814\t")2026-11-01T01:30:00-04:009999999999.99·Shipped (raw value " Shipped")
10ing-1026-j7aSO-0001320088142024-02-29T12:00:00Z0shipped
11ing-1007-m5fSO-0001720076002026-11-01T01:30:00-04:00-0cancelled⇥ (raw value "cancelled\t")
12ing-1013-niSO-0002430044752026-06-30T23:59:59.123456+05:305.PENDING
13ing-1038-an8SO-0001930076002026-03-08T01:30:00-05:00007.5·Shipped (raw value " Shipped")
14ing-1019-iisSO-0002130044752026-01-15T10:00:00+01:00·42.10· (raw value " 42.10 ")shipped
15ing-1022-e6tSO-0001120025182026-11-01T01:30:00-04:00-0cancelled⇥ (raw value "cancelled\t")
16ing-1020-e8wSO-0002130044752026-01-15T10:00:00+01:00·42.10· (raw value " 42.10 ")shipped
17ing-1018-fwmSO-000112··⇥ (raw value " \t")2026-11-01T01:30:00-04:0012.50cancelled⇥ (raw value "cancelled\t")
18ing-1009-c3dSO-0001430014192026-11-01T01:30:00-04:00-15.25returned
19ing-1030-dmeSO-0001230044752024-02-29T12:00:00Z9999999999.99ship ped
20ing-1004-xu4SO-0001520056422026-11-01T01:30:00-05:00·42.10· (raw value " 42.10 ")cancelled⇥ (raw value "cancelled\t")
21ing-1002-8kySO-0001110088142026-11-01T01:30:00-04:0012.50cancelled⇥ (raw value "cancelled\t")
22ing-1025-j4wSO-0002320030242026-01-15T10:00:00+01:000cancelled⇥ (raw value "cancelled\t")
23ing-1031-upvSO-0002210025182026-03-08T01:30:00-05:005.pending
24ing-1029-gdaSO-000131·006281⇥ (raw value " 006281\t")2026-13-01T00:00:00Z1e3·Shipped (raw value " Shipped")
25ing-1024-imkSO-0002320030242026-01-15T09:00:00Z0cancelled⇥ (raw value "cancelled\t")

Run preview · in your browser, not a grade

Loading the preview runtime…

Check answer · graded on an isolated server, including hidden datasets

Your code runs in a locked-down machine with no network, on the visible data and on hidden datasets that follow the same rules.

Learn from this exercise

A worked solution becomes available after your first check or once you've used every hint.

AI tutor · may be wrong; the grader decides

SQL · DuckDB 1.5.1Preview runtime loadingNot saved yet · sign in to save to your account