Data Cleaning Exercises

Module 11

Validation rules

Required fields, ranges, allowed categories, cross-field logic, severity and exception reports.
  1. Quarantine with reasons, in rule order

    Send invalid rows to an exceptions table with the first failing reason so every input row is accounted for. · 10 min

  2. Ranges and the order of rules

    Turn business limits into range rules, check type before range, and report the first failing rule in a declared order. · 9 min

  3. Cross-field rules

    Write rules that compare columns within a row, such as end date after start date, and report the first failure. · 8 min

  4. Required fields

    Check required fields after normalization, so blanks and padding count as missing. · 6 min

Module 11 quiz

Practice questions: choose an answer and check it to see the explanation. Nothing here is timed or graded.

  1. 1. Why check rules in a fixed order?
  2. 2. Rule: quantity between 1 and 500. Which values fail: 0, 1, 500, 501?
  3. 3. A price of 0.00 appears. The rule says prices must be greater than zero. What do you do?
  4. 4. What is a cross-field rule?
  5. 5. A rule fails for 40% of rows. What should you do before quarantining them?
  6. 6. Rules in order: missing_quantity, bad_quantity, quantity_out_of_range. A row has quantity "abc". Which reason?