Data Cleaning Exercises

Module 15

Outliers and context

Flag anomalies without deleting legitimate extremes or silently changing the population.
  1. Flag outliers, don't delete them

    Mark unusual values for review while keeping the population intact. · 7 min

  2. Flag with a reason

    Mark unusual values for review with a declared rule while keeping every row. · 7 min

  3. Outlier or error?

    Separate values that are unusual but real from values that break a rule. · 7 min

  4. Robust statistics

    Use medians and quantiles to describe data that contains extremes. · 6 min

Module 15 quiz

Practice questions: choose an answer and check it to see the explanation. Nothing here is timed or graded.

  1. 1. A $48,000 sale is far above the usual range. What should a cleaning pipeline do?
  2. 2. Refund rows have negative amounts. Are they errors?
  3. 3. A rule flags amounts "greater than 5000.00" but the code uses >=. What changes?
  4. 4. A sale of 48,000 appears in a shop where most sales are under 100. What should cleaning do?
  5. 5. Why are negative amounts not automatically errors in a transactions table?
  6. 6. A pipeline removes values more than 3 standard deviations from the mean before reporting. What is the risk?
  7. 7. What is the difference between flagging and filtering an outlier?
  8. 8. Sensor readings jump to 9999. The device manual says 9999 means "no reading". What should cleaning do?
  9. 9. Values 10, 11, 12, 13, 500: which statistic is barely affected by the 500?