Data Cleaning Exercises

Module 4

Text and Unicode

Trim explicit whitespace, normalize permitted forms, preserve names and diagnose invisible characters.
  1. Trim the right characters, and only those

    Remove padding from identifiers using a character set you can name, without changing what's inside the value. · 8 min

  2. Normalize Unicode, keep the accents

    Make visually identical names equal with NFC normalization, without stripping accents. · 8 min

  3. Find invisible characters

    Diagnose why identical-looking values don't match by inspecting lengths and code points. · 7 min

  4. Case is a policy decision

    Change case only where a declared policy says case doesn't matter, and keep the original elsewhere. · 6 min

Module 4 quiz

Practice questions: choose an answer and check it to see the explanation. Nothing here is timed or graded.

  1. 1. Two names look identical but don't compare equal. The most likely cause is:
  2. 2. DuckDB: what does trim(' a\u00a0') return when the value ends with a no-break space?
  3. 3. Names contain double spaces inside them ("Ana Lopez"). The rules only say to trim the ends. What should you do?
  4. 4. A team lowercases names before storing them so that comparisons work. What is the problem?
  5. 5. What is the difference between NFC normalization and transliteration?
  6. 6. Two keys look identical but don't join. Printing their lengths shows 6 and 7. What is a likely cause?
  7. 7. Product codes have trailing tabs from a copy-paste. The rule says trim ASCII whitespace. What happens to the tabs?