Data Cleaning Exercises

Module 8

Types and identifiers

Leading zeros, large IDs, booleans, nullable types and lossless serialization.

  1. Identifiers are text; flags are booleans

    Keep identifiers as exact text and turn yes/no flags into real booleans through a declared mapping, without treating a missing answer as "no". · 8 min

  2. Parse booleans from an approved list

    Turn many spellings of yes/no into true/false with an explicit list and keep unknown answers separate. · 7 min

  3. Long identifiers lose digits

    Keep long numeric-looking ids exact by never storing them as floating point. · 6 min

  4. Nullable booleans

    Store yes/no/unknown with a type that keeps unknown distinct. · 6 min

Module 8 quiz

Practice questions: choose an answer and check it to see the explanation. Nothing here is timed or graded.

  1. 1. In a spreadsheet-style float conversion, what can happen to the ID 12345678901234567890?
  2. 2. An 18-digit account id 123456789012345678 is stored as a float64. What can happen?
  3. 3. An opted_in column contains Y, N, yes, no, 1, 0 and blank. How should blanks be treated?
  4. 4. Why keep postcodes as text even when all values look numeric?
  5. 5. pd.read_csv on a column with values 007, 010, 100 and default settings stores?
  6. 6. Why is a nullable boolean better than mapping missing flags to False?
  7. 7. Excel shows a 16-digit card-like id as 1.23457E+15. What should you do?