pandas cleaning and debugging

1. mid

   1. Name two pandas defaults that can silently change data during cleaning, and how you override them.

   What it tests: Awareness of pandas defaults.

   Model answer and scoring

   merge matches missing keys to each other: drop missing keys from the lookup side first. read_csv converts text like "NA" to missing and infers types that drop leading zeros: use dtype="string" and keep_default_na=False. Also .str.strip() without arguments removes all Unicode whitespace, and float arithmetic on money adds rounding error; use Decimal.

   Strong answers

   • At least two precise defaults with fixes.

   Red flags

   • Vague "pandas can be tricky".

   Practise it: Join orders to customers without matching blanks

2. junior

   2. Why did an integer column turn into floats after you introduced missing values?

   What it tests: Nullable dtypes.

   Model answer and scoring

   NumPy's int64 can't represent missing values, so pandas falls back to float64 with NaN. The nullable Int64 dtype keeps integers and uses <NA> for missing values.

   Strong answers

   • Names Int64.

   Red flags

   • Fills missing with 0 to keep ints.

   Practise it: Label unknown categories, keep missing numbers missing

3. mid

   3. You sort and then drop duplicates to keep a survivor, but results vary between runs. Why?

   What it tests: Stable sorting and complete orderings.

   Model answer and scoring

   Either the ordering isn't complete (ties are left to sort order) or the sort isn't stable. I sort by every survivor rule column including a unique tie-break such as id, and use kind="mergesort" for a stable sort before duplicated().

   Strong answers

   • Complete ordering plus stable sort.

   Red flags

   • Doesn't know drop_duplicates keeps the first row by position.

   Practise it: Resolve customer records by normalized email

4. senior

   4. How would you structure pandas cleaning code so it can run on next month's file?

   What it tests: Reproducible pipelines.

   Model answer and scoring

   A clean() function that takes raw frames and returns clean and exception frames without mutating inputs; explicit dtypes on read; rules as named steps; and tests on a few hand-made edge cases plus a reconciliation check that input rows equal clean plus exceptions. Then it can run unchanged on a new batch.

   Strong answers

   • Pure function, explicit types, tests, reconciliation.

   Red flags

   • A notebook with cells run in a remembered order.

5. junior

   5. Why does df[df.x > 0]['flag'] = 1 not update df?

   What it tests: Chained assignment.

   Model answer and scoring

   Under Copy-on-Write (the only mode in pandas 3.0), df[df.x > 0] behaves as a new object, so the second assignment changes that temporary result and never df; pandas warns with ChainedAssignmentError.

   I use one indexing step: df.loc[df.x > 0, 'flag'] = 1.

   Strong answers

   • Explains Copy-on-Write.
   • Uses .loc in one step.

   Red flags

   • Ignores the warning.

6. junior

   6. How do you read a CSV in pandas without changing any values?

   What it tests: Explicit import.

   Model answer and scoring

   pd.read_csv(path, dtype=str, keep_default_na=False) keeps every value as text, including leading zeros and literal strings like NA, and empty fields as empty strings.

   Then I convert columns with explicit rules and decide where blanks become missing.

   Strong answers

   • dtype=str and keep_default_na.
   • Explicit conversions.

   Red flags

   • Relies on inference.

   Practise it: Keep identifiers as text and parse yes/no flags

7. mid

   7. How do you keep money exact in pandas?

   What it tests: Exact arithmetic.

   Model answer and scoring

   Floats can't represent cents exactly. I parse amounts into decimal.Decimal objects or integer cents and only format at the end.

   I validate the text format first so a value like 1.234 isn't silently rounded.

   Strong answers

   • Decimal or integer cents.
   • Validates before converting.

   Red flags

   • Uses float and rounds at the end.

   Practise it: Flag large transactions, keep refunds

8. mid

   8. A groupby on region loses rows with a missing region. Why and how do you fix it?

   What it tests: groupby defaults.

   Model answer and scoring

   groupby drops missing keys by default (dropna=True).

   I either label them explicitly (fillna('unknown')) if the rules allow, or use dropna=False so they appear as their own group.

   Strong answers

   • Knows dropna default.

   Red flags

   • Doesn't notice lost rows.

   Practise it: Label unknown categories, keep missing numbers missing

9. junior

   9. How do you turn one column per question into one row per answer?

   What it tests: Reshaping.

   Model answer and scoring

   df.melt(id_vars=['respondent'], value_vars=['q1','q2','q3'], var_name='question', value_name='score'), then drop rows where score is missing.

   Unanswered questions are not scores, so they are dropped rather than filled.

   Strong answers

   • Uses melt.
   • Drops missing scores.

   Red flags

   • Fills unanswered with 0.

   Practise it: Unpivot survey answers without inventing scores

10. senior

    10. How do you make a pandas cleaning function trustworthy for a team?

    What it tests: Testing and contracts.

    Model answer and scoring

    I write clean(df) as a pure function that returns clean rows and exceptions, with tests for edge cases (padding, missing keys, duplicates, bad types) and invariants (row reconciliation, key uniqueness, dtypes).

    I pin dtypes with nullable types and check the output schema before returning.

    Strong answers

    • Pure function, tests, invariants.
    • Schema check.

    Red flags

    • Notebook cells run by hand.

Model answers are original study material for practice. They are not employer-endorsed and don't guarantee an outcome.