Before this: Override the pandas defaults that change your data
The idea
A mask is a Series of True/False, one per row: df["amount"] > 5000. Combine masks with & and | inside parentheses.
Update with df.loc[mask, "col"] = value. Writing df[mask]["col"] = value may modify a temporary copy and leave df unchanged.
Missing values make comparisons False in pandas, so a mask built on a column with gaps silently excludes them; decide explicitly what they should be.
Worked example
Input: amount: 100, 6000, <NA>
- Visual
Add step → Flag rows → needs_review where amount > 5000
- SQL (DuckDB)
amount > 5000 AS needs_review -- NULL stays NULL in SQL
- pandas
df["needs_review"] = False df.loc[df["amount"] > 5000, "needs_review"] = True
Result: pandas: False, True, False (missing becomes False); SQL: false, true, NULL — decide which you want
Common mistakes
- Chained assignment.
- Using and/or instead of &/|.
- Forgetting how missing values behave in masks.
Check yourself
Why can df[mask]['x'] = 1 fail silently?
The first indexing can return a copy, so the assignment doesn't reach df.
Practice it
Flag large transactions, keep refunds.