Before this: Trim the right characters, and only those
The idea
"é" can be stored as one character or as "e" plus a combining accent. Both look the same and compare as different. NFC normalization composes them into one form.
Transliteration (José → Jose) is not normalization. It changes the name and can merge different people. Keep the letters; normalize the form.
Unusual spaces such as no-break (U+00A0) and ideographic (U+3000) also hide in names. Trim them with a Unicode-aware character set when the rules say so.
Worked example
Input: "Jose\u0301" (decomposed) and "José" (composed)
- Visual
Trim name (all Unicode whitespace) → Normalize Unicode (NFC)
- SQL (DuckDB)
nfc_normalize(name)
- pandas
df["name"].str.normalize("NFC")
Result: Both become "José" and compare equal
Common mistakes
- ASCII-folding names.
- Trimming only ASCII spaces when the rules say Unicode whitespace.
- Using NFKC/NFKD when the rules say NFC.
Check yourself
Is removing accents a valid normalization here?
No. It changes the name; use NFC.
Practice it
Trim and NFC-normalize every name.