Before this: Import everything as text first
The idea
Text files are bytes. UTF-8 stores é as two bytes; reading them as Windows-1252 shows é. That garbage is called mojibake.
Fix it at import: declare the encoding the file was written in. Search-and-replace afterwards misses cases and can corrupt correct text.
If you don't know the encoding, test a few candidate decodings on rows with accented names and pick the one that reads correctly; record it.
Worked example
Input: bytes for 'José' written as UTF-8, read as cp1252
- Visual
Import settings → Encoding: UTF-8
- SQL (DuckDB)
read_csv('people.csv', encoding = 'utf-8', all_varchar = true)- pandas
pd.read_csv("people.csv", encoding="utf-8", dtype=str)
Result: José instead of José
Common mistakes
- Letting the tool guess the encoding.
- Repairing mojibake after the fact.
- Mixing files with different encodings without noting it.
Check yourself
Why not fix é with search and replace?
It misses other characters and may change correct text; declare the right encoding instead.
Practice it
Normalize names without changing letters.