Before this: Identifiers are text; flags are booleans
The idea
float64 holds about 15–17 significant digits. An 18-digit id converted to float can change its last digits, merging different records.
Spreadsheets show such ids in scientific notation and may save them that way.
Read ids as text from the source and compare them as text.
Worked example
Input: id: '123456789012345678'
- Visual
(no Convert type on id)
- SQL (DuckDB)
id stays VARCHAR
- pandas
pd.read_csv(f, dtype={"id": "string"})
Result: id unchanged; as float it would print 1.2345678901234568e+17
Common mistakes
- Numeric inference on ids.
- Saving ids through spreadsheets.
- Comparing ids as numbers.
Check yourself
Why can't float64 store every 18-digit integer?
It has 53 bits of precision, about 15–17 decimal digits.
Practice it
Keep identifiers as text and parse flags.