Guide · updated 2026-11-04
Reading CSV files without changing your data
Delimiters, quoting, encodings and type inference: how to import raw files so nothing changes before your rules run.
Read everything as text first
CSV has no types. Readers guess, and guesses drop leading zeros, misread dates and round long ids. Read every column as text, keep the raw file untouched, and convert columns later with explicit rules.
Declare the format
State the delimiter, quote character, encoding and header row. European exports often use semicolons; legacy systems often use Windows-1252. A wrong encoding turns é into é.
- pandas: dtype=str, keep_default_na=False, encoding=…, sep=…
- DuckDB: read_csv(..., all_varchar=true, delim=…, header=true)
Check what you loaded
Compare the row count with the source system or the file's line count, look for rows with too many or too few fields, and record file name, size and delivery date for lineage.