Data Cleaning Exercises

Guide · updated 2026-11-04

Dates and time zones for analysts

Ambiguous date formats, timestamps with and without offsets, instants versus local times and daylight saving, explained with cleaning rules.

Dates: declare the convention

A slash date like 03/04/2026 is 4 March in the month-first convention and 3 April in the day-first one. ISO 8601 dates (2026-04-03) are unambiguous. Parse slash dates only with an explicit, declared convention per row or per file, and quarantine the rest as ambiguous.

Timestamps: an offset makes an instant

2026-02-01T01:00:00+01:00 and 2026-02-01T00:00:00Z are the same instant written two ways. A timestamp without an offset is a local time in an unknown zone, not an instant. Store instants in UTC, and quarantine offset-less timestamps unless the rules say which zone they use.

Compare instants, not text

Sorting timestamps as text compares characters. With mixed offsets, the text order and the time order differ. Parse to a timestamp-with-time-zone type before choosing the latest record.

Local days and daylight saving

To count events per local day, convert each instant to the user's IANA time zone (such as Europe/Berlin) and take the date. Fixed offsets are wrong for half the year in zones with daylight saving, and a local day can be 23 or 25 hours long.

Practise it