Reference
Operation reference
12 cleaning operations, each explained for the visual workspace, SQL (DuckDB) and pandas, with the defaults that differ between them.
| Operation | Visual workspace | SQL (DuckDB) | Python (pandas) |
|---|---|---|---|
| Trim whitespaceRemove a declared set of characters from both ends of a text value without touching the inside. | Visual workspace | SQL (DuckDB) | Python (pandas) |
| Change caseLowercase or uppercase every value so that spellings differing only in case compare equal. | Visual workspace | SQL (DuckDB) | Python (pandas) |
| Turn blank text into missingConvert empty text into a missing value so blanks and missing values are handled by one rule. | Visual workspace | SQL (DuckDB) | Python (pandas) |
| Convert a column's typeParse text into a typed value (integer, exact decimal, date, timestamp with time zone, boolean) and decide what happens to values that don't parse. | Visual workspace | SQL (DuckDB) | Python (pandas) |
| Check allowed valuesQuarantine rows whose value is not in an approved list. | Visual workspace | SQL (DuckDB) | Python (pandas) |
| Map values with a dictionaryReplace each spelling with its approved value using a reviewed dictionary, and decide what happens to unmapped values. | Visual workspace | SQL (DuckDB) | Python (pandas) |
| Fill missing valuesReplace missing values with a stated value: a visible label for a category, or a fitted value with a flag for a measurement. | Visual workspace | SQL (DuckDB) | Python (pandas) |
| Flag rowsAdd a true/false column that marks rows matching a condition, keeping every row. | Visual workspace | SQL (DuckDB) | Python (pandas) |
| Remove duplicates with a survivor ruleKeep one row per key, chosen by a complete deterministic order, and report the others. | Visual workspace | SQL (DuckDB) | Python (pandas) |
| Look up values from another tableBring columns from a lookup table by key, keeping every row and never matching missing keys. | Visual workspace | SQL (DuckDB) | Python (pandas) |
| Pivot long data to wideSpread a key column into declared columns, one row per index, leaving absent combinations missing. | Visual workspace | SQL (DuckDB) | Python (pandas) |
| Unpivot wide data to longTurn several columns into key/value rows, without inventing rows for missing values. | Visual workspace | SQL (DuckDB) | Python (pandas) |