Reference · Visual workspace
Remove duplicates with a survivor rule in Visual workspace
Keep one row per key, chosen by a complete deterministic order, and report the others.
When to use it
Repeated records at the table's grain: re-sent rows, versions, the same customer from several systems.
How
Add step → Rows → Remove duplicates. Tick the key columns, add order columns for which row survives, and give a reason for the others.
One row per (order_id, line_id), keep first by version desc
- Convert columns to their types first so 10 sorts above 9.
- Missing key values group together; require a value first if they must not be merged.
Pitfalls
- Deduplicating at the wrong grain.
- An incomplete order that leaves ties to chance.
- Comparing numbers or timestamps as text.
Lesson: Find the row grain before removing duplicates
Practise it: Deduplicate order lines at the right grain