Data Cleaning Exercises

Data Contracts Explained for Analysts

What a data contract is, what goes in one, how it differs from a data dictionary or tests, and how an analyst can start one, with an ODCS example and SQL checks.

By Tnas · Published · 10 min read

A diagram of a data contract between a producer team and an analyst, listing schema, meaning, quality rules, freshness, owner and change process

A data contract is a written, versioned agreement between the team that produces a dataset and the people who use it. It says what columns the data has and what they mean, which quality rules it meets, how fresh it will be, who owns it, and how changes will be announced. The point is simple: when an upstream team renames a column or adds a new status code, the change should arrive as a planned update, not as a broken dashboard on Monday morning.

The term is often discussed as an engineering topic, but analysts are usually the people who find out when an unwritten agreement breaks. This post explains what goes into a contract, how it differs from things you may already have, and how to start one from the consumer side, with an example in the Open Data Contract Standard format and the SQL checks it turns into.

Key Takeaways

  • A data contract covers five things: schema, meaning, quality rules, service levels such as freshness, and ownership with a change process.
  • It differs from a data dictionary because it is a commitment with an owner and a version, and from tests because it is agreed with the producer rather than written alone by the consumer.
  • Open standards exist. The Open Data Contract Standard (ODCS) defines a YAML format; dbt model contracts enforce column names and types at build time.
  • Analysts can start one by writing down the columns and rules their work depends on, agreeing them with the producer, and checking them on every load.

What problem does a data contract solve?

It replaces assumptions with agreements. Every report rests on beliefs about its inputs: order_id is unique, status is one of three values, amount is in euros, yesterday's data is there by 6 a.m. When those beliefs are unwritten, the producer has no way of knowing they exist, and any change they make can break something they have never heard of.

Three kinds of upstream change are the usual sources of trouble:

  • Shape changes: a column renamed, removed or retyped.
  • Meaning changes: the same column now holds something different, such as amount switching from gross to net, or a new status code appearing.
  • Delivery changes: the data arrives later, less often, or only partly.

Shape changes at least tend to fail loudly. Meaning and delivery changes usually do not: the query still runs and the numbers are simply wrong. A contract makes all three explicit, so they can be checked and negotiated.

What goes into a data contract?

A contract covers five areas. The names vary between standards, but the content is consistent:

Area What it states Example
Schema Columns, types, keys, which fields are required order_id text, primary key, required
Meaning What each column represents, units, allowed values amount is the order total in EUR including VAT
Quality Rules the data must meet, with thresholds No duplicate order_id; no null customer_id
Service levels Freshness, delivery schedule, retention Data for day D complete by 06:00 UTC on D+1
Ownership and change Who owns it, how to reach them, how changes are versioned and announced Breaking changes get a new major version and 30 days' notice

The Open Data Contract Standard, published by the Bitol project, organises a contract into sections such as fundamentals, schema, data quality, support and communication channels, team, roles and service-level agreement, and says a contract should be platform agnostic (Bitol, Open Data Contract Standard). You do not have to adopt ODCS to benefit from a contract, but its section list is a useful checklist.

What does a data contract look like?

Here is a small contract for an orders table, written with field names from the ODCS example contract and its data-quality library. It is trimmed to what an analyst would care about first:

kind: DataContract
apiVersion: v3.1.0
id: orders-contract
version: 1.2.0
status: active
description:
  purpose: One row per customer order, for revenue and fulfilment reporting.
schema:
  - name: orders
    physicalType: table
    properties:
      - name: order_id
        logicalType: string
        primaryKey: true
        required: true
        quality:
          - metric: duplicateValues
            mustBe: 0
      - name: customer_id
        logicalType: string
        required: true
        quality:
          - metric: nullValues
            mustBe: 0
      - name: status
        logicalType: string
        description: Order status. Allowed values are pending, shipped and cancelled.
        quality:
          - metric: invalidValues
            arguments:
              validValues: ['pending', 'shipped', 'cancelled']
            mustBe: 0
      - name: amount
        logicalType: number
        description: Order total in EUR including VAT.
      - name: updated_at
        logicalType: timestamp
slaProperties:
  - property: latency
    value: 1
    unit: d
team:
  name: orders-platform
support:
  - channel: '#orders-data'
    tool: slack

The quality rules use the standard's library metrics (nullValues, invalidValues, duplicateValues and others), each compared with an operator such as mustBe or mustBeLessThan (Bitol, ODCS "Data Quality"). Treat the exact apiVersion as whatever version your tooling supports; the structure is what matters here.

Notice what the contract does not do: it does not describe how the producer builds the table. It describes only what the consumer can rely on.

How does a contract become checks?

A contract is only useful if something checks it. Each rule maps to a query that measures one number and compares it with the threshold. Here is a day's orders table with three upstream problems: a duplicated order, missing customer ids, and a status value the producer added without telling anyone.

SET TimeZone = 'UTC';

CREATE TABLE orders AS
SELECT * FROM (VALUES
  ('O-1001', 'C1', 'shipped', 49.90, TIMESTAMPTZ '2026-11-04 18:20:00+00'),
  ('O-1002', 'C2', 'pending', 12.00, TIMESTAMPTZ '2026-11-04 19:05:00+00'),
  ('O-1003', NULL, 'shipped', 80.00, TIMESTAMPTZ '2026-11-04 19:40:00+00'),
  ('O-1003', NULL, 'shipped', 80.00, TIMESTAMPTZ '2026-11-04 19:40:00+00'),
  ('O-1004', 'C4', 'on_hold', 15.00, TIMESTAMPTZ '2026-11-04 21:10:00+00')
) t(order_id, customer_id, status, amount, updated_at);

The contract's rules, as one query (the run time is fixed at 06:00 UTC on 5 November so the output is reproducible; use now() in production):

WITH m AS (
  SELECT count(*) - count(DISTINCT order_id) AS duplicate_order_ids,
         count(*) FILTER (WHERE customer_id IS NULL) AS null_customer_ids,
         count(*) FILTER (WHERE status NOT IN ('pending', 'shipped', 'cancelled'))
           AS invalid_statuses,
         date_diff('hour', max(updated_at), TIMESTAMPTZ '2026-11-05 06:00:00+00')
           AS hours_since_update
  FROM orders
)
SELECT 'order_id: duplicateValues = 0' AS rule, duplicate_order_ids AS value,
       duplicate_order_ids = 0 AS passed FROM m
UNION ALL
SELECT 'customer_id: nullValues = 0', null_customer_ids, null_customer_ids = 0 FROM m
UNION ALL
SELECT 'status: invalidValues = 0', invalid_statuses, invalid_statuses = 0 FROM m
UNION ALL
SELECT 'latency <= 24 hours', hours_since_update, hours_since_update <= 24 FROM m;

┌───────────────────────────────┬───────┬─────────┐
│             rule              │ value │ passed  │
├───────────────────────────────┼───────┼─────────┤
│ order_id: duplicateValues = 0 │     1 │ false   │
│ customer_id: nullValues = 0   │     2 │ false   │
│ status: invalidValues = 0     │     1 │ false   │
│ latency <= 24 hours           │     9 │ true    │
└───────────────────────────────┴───────┴─────────┘

With a contract, each failure has a clear next step. The duplicate and the null customer IDs are the producer's breach, so the report goes to the owner named in the contract. The on_hold status is a change that skipped the change process: either the producer adds it to the contract in a new version, or they stop sending it. Without a contract, all three would be "bad data" that an analyst quietly patches in a downstream query, and the same problems would come back next week. The exercise errors versus warnings practises deciding which failures block a load, and the guide on writing a data quality report covers how to send findings back to a producer.

How is a contract different from a data dictionary, tests or a schema?

These overlap, and a contract often reuses all of them. The difference is who commits to what:

Who writes it What it promises Enforced?
Data dictionary Usually the producer or a data team Describes columns as they are No; it can drift from reality
Tests Usually the consumer Checks what the consumer assumes Yes, but the producer may not know they exist
Schema The producer's system Names and types only Yes, for shape only
Data contract Producer and consumer together Shape, meaning, quality, service levels, change process Yes, and failures have an owner

A data dictionary is the natural starting point: a contract is roughly a dictionary that someone has agreed to keep true, plus rules and a change process. The lesson check the data against its dictionary shows the gap that appears when nobody does.

Tooling draws the same line between shape and content. dbt's model contracts declare every column's name and data type, and a "preflight" check stops the model from building if its query returns a different shape; dbt's documentation is explicit that contracts validate shape at build time while data tests validate content afterwards (dbt docs, "Model contracts"). A full data contract needs both.

What counts as a breaking change?

A breaking change is one that can make a correct downstream query wrong or make it fail. dbt's contract documentation lists removing a column, changing a column's data type, and removing or modifying a constraint as breaking changes (dbt docs, "Model contracts"). From an analyst's side, meaning changes belong on the list too, even though no type changes:

  • Breaking: removing or renaming a column, changing a type, making a required field optional, changing units or the definition of a value, adding a value to a column that reports filter or group on, increasing the delivery delay beyond the agreed level.
  • Usually non-breaking: adding a new optional column, adding documentation, tightening a rule (fewer nulls than promised).

A common convention is semantic versioning, which the contract's version field carries: a breaking change raises the major version (1.x to 2.0) and comes with notice and, ideally, a period where both versions are available. Adding a value to a category list deserves special attention, because producers often treat it as harmless. The lesson reviewing a mapping change shows why a new code is never just a new code downstream.

How can an analyst start a data contract?

Start from what you consume, not from a standard. You do not need permission to write down your own dependencies, and that list is the first draft of a contract.

  1. List the columns your work actually uses for one important dataset, with the type and meaning you assume for each. Most reports use far fewer columns than the source has, and that narrows the conversation.
  2. Write down the rules you rely on: which key is unique, which fields must be present, which codes you expect, when the data must arrive. If you are not sure, profile the data and ask; the guide on clarifying questions to ask about messy data has a list to start from.
  3. Turn each rule into a check that runs on every load, like the query above. Now the assumptions are visible, and a failure tells you which one broke. The checks are the same ones described in data quality checks every pipeline should run, and for file deliveries the header and type checks in validating a CSV's schema before loading come first.
  4. Take it to the producer. Ask them to confirm, correct or reject each line, and to name an owner and a channel for changes. Expect some rules to be rejected; finding out that status has eleven values, not three, is the contract doing its job.
  5. Version it and keep it next to the code, so a change to the contract is reviewed like a change to the pipeline.

The conversation in step 4 is a stakeholder skill as much as a technical one. The interview questions on stakeholders are good practice for it, and the lesson on writing the handoff note covers documenting the rules you agreed.

Frequently asked questions

No. It is an internal technical agreement between teams, usually a YAML or Markdown file in version control. It has no legal force; what makes it work is that it has an owner, it is checked automatically, and changes go through an agreed process.

Do I need a special tool to use data contracts?

No. A contract can start as a table in a document and a set of SQL checks. Standards such as ODCS and tools that read them help once you have several contracts, because they make the format consistent and the checks automatic.

Who owns a data contract?

The producer owns the promise, because only the producer can keep it. Consumers own the request: they say what they depend on and raise breaches. Both sign off on changes.

Sources

The SQL in this post was run on DuckDB 1.5.1, and the output shown is from that run with the type row removed.