Skip to content
Vibedata

Provocation

A data contract is a published boundary, not documentation

Tests enforce what someone remembered to assert. A data contract states the whole promise for one relation, and the platform tests are generated from it.

A survey stake driven into a pale blue field, its white shaft cropped by the bottom edge. A small white plaque with two cyan rules sits on the shaft. A navy cap at the top carries a white stamped seal, and a taut navy line runs from the cap to the left edge of the frame.
A promise you can point to.

By Vibedatadata-contractsodcssemantic-modeldbtdlt

Recently, in our daily engineering scrum, the decision took one sentence: "Something which is an Apache standard on data contracts. So we'll make that as a first-class object within our context."

The standard is ODCS v3.2.0. Its home is beside the Domain Data Model and the ontology. The contract owns the promise. The model and the ontology own the meaning.

That distinction is the argument. A data contract states what a consumer can hold you to for one relation: its grain, its properties and types, its keys and relationships, its quality rules, and the platform that serves it (Bitol). The standard's own definition is plain: "A data contract defines the agreement between a data producer and consumers" (Bitol).

When that statement lives only in tests, it is the union of the assertions someone remembered to write.

A test asserts a column; a contract states the promise

A dbt test says a column is not null, unique, or accepted by a list. dbt can also enforce a model's output columns and types at build time. Neither states the whole promise for a relation.

The gap is narrow and real. A dropped or renamed column that carries a test fails the build. One that carries no test disappears silently, because no artifact states the promise as a whole.

The tools are split by job. dbt's build-time check covers models only. It says nothing about the raw relations a loader creates, the seeds, or any relation a non-dbt job reads. dbt tests are hand-written per column, so the promise exists only as the sum of what someone remembered to assert.

A data contract is authored first. You profile the data, write the contract, generate the enforcement, and then write the code to satisfy it. The contract states the promise before the code exists, and the build refuses a change that breaks it.

Our reference domain is the test. Its contract branch carries 93 contracts, one per relation across ten schemas: 63 dbt models, four seeds, and 26 dlt-loaded raw tables. The quality rules moved out of the dbt properties files and into the contracts, and the generator writes them back. The same 150 data tests run before and after.1

That last number matters. The contracts did not add a parallel test suite. They gave the existing tests a source, and they covered the relations the tests never reached.

The Domain Data Model and the Ontology sit at the top, owning analytical meaning and business identity. Below them, Physical columns feed the Data contract where the model declares no field. The Data contract is the focus, carrying the promise for one relation. It generates the enforcement below it: dbt enforcement and tests, and dlt column hints. Those enforce the Built relation, and the relation check compares the built relation back to the contract.
The contract is the boundary; the model and the ontology keep the meaning.

The contract is a third category, and it owns only the promise

The Domain Data Model is the source of truth for the analytical layer: datasets, fields, logical sources, metrics, relationships, and the physical inventory. The ontology is the source of truth for business identity: concepts and relationships, and the only artifact that says an object is the same object across models.

The data contract sits beside them. It owns the promise for one relation and never the meaning.

Where the model declares a field, the contract projects that field's logical type from the model. Where the model is silent, the contract takes the physical column's type. A disagreement about a declared field resolves in the model's favour.

ArtifactWhat it ownsWhat it may not do
Domain Data ModelAnalytical meaning: datasets, fields, logical types, metrics, relationships, physical inventoryPromise a shape to a consumer
OntologyBusiness identity: concepts and relationships across modelsCarry measures, physical detail, or an executable rule
Data contractThe promise for one relation: grain, properties and types, keys, relationships, quality, serving platformOriginate a business definition
PlatformWhere the relation physically lives, and how the platform enforces itDecide what the promise is

The separation tells you which artifact to change when something moves. A definition changes in the model or the ontology. A promise changes in the contract. A platform-specific test changes because the generator re-renders it.

A catalog lists what exists. A contract states what a consumer is promised, and a build can refuse a change that breaks it.

The model is a curated view. In our reference domain it declares 149 fields on 17 of the 34 gold tables, and 59 of the 93 relations have no canonical entry at all. A design that derives the contract from the model cannot cover the majority of what needs a contract. That is why the contract is authored, and why it projects only the fields the model declares.1

Generated enforcement is what keeps the boundary from drifting

The obvious failure of a separate artifact is drift. The data contract says one thing, the warehouse does another, and nobody notices until an incident.

The answer is not to hand-write the same promise twice. It is to generate the platform enforcement from the contract.

For dbt, the generator writes an enforced contract, a data type per column, and the quality rules as data tests. For dlt, it writes column hints under the freeze each pipeline already declares. A hand edit to the generated half fails the check instead of surviving.

There is a floor on gold. If a gold relation's canonical document declares fields, the contract must declare every one of them. A contract that promises nothing cannot pass by staying quiet.

The checks split in two. A tool proves the structure: schema validity, one contract per relation, no orphans, no empty sections, the platform declaration, gold agreement, and current generated keys. A reviewer proves the judgement: whether the grain is true, whether required is honest, whether a rule encodes the real business answer, and whether a declared change is breaking.

What the tool provesWhat the review decides
The contract validates against the ODCS schemaThe grain is true
One contract per relation, no orphan contractsrequired is honest
No empty section, no key outside the standardA quality rule encodes the business answer
The platform declaration matches the mapA declared change is breaking
Gold agreement on declared fieldsThe contract means what its document means
Generated dbt keys and dlt hints are currentEvery finding is answered or visibly open

The reference domain demonstrates both directions. A dbt model that gains a column with no contract entry fails its build. A dlt source that yields an undeclared column fails its load, and the rows from the first load stay readable. A drop is different: the freeze misses it, so the relation check catches it at verification.

The duplication objection is right about the risk and wrong about the alternative

The strongest objection deserves its full weight. Data contracts already exist as dbt tests and column hints. A separate object duplicates the catalog, adds a file to maintain, and can drift from the warehouse. Keep the tests; skip the artifact.

The first half is true. A hand-maintained contract beside a hand-maintained test suite is two promises, and they will disagree.

Skipping the artifact leaves a promise that exists only as the union of tests someone remembered to write.

dbt's build-time check covers models only. It says nothing about the 26 raw relations a loader creates, the four seeds, or any relation a non-dbt job reads.1 The tests that do exist are per column. A relation can lose a column that no test names and pass every check.

The catalog lists columns. The contract states grain, keys, relationships, quality, and the serving platform. None of those are in the column list.

ObjectionWhere it holdsWhat changes it
A separate contract duplicates the catalogWhen the contract is hand-maintained beside the testsGenerate the enforcement; check the contract against the model
A separate contract can drift from the warehouseAlways, if nothing compares it to the built relationRun the relation check; a missing catalog fails rather than skips
The tests already enforce the promiseFor the columns someone wrote a test forState the whole promise, then generate the tests from it

Drift is a real risk. It is the reason the contract is checked against the built relation as well as its own schema. A contract that passes a file check and describes a table that no longer exists is still a broken promise.

The objection gets the cost right. The alternative it proposes keeps the cost and loses the boundary.

Author the contract before the code

The order is the method. Profile the data, write the contract, generate the enforcement, then write the code to satisfy it. A new relation with no landed table is declared from its design, the profile runs on the first build, and any question the profile raises goes to a review artifact instead of being guessed into the contract.

Onboarding an existing repo runs the other way. A reverse pass declares only what the catalog, the existing tests, and the descriptions establish. It invents no quality rule, no key, and no required: true. Every doubt becomes a finding with a question that settles it.

The contract only needs the sections that assert something. ODCS has pricing, support, team, roles, and SLA sections. A domain that has no answer for them omits them. An empty section claims a promise nobody made.

The standard is platform-agnostic. "The contract should be platform agnostic." That is the standard's own line (Bitol).

The enforcement is platform-specific. That is the split: one contract, many platform renderings.

Here is the smallest version of the method.

  1. Pick one relation your team argues about.
  2. Write its grain in one sentence.
  3. List the properties and types a consumer is promised.
  4. Name the keys, relationships, and quality rules that would make a change breaking.
  5. Generate the platform enforcement, then compare the contract to the built relation.

The gate also earns back its cost. Each contract that catches a break before merge removes a review question from the next change. The productivity claim is that the same team ships more with Vibedata: less time to PR and less time per PR, once the team changes what it focuses on and how it builds, and review compounds as each accepted change shrinks the next review.

Which relation in your warehouse has a promise that exists only as the tests someone remembered to write? Write that promise down before you write the next test. If the contract and the test disagree, the gate finds it before the incident review does.

References

Footnotes

  1. Reference-domain implementation of ODCS v3.2.0 and the accepted data-contract decision: 93 contracts across ten schemas, 63 dbt models, four seeds, 26 dlt-loaded raw tables, 150 data tests, 149 declared fields on 17 of 34 gold tables, and 59 of 93 relations without a canonical entry. ↩ ↩2 ↩3

Get the weekly roundup by email

One email a week: what got easier in Vibedata, a recipe to try yourself, and where data engineering is heading.