Skip to content
Vibedata

Proof

One advisor's warning: the CDO is liable for the number as well as the context

A banking-industry advisor told us the chief data officer is now exposed for the authenticity of reported data, as well as the context agents receive. Our verifying skill traces a requirement to the assets and tests behind it, then records evidence gathered independently at a named commit, so a team can show why a number is right and say where it could not verify.

A white report page sits on a pale powder field, cropped by the right edge, with a large boxed number on it and a navy seal on the signature line below. Three small asset cards labelled ASSETS, TESTS and EVIDENCE stand on the left. Three evidence tags, each stamped with a short commit hash, sit between the cards and the number; cyan lines join each card to a tag and each tag to the number.
The signature is on the number, so the evidence has to travel with it.

By Vibedatacdodata-authenticityverificationregulatory

A banking-industry advisor we spoke with recently put the change in one sentence: the chief data officer is now exposed for the authenticity of reported data, as well as for the context agents receive.

"Not only context, the chief data officer is the data owner. So apart from context, um, more and more they are liable, like it has happened recently, they are liable to provide the authenticity of the data, right? Basically, what data."[4]

That is one advisor's account from one partner call. It still names a shift the buying conversation has been moving toward: the data owner is asked to defend the number, and the agent is one of the things that produced it.

The first question in the room was whether an agent could read the warehouse. Could it find the table, understand the grain, follow the lineage, answer the business? Those questions still matter. The question behind them is older and harder: when the number reaches the board, who can show that it is right?

In the advisor's account, the CDO signs for the number. That makes proof of the number a product requirement.

When the CDO signs the number, the number needs its own proof

The advisor's phrase was the authenticity of the data. A number can be described correctly and still be wrong. A revenue figure can rest on the right definition, the right table and the right filter, and still come from a pipeline whose last run failed.

Context alone does not answer the buyer's question. Context makes a number usable. Proof makes it defensible.

The principle is not new. The Basel Committee's risk-data standard requires a bank to demonstrate the accuracy and completeness of its risk data (BCBS 239).[3] The advisor's point is that the responsibility now has a person's name on it.

For a data engineer, that sounds like a compliance problem. For the team, it is a verification problem. The work is to produce a record that a person can read, check and defend.

Context explains a number; it does not establish that the number is right

A semantic model tells an agent that net revenue means invoice total minus credit notes, that the order date is the booking date, and that the customer table is the one to join. That is necessary. It is not a test.

A context layer can be correct while the number is still wrong. The model can be stale, the filter can be wrong, the upstream table can have a duplicate, the test can have been removed. None of those failures is visible in a definition.

The two artifacts answer different questions. Context answers what this number means. Verification answers how you know it is right, and what you checked.

The buyer wants a record that survives an audit. A better glossary does not provide one.

A requirement is the unit that can be traced and defended

Our verifying skill starts with a requirement: one acceptance unit that says what the change is supposed to make true. The skill seeds one Coverage row for every requirement and one for every scope row in the plan, before it runs a check.[1][2]

If the row is written after the evidence, the gap becomes a matter of memory. If it is written first, an uncovered row is structural and visible.

For each row, the skill names the data assets that implement the requirement with describe_asset, and reads their impact closure with get_lineage. It calls list_assets with since set to the commit the evidence was gathered at, before any diff or gate.[1]

The output is a table a person can read: the requirement, the assets, the evidence, the result, the commit. The person who signs the number can see the row for the number they are signing without reading a dbt graph.

A horizontal coverage diagram. A requirement card, R-01, flows to an assets card naming describe_asset and get_lineage, then to an independent evidence card naming a query, count and boundary case, then to a coverage row card showing verified, returned or blocked. A declared tests card branches upward from assets and is labelled a pointer, not evidence. A commit card below the evidence is stamped at the evidence commit and connects to the row.
A coverage row carries the requirement, its assets, independent evidence and the commit it was gathered at.

Declared tests point at what to check, not at what proves it

The most common mistake is to treat a green test run as the proof. A declared test is a pointer. It tells the skill where to look and what the team expected. It is not independent evidence of the number.

A test can pass because it was weakened, because it covers a different grain, or because it was never run against the rows that matter. The skill treats declared tests and earlier results as pointers to what to check, and never as a Coverage row's evidence.[1]

Evidence is gathered independently of the code under test: a query that recomputes the metric from the source, a uniqueness count at the grain, a population count, a boundary case. A data claim needs a data test on real rows. A unit test on a fixture proves logic, and the skill keeps the two instruments separate.[2]

QuestionWhat answers itWhat does not
Is the logic right?A unit test on a fixtureA green build
Is the number right?A data test or independent query over real rowsA passing unit test
Is the number still right?Evidence gathered at the current commitLast month's result
Did anything change?since on the asset and its closureA green build with no diff

A green build proves one thing: the code builds.

Evidence has to carry the commit it was gathered at

The skill stamps every row with the revision the evidence was gathered at. That stamp is what stops a confident answer from going stale.

When anything in the row's impacted paths changes after that commit, the evidence is cleared and re-gathered. That includes the artifact, its upstream, its tests, the macros and the project config. If the recorded revision is unreachable, every row is treated as stale.[2]

The understanding tools make the mechanical part faster. list_assets with since reports which assets are changed, unchanged or new. Macros and project config still need a diff. The rule is the same either way: evidence without a current revision is not evidence.[1]

The stamp makes the record traceable. A reviewer can take the row, see the commit, and re-run the same query against the same data; with a preserved snapshot of the source rows and environment, they can compare answers. Without that snapshot, the commit only pins the code and the query; the data can move underneath it, so the same answer is not guaranteed. A dashboard screenshot cannot show even the commit.[1]

A system that cannot verify has to say so

The most dangerous output is a green check for something the skill could not check.

The skill reads each asset's latest result and freshness through describe_asset, and writes a status that is not current into the row: stale, failed or never run. It does not promote a missing result to a pass.[1]

The verification ends with one of three verdicts: verified, returned or blocked. A gate on the intent's own rows that could not run is blocked, never verified with a limit. A limited verification is not a verified one.[1][2]

This is also the honest answer to the buyer. The record shows what was checked, at which revision, against which data, and what remains unverified. The limits are part of the proof.

A product cannot take that liability away. It can make the account defensible.

The buying question moves from context to coverage

For the AI-native data engineer, the useful change is that the trace is automatic enough to run before every pull request, and specific enough to argue with. The row names the asset, the evidence and the commit. A reviewer can reject the evidence without rejecting the whole run.

For the data reliability engineer, the useful change is the staleness rule. An alert that points at a number without a revision is a new incident waiting to happen. A row that clears itself when the impacted paths move is a smaller, more honest queue.

For the buyer, the question moves from whether the agent can read the data to whether the team can show why this number is right, and where it could not. That is a shorter conversation, and a harder one to win with a demo.

The productivity claim is that the same team ships more with Vibedata: less time to PR and less time per PR, once the team changes what it focuses on and how it builds, and review compounds as each accepted change shrinks the next review. We have not measured verification's effect on either measure, so the claim here is the mechanism.

Start with one number your team reports

Start with one number that someone signs. No new platform is required.

  1. Pick one number the business reports, preferably one a CDO would sign.
  2. Write the requirement as a sentence that could be false.
  3. Name the assets that implement it and their closure. List the declared tests as pointers to what to check.
  4. Gather evidence independently of the code: a query from source, a uniqueness count at the grain, a boundary case.
  5. Stamp the evidence with the commit. When the impacted paths move, clear the row.
  6. Write down what you could not verify. That line is part of the record.

Then hand the row to the person who signs the number. What matters is whether the row gives them something they can defend, whatever the pipeline status says.

Which number would you least like to defend without its coverage row?

References

  1. Verifying skill, plugins/vibedata-data-engineering/skills/verifying/SKILL.md, accelerate-data/vibedata-data-engineering (private repository).
  2. Verifying functional specification, docs/functional/plugin-skills/verifying/README.md, same private repository.
  3. Basel Committee on Banking Supervision, Principles for effective risk data aggregation and risk reporting.
  4. Partner call with a banking-industry advisor, anonymized.

Get the weekly roundup by email

One email a week: what got easier in Vibedata, a recipe to try yourself, and where data engineering is heading.