Skip to content
Vibedata

Prove a change is safe before you ship

Prove an existing dbt change is safe to merge without modifying it

  • When a pull request changes dbt models and nobody wants to merge it on faith.
  • When a change looks small in the diff but touches a model several marts depend on.

Build the changed dbt models in an isolated copy of the estate and reconcile the result row by row against the production baseline before the pull request is opened. When the changed object is incremental, this additionally proves an idempotent rerun, unique-key integrity, and the agreed late-arriving-data behaviour. Covers proving the change is safe, not authoring the change itself.

Area
Data quality
Runs on
  • Microsoft Fabric Lakehouse
  • Microsoft Fabric Warehouse
  • MotherDuck
  • DuckDB
Built with
  • dbt
Readiness
SupportedEverything this Recipe composes runs today, without a case that proves this exact shape.

Sample This Recipe has not been materialized in the Cookbook repository yet. Its trigger, description, prompt, agent guidance and acceptance conditions are prototype drafts. Its name, job, area and readiness come from the reconciled Cookbook seed snapshot. Readiness is a separate question from this one: it says whether the capability exists, not whether the writing has been reviewed.

Use this Recipe

Use the VibeData Recipe `prove-dbt-change-safe` at https://getvibedata.ai/cookbook/prove-dbt-change-safe Read the Recipe and execute it in the context of the current Intent.

Recipe id prove-dbt-change-safe · Not yet materialized in the Cookbook repository, so the pointer addresses this page.

Verified by

What has to be observably true before this Recipe is finished.

  • the changed models build clean in the isolated copy with no manual intervention
  • a row-by-row diff against the production baseline is attached to the pull request
  • an incremental change additionally shows a repeated rerun creates no duplicate keys
  • the agreed late-arriving-data case, where applicable, updates exactly the expected records
Recipe promptThe task specification the agent reads. Reference only — it is not what you copy.
Deliver: Prove an existing dbt change is safe to merge without modifying it.

Build the changed dbt models in an isolated copy of the estate and reconcile the result row by row against the production baseline before the pull request is opened. When the changed object is incremental, this additionally proves an idempotent rerun, unique-key integrity, and the agreed late-arriving-data behaviour. Covers proving the change is safe, not authoring the change itself.

Execute inside the current Intent. Its Domain, repository, platform, environment and attached sources are the context for this work — read them rather than asking for them.

The work is done when:
- the changed models build clean in the isolated copy with no manual intervention
- a row-by-row diff against the production baseline is attached to the pull request
- an incremental change additionally shows a repeated rerun creates no duplicate keys
- the agreed late-arriving-data case, where applicable, updates exactly the expected records

Report the evidence for each condition above with the result. A condition you cannot meet is something to say, not something to work around.
Agent guidanceHow the agent approaches the work, and what it will not do.

Change nothing about the object under test; that it was not touched is what makes this a proof. Run both forms over the agreed slice in an isolated copy and report the difference as numbers a reviewer can read without opening the code.

Composes

  • test authoring and baseline comparison
  • dbt in the project

Asks first

Semantic decisions the Intent cannot supply. Never context Studio already holds.

  • which Fabric target this work lands on, when the Domain carries both a Lakehouse and a Warehouse
  • the comparison slice, and what counts as unchanged for this object

Guardrails

  • Build in an isolated copy. Production is read, never written.
  • Do not edit the object being held constant. Parity that required a change to the original is not parity.

There is a change in front of me and I want to know — before anyone reviews it — exactly which numbers move and why. I am not asking for the change to be improved. I am asking whether it is safe.

Someone has changed how fct_revenue calculates closed-won revenue: it was matched on StageName text where it should honour IsWon. The fix is four lines of SQL. The risk is not the SQL, it is that nobody can tell whether those four lines move revenue by nothing, by rounding, or by eleven percent — and nobody finds out until someone in finance does.

Today you either copy the mart to a scratch schema and diff it by hand, or you merge and hope the review catches it. The first takes an afternoon and the second is not a review, it is a wager.

Here the change is left exactly as it is. Your data engineering agent builds it in an isolated copy of your estate — in your environment, on your own platform — runs the current definition and the changed one side by side, and hands back the row-level difference before a reviewer reads a line of the diff.

Nothing about the change is edited. That is what makes this a proof rather than a piece of work: if every implementation change disappeared, you would still have received what you asked for.

What you need

  • The change, as a branch or a pull request. It is read, not rewritten.
  • A deployed baseline to compare against — the production run the current numbers came from.
  • The dbt project that builds it, and somewhere isolated it is allowed to run.

How it goes

  1. Point at the change and say what the numbers are supposed to mean. "Does closed-won honouring IsWon move anything finance reads?"
  2. Agree the comparison slice. Which models, at which grain, over which window — chosen from the change itself rather than from the whole warehouse.
  3. Let it build in the isolated copy. Production is untouched throughout, which is what makes the next step safe to run at all.
  4. Reconcile against the production baseline. You get rows added, rows dropped, and every row whose value moved, with the old and new value side by side.
  5. Read the reconciliation, not the SQL. If eleven opportunities moved, you want to know they were the eleven with ambiguous stage semantics.
  6. Attach the result to the review. The reviewer reads what changed in the numbers, not what changed in the code.

When the changed object is incremental

The contract gets stricter rather than becoming a different Recipe. An incremental model is additionally run twice to prove a repeated run creates no duplicate keys, its unique key is checked for integrity, and the agreed late-arriving case is exercised to prove it updates exactly the records it should.

What you end up with

A merged change, and a written record of which numbers moved and which decision moved them. That record is what you send finance in three weeks when they ask why the quarter restated.

When this is not the Recipe you want

If the model has never been deployed there is no baseline to reconcile against. Build it and prove it against agreed expected results instead — every Build Recipe carries its own proof, so you are not missing a step by skipping this one.

Next to this one