Skip to content
Vibedata

Use cases

Agentic data engineering on the platform you already run

Your data engineering agent covers three use cases, and they run as one process. Data Product Delivery builds a change, Data Quality Assurance proves it, and Data Lineage & Traceability explains what production holds. From intent to trusted data.

Data Product Delivery leads to Data Quality Assurance. Assurance leads to an independent promotion gate, and the gate leads to production. Data Lineage & Traceability reads production and ends in one of two outcomes: an explanation, or a new Intent that goes back to Data Product Delivery. From intent to trusted data.
Delivery and assurance feed the gate; lineage explains or opens new work

Start from what you run

Connect the Git repository, lakehouse and secret store you already use. The agent reads your existing models and marts first. Agent Readiness then points to the work worth doing next.

A Domain's three connections: a GitHub repository, a Microsoft Fabric Lakehouse and an Azure Key Vault secret store
Three connections you already have
  1. Your estate points to the next useful work

    Onboarding reads your estate and lists findings: the fixes that would unlock the most, and follow-ons from Intents you have completed.

    A Domain page proposing the next work: three fixes that unlock the most, and three follow-ons from completed Intents
    Sample
  2. See the evidence, then choose

    Each finding shows the evidence behind it, measured on your estate. A person chooses what to address.

    The evidence behind the findings: relation coverage, producer attribution and buildability, measured on the estate
    Sample

Data Product Delivery

What am I supposed to modify or build? Delivery answers with a candidate change, requirements and tests, built in isolation, ready for assurance. The walkthrough, on your data platform (Microsoft Fabric in this example), splits airport revenue for the board.

An Intent starts from a Linear ticket, a request as written, or an agreed spec. It then runs six stages in order: Requirements, Design, Plan, Build, Verify and Pull request. A person approves at four human gates: after Requirements, after Design, after Plan, and at the pull request, which a person reviews and merges.
From request to pull request, with a person at four gates
  1. The request arrives as it was written

    An Intent starts from a Linear ticket, a request as written, or a spec you already agreed. Here the business request is pasted unedited.

    A business request for an airport board pack, pasted unedited into a new Intent
    Example: Microsoft Fabric
  2. Every change gets its own space to be wrong

    Each Intent gets its own branch, conversation and workspace. It builds only the models it changes; Fabric reads the rest from production.

    Intent details: a Build Intent with its own GitHub branch and a newly provisioned Fabric workspace
    Example: Microsoft Fabric
  3. It explores first, then asks only what it cannot settle

    It reads your code and existing marts before it asks anything. Here it asks which airport zones to include, using the Domain's own zone convention.

    The agent asks which airport zones to include, grounded in the Domain's own zone convention
    Example: Microsoft Fabric
  4. The answers become a requirements spec in git

    The spec in git holds the objective, source, target, deliverables, acceptance criteria and what is out of scope. You approve it before design begins.

    The Intent's requirements spec in git: its objective, Fabric destination, kinds of work and rationale
    Example: Microsoft Fabric
  5. It profiles a data slice before it designs

    Your agent, on your compute, profiles a data slice: grain, joins, sizing and reconciliation, committed to git. Terms land in CONTEXT.md, which every agent reads.

    The data slice skill checking coverage and reconciliation, then committing its evidence to git
    Example: Microsoft Fabric
  6. One design record per model, in git

    The agent decides whether to extend a model or create one. Each record states the grain, every decision with its citations, and the options rejected.

    A design record for the monthly airport revenue fact: its grain and numbered decisions with citations
    Example: Microsoft Fabric
  7. Four human gates

    A person approves after requirements, after design and after the plan. The fourth gate is the pull request, which a person reviews and merges.

    A design approval question waiting for a person's answer
    Example: Microsoft Fabric
  8. The build runs on subagents

    Model-tiered by stage, or pinned for the Intent. Isolation lets several Intents run unattended in parallel on one repository, each on its own branch.

  9. Every requirement is verified with evidence

    The plan in git gains its execution evidence: queries, test results and gate outcomes, checked against each requirement in the original spec.

    The plan's execution evidence: each task's command, its result and the requirements it covers
    Example: Microsoft Fabric
  10. It ends in a pull request

    The output is native to Microsoft Fabric: Lakehouse, ingest pipeline, dbt job and semantic model. CONTEXT.md, ADRs and docs stay aligned with the code.

    The pull request summary, with the decisions made during the run
    Example: Microsoft Fabric

Data Quality Assurance

Is this what you asked for? Independent checks on the airport pull request (tests, data diff, reconciliation) give a gate the evidence to promote the change, return it for correction or hold it. A deploy lock sits behind the gate.

Three checks feed one gate. The change goes forward through the deploy lock, returns for correction, or waits.
Independent evidence decides what the gate does.

Data Lineage & Traceability

What does this number mean? Take airport revenue: Vibedata explains, or investigates further, tracing it through sources, transformations, the spec, design records, ADRs, run history and memory. The result is an accepted explanation, or a new Intent.

A result in question traced through the steps and decisions behind it, ending in an accepted explanation or new work.
A number, traced back to the decisions behind it.

What it remembers

Domain memory is shared; User memory is private to you, across projects. Each Intent session opens with both: pinned facts first, then the newest, within a budget, retired facts excluded. CONTEXT.md, ADRs, design records and AGENTS.md in git outrank memory.

At session start, Domain memory (shared Domain facts) and User memory (private facts across projects) are composed into the Intent session context: pinned facts first, then the newest, within a budget, with retired facts excluded. Committed decisions, Domain context, design records and AGENTS.md outrank retained memory: remembered facts supply context; committed truth takes precedence.
Domain and User memory compose into each Intent session's context

The same loop on each platform

What differs on each platform in production today: DuckDB, Microsoft Fabric (Lakehouse) and MotherDuck. Ephemeral workspace, per Intent. DuckDB: Each Intent gets its own copy of your DuckDB database file. Microsoft Fabric (Lakehouse): Each Intent gets its own empty Fabric workspace and Lakehouse. MotherDuck: Each Intent gets a zero-copy clone of your production database. Unbuilt models, the models the Intent has not built. DuckDB: Read from the Intent's own copy. Nothing extra to resolve. Microsoft Fabric (Lakehouse): Read in place from your production Lakehouse, live. Nothing is copied. MotherDuck: Read from the clone, as production stood when the Intent started. Native output. DuckDB: Standard dlt and dbt code, in your own GitHub repositories. Microsoft Fabric (Lakehouse): Standard dlt and dbt code plus Fabric's own pipelines, in your GitHub repositories. MotherDuck: Standard dlt and dbt code plus MotherDuck's own pipelines, in your GitHub repositories. Promotion. DuckDB: Pull request merge. CI runs static checks; it deploys to no data platform. Microsoft Fabric (Lakehouse): Pull request merge. CI builds and tests in its own workspace, then your CD deploys. MotherDuck: Pull request merge. CI builds in a per-PR zero-copy database, then your CD deploys.
What differs on each platform in production today

Vibedata runs on DuckDB, Microsoft Fabric and MotherDuck in production today. Four things differ by platform: the workspace, how unbuilt models are read, native output and promotion.

Questions

What are Vibedata's main use cases?

Three, run as one process: Data Product Delivery, Data Quality Assurance, and Data Lineage & Traceability. Together they take work from intent to trusted data.

What does Data Product Delivery answer and produce?

It answers 'What am I supposed to modify or build?' with a candidate change that has requirements and tests, built in isolation and ready for assurance.

What does Data Quality Assurance answer and produce?

It answers 'Is this what you asked for?' with independent evidence for a gate that promotes, returns the change for correction, or holds it. Checks run in isolation.

What does Data Lineage & Traceability answer?

It answers 'What does this number mean?' by tracing sources, transformations, definitions and decisions to an evidence-backed explanation, or to new intent when change is needed.

How do the three use cases connect?

Delivery feeds assurance, an independent promotion gate promotes to the production baseline, and lineage reads that baseline. Lineage either closes the question or raises new intent.

Two doors

Two ways to run it: the hosted sandbox on seeded data, or your cloud marketplace in your own subscription

Start in the hosted sandbox, or deploy from your cloud marketplace into your own subscription. Both run the loop on this page.