Agentic data engineering is delegation, not automation

On 22 October, we ran a demo and put the argument plainly: "don't think about this as automation. We really want you to think about this as delegation."

In the demo, we showed a data team re-engineering an Oracle EBS and ODI estate onto Microsoft Fabric. The agent took a scoped request and stopped at the pull request. Three review gates sat between the request and the PR. The agent could not write to production.

That sentence describes a different relationship from the one the word automation implies. Automation is a useful word for a process you already know. Delegation is the right word for work where the path is not known when you start. The work that consumes a data team's week is usually the second kind, and the word you use for your data engineering agent decides what you build around it.

Automation describes a known path. Data work rarely has one.

A nightly load that reconciles two tables is automation. The path is known, the steps are fixed, and the work is to run them reliably. So is a report that refreshes on a schedule, or a check that fails when a row count crosses a threshold.

Data engineering includes plenty of that work. The expensive work sits elsewhere.

Why is the pipeline late? Which table is the source of truth for an entity, and which one only looks like it? How do you re-engineer an ODI job onto Fabric without changing the numbers the business already trusts?

You cannot write the steps for those questions before you start, because the steps depend on what the agent finds. That is the distinction Anthropic draws in Building effective agents: workflows are "orchestrated through predefined code paths", while agents "dynamically direct their own processes and tool usage". Agents fit "open-ended problems where it's difficult or impossible to predict the required number of steps".

The word automation picks the first shape. If you call the whole category automation, you will build a workflow with a console of approvals, and you will spend the budget on the part of the work that was already predictable. The open-ended work stays with your team, now with another dashboard.

Automation earns its place in the nightly load. A team that automates that work has more room for the work that needs judgment. The mistake is to let the known part define the frame for the whole category. A tool that runs the known part and calls the unknown part "human work" has drawn a line around the expensive problem.

Our demo put three gates between the request and the pull request.

The agent produced a requirement, then a design, then a plan. Each one stopped for review.

Each gate put an artifact in front of the reviewer. The reviewer could reject a requirement, a design or a plan, and the agent had to revise before the next gate.

The earlier the gate, the cheaper the correction. The same wrong join is a paragraph at the requirement gate and an incident in production.

Only after those three gates did the agent open a pull request. It could not merge, and it could not write to production. The buyer never orchestrated each step. The buyer reviewed outcomes at the points where a wrong turn is cheap to fix.

The shape is an engagement: agree the scope, hand over the work, define the tests, review selectively. The agent directs its own process inside that contract. The contract decides where its authority ends.

The difference shows up in what you measure and what you let the agent touch.

The measures are the part teams skip. Automation counts steps completed, approvals granted, and time saved per task. Delegation counts accepted changes. The two measures are time to PR and time per PR, where time per PR is the total human effort per accepted change, spec, review and rework included.

Read the pair together. Time to PR alone rewards small, safe changes. Time per PR alone rewards shipping anything. The pair says whether the team gets more accepted changes for the same human effort.

That is the frame behind Vibedata's productivity claim: the same team ships more with Vibedata. The gain depends on changing what the team focuses on and how it builds, and review compounds as each accepted change shrinks the next review. A team that keeps the automation frame can still get faster typing. It cannot get the compounding, because the reviewer stays the bottleneck at every step.

Authority is the other half. Automation often grants production credentials because each step was approved in advance. Delegation grants repository and pull-request authority, then stops. The boundary is explicit, so the DRE who owns the incident queue can see which changes passed a gate and which never reached production.

The DRE feels the difference first. In an automation model, the DRE approves the run and then owns the incident when the run goes wrong. In a delegation model, the DRE owns the gate that catches the bad change before the PR, so fewer bad changes reach production through the agent.

The loss-of-control objection names the right failure.

The objection deserves a fair hearing. Automation sounds safer because a human stays in every step, and the agent's authority is limited to the task in front of it. Delegation sounds like handing a production lakehouse to a model and hoping.

The objection is right about abdication. Hand over an outcome with no scope, no tests, no gates and no write boundary, and you have abandoned the work. That is abandonment, and it should not ship. The failure mode is real, and it is the reason the contract matters.

The objection is wrong about where control lives. A human on every step is a control only if the human reads every step. At volume, approval becomes a click. The control is nominal, and the risk is hidden behind a green check.

Ask a team with a step-approval console which of yesterday's approvals they remember. The answer is a short list, and the console will show a long one. Delegation moves the human to three gates and one boundary, where attention can be real.

The write boundary is the strongest control in the comparison. The agent stops at the pull request; production remains a human merge.

An automation that runs the job already has production write. In that model the human approves the step definition before the run. The outcome arrives later, often as an incident. The approval feels tighter and covers less.

Safer and more controlled are different properties. Automation is safer for a known, low-variance step. For open-ended work, a gate on the outcome is a stronger control than a click on every step.

If a vendor answers "what can the agent never touch?" with "nothing, a human approves each step", that answer describes a queue. A boundary names what the agent cannot do.

That question is also the one a buyer should ask in a proof of concept. A vendor can show a fast demo of an agent doing steps. The harder demo is the agent stopping at a gate, showing the artifact it produced, and waiting. Ask for that demo, because the pause is where the buyer's authority lives.

Delegation is how a data team already works with people.

A senior contractor gets a scope and a definition of done. You agree the acceptance tests, review the design and the plan, then review the pull request. You do not stand behind them while they write every line.

If you did stand behind them, you would be buying hands. The contract is what lets you buy judgment. It is also what lets you defend the change to risk: the scope was agreed, the tests were defined, the gates were passed, and production was never in the agent's hands.

The comparison has limits. An agent will not ask a clarifying question at the water cooler, and on its first change it does not carry the political context of a re-engineering program. The requirement and design gates are where that missing context gets caught, and they are also where the review effort comes down.

Onboarding writes the repo's conventions and decisions into files before the first change. Compound engineering keeps what each reviewer corrects, so the context a person adds at a gate in this cycle is context the agent already has in the next one. Each cycle, the same gate takes less review. That is the agent's version of asking at the water cooler.

The contract also changes the team's job. Someone has to own the acceptance tests, keep them current, and decide what each gate is for. That work takes as much skill as writing the pipeline. It is the part of the job that compounds, because the next change starts from a better test set.

The move is to write the contract you would write for a person, then make the agent stop at the PR. Your team probably already has the pieces: a ticket with a scope, a definition of done, a design review, a test plan. The work is to put them in front of the agent at the points where it has to stop.

A delegation contract names what stays yours.

Five questions separate a delegation contract from a demo.

1. What is the unit of work: a step, or a scoped outcome?
2. Which gates exist, and what does each gate review?
3. What can the agent never touch?
4. Where is the acceptance test, and who owns it?
5. What do you measure: steps completed, or accepted changes?

The first question decides the rest. If the unit is a step, you will build a workflow and measure steps. If the unit is a scoped outcome, you will build gates and measure accepted changes. A product that answers "both" has avoided the choice, and the buyer will feel that in the demo.

The answers do not need to be elaborate. A scoped request, three gates, a test set and a write boundary are enough to start. What they rule out is the vague middle, where a team buys an agent and keeps every approval, then calls the result control.

The last change your team reviewed is the test. Was the reviewer approving a step, or an outcome? That answer tells you which operating model you are running today.

References

• Anthropic, Building effective agents, 19 December 2024.