Bound the agent with the semantic model before it touches the lakehouse

The same column-mapping task took 38 minutes, then 21, then 5.3.¹ The engineer who ran it attributed the gain to bounding the agent's search.

In a recent internal engineering session, we timed the same file-to-lakehouse mapping three ways. One run had a warehouse connector and schema search. The next added the semantic model and a physical table map as a first pass. The last ranked candidates from the semantic model and queried only the relations it trusted.¹

One of the engineers put the difference plainly:¹

"If you give it the semantic model and do the right bounding, don't let it— so that's 21 to 5 minutes. The only difference is you tell it that, boss, I don't want you to do unbounded."¹

The last number rounds to five. The first bound saved 17 minutes. The second saved another 15.7.¹

The search was the expensive part

An unbounded agent with a warehouse connector can read schemas, list tables, sample columns, and guess. That is the behavior a capable agent defaults to. It also spends minutes and queries on relations that were never candidates.

The semantic model plus a physical table map gave the agent a short list of relations that could plausibly supply each column. The agent still had to choose among them. Ranking candidates from the semantic model removed most of that choice.

All three numbers come from one internal run of the same task.¹

A semantic model is a candidate list

A catalog tells you which tables and columns exist. A semantic model tells you which logical field maps to which physical column, in which relation, at which grain. That is the object that bounds a search.

Our semantic layer is a set of Apache Ossie documents authored beside the pipeline and the data model. Each dataset declares its fields, the physical tables and joins behind them, and a column map from each logical field to the column that supplies it. A derived rate or ratio binds to no single column; it resolves through its input metrics. Each source declares the grain it holds and the rows it covers.²

The map also records what it cannot answer. A relation with no honest mapping is listed as an exclusion with a reason. A field with no supplier is a finding the model check reports. A refusal is useful because the agent learns which candidates are out, as well as which are in.

In the mapping task, this was the difference between browsing schemas and reading a candidate list. The agent spent its time on the relations the map named, and skipped the ones it could rule out.

Rank candidates before you open a connection

The map gives the candidates. Ranking decides the order. The agent reads the semantic model first, ranks the relations by how well they match the requested field, grain and population, and queries only the high-confidence candidates.

The gain is a tool improvement. The tool returns a smaller set and checks a named candidate instead of probing the lakehouse.

The result was 5.3 minutes.¹ The two earlier runs spent their minutes on exploration. This one spent its minutes on the answer.

Stale metadata is a reason to check the map

The strongest objection is real. A semantic model can be stale or incomplete. A frontier agent with good schema search can sometimes find the right columns without one. If the map is a second thing to maintain and it drifts, the agent inherits the drift.

The answer is to check the map. Handing the agent the whole lake keeps the original cost. Our semantic layer makes the check part of the change: the map is authored beside the pipeline, every declared field needs a supplier or a recorded finding, and a relation with no honest mapping is excluded with a reason.

The post-build comparison runs when the target is authorized. The check reports what it could not run, and it separates a structural or supplier check from data correctness. The generated level tests warn; they do not prove that the declared keys hold.

Where the map is silent, the agent should stop and ask. That is slower on the one relation the map does not cover, and faster on the ones it does.

The objection also gets the model argument half right. Better tools and better models do help. Bounding is a tool improvement: it gives the model a smaller, relevant context and a set of candidates it can check. Anthropic makes the same point in its guidance on context engineering: the goal is the smallest set of high-signal tokens, and runtime exploration is slower than retrieving pre-computed data (Anthropic).

The two approaches work together. A better model makes each search step faster. The map removes steps the model would otherwise take.

The first pass is the one you can copy

Our semantic layer is one way to run the first bound. The minimum is a map with three properties: it names the physical relation for each logical field, it states the grain of each candidate, and it says when a relation is not a candidate.

Start with the task your team repeats: a file-to-lakehouse mapping, a metric definition, a column that two systems name differently. Write down the fields and the candidate relations before the agent starts.

1. Pick one repeated mapping task and record the answer a person would give.
2. Write the logical fields and the physical relations that supply them, with the grain and any row filter.
3. Mark the relations that cannot answer the question, and why.
4. Let the agent rank the candidates from the map, then query only the ones it ranks high.
5. Compare the time and the query count with the unbounded run.

The measure that moves is time per task and the number of queries it takes to get there. The timings above come from one task and do not measure a company-wide result. The productivity claim for Vibedata is separate: 5x, the same team ships five times more with Vibedata, measured as less time to PR and less time per PR once the team changes what it focuses on and how it builds, with review compounding as each accepted change makes the next review cheaper.³

The bound is a smaller question, asked before the agent opens a connection. Which mapping task in your warehouse is still a forty-minute search for your data engineering agent?

References

• Anthropic, Effective context engineering for AI agents, September 2025.
• Accelerate Data, Canonical semantic model, vibedata-official.

Footnotes

1. Measured in a recent internal engineering session on the same file-to-lakehouse column-mapping task. The engineer who ran it attributed the gain to bounding the search. Wall-clock times: 38 minutes with unbounded schema search, 21 minutes with the semantic model and physical table map as a first pass, and 5.3 minutes when the agent ranked candidates from the semantic model and queried only high-confidence relations. One task, three runs; this is not a benchmark. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹

2. The semantic-model authoring reference in accelerate-data/vibedata-official: every gold table gets an honest physical handle or an evidence-backed refusal note, a valid column-map string need not name a real output column, a derived rate or ratio binds to no column and resolves through its input metrics, and the map records unknown business meaning as a finding, with no invented mapping. ↩

3. Vibedata's productivity target, recorded at POC against the team's own baseline in accepted changes to a data product. The timings above are one internal task and do not measure it. ↩