Agent platforms cannot build data engineering agents that stick. Buy the environment they need.

On 23 October 2026, CrewAI published an analysis of about 56,500 distinct builds on its own platform. Those builds ran more than 307 million times. About 4,100 were still in use.

CrewAI is reporting on its own platform, so the figure describes what its users built. That is roughly 7%, and it is the number the article leads with.

CrewAI's category chart counts 4,646 data engineering and reporting builds, the second-largest category in its back office bucket. That is the demand signal, and it is a strong one: thousands of builds aimed at the work.

Agent platforms have made the build cheap. They have not made the agent stick. A build is the thing you assemble and run; the system around it is what keeps it running when the work stops being predictable. For your data engineering agent, that difference is the whole argument between build and buy.

A build is not a system

An agent platform gives you a model, a prompt, a set of tools and a place to connect them. That is enough to produce a demo, and it is enough to produce a working agent for a task whose input and output never change. It is not enough to produce a data engineering agent that survives a Tuesday.

The system around your agent is the world it works in. It is the place the agent works, the rules it runs into, the knowledge it starts with and the platforms it can reach. None of that comes from the model, and none of it comes from a better prompt.

The model is a commodity dependency. The system around the agent is the product. A platform makes the first part easy, and leaves the second part to you.

CrewAI's own numbers show which agents survive

The article separates the builds that survive from the ones that do not by the shape of the work. Category prestige does not predict it. The agents that survive, CrewAI says, "know exactly what they'll get and exactly what to hand back." The ones that struggle "never know what's coming next."

"Every run is a new problem," the article says. A document agent gets an invoice and returns the vendor, the amount and the due date, because the input has the same shape every time. A data engineering agent gets a request that assumes the answer, and the answer depends on your tables, your grain, your business rules and your history.

Source: CrewAI 1, The Context Window, 23 October 2026.

Read the table as a description of work. It is not a league table.

Document processing sticks at 11% 1 because an invoice is an invoice. Support operations sticks at 3% in the same analysis because a support ticket can be a refund request, an angry customer or a question nobody planned for. Software engineering, the largest category on the platform, sticks at 5.7%.

Data engineering is open-ended work with a predictable-looking job title

Add a column to the revenue model. The request sounds specific, and the hard part is everything it leaves out. Revenue means one thing in the billing system, another in the CRM and a third in the board deck. The join that looks obvious drops rows on a good day, and the table everyone calls the source of truth is a view maintained by a dashboard nobody owns.

An agent can write the SQL for that request in seconds. What it cannot do is know which definition wins, which table is authoritative, which pipeline is fragile or which rule the team settled on after the third argument. That knowledge sits outside the schema, and most of it is not written down anywhere the agent can read.

That is the open-ended work CrewAI describes. The platform can make the build cheaper every quarter. It cannot make your estate less particular. A better model will write better SQL against the same missing context, and the review will catch the same wrong assumption.

The 4,646 builds show that people want this work done. Whether the builds work is a separate question. The same number carries both messages, and only one of them is comfortable for the platform.

The system around your agent is the part you would have to build yourself

The system around your agent has to supply four things a general coding agent lacks for data work: isolation, guardrails, context and cross-platform reach. They are a set: a system that supplies three of them is a demo with a missing wall.

Isolation runs at three levels: the agent's runtime, the code it changes, and the data platform itself. The data level is a throwaway copy of the lakehouse on platforms that support one, and an empty workspace reading production live on Microsoft Fabric. The mechanism differs by platform; the guarantee is the same: a wrong change cannot corrupt shared code, shared compute or shared data. That is what makes it safe to let the agent try.

Guardrails are deterministic checks that a merge cannot skip, plus a deploy lock the agent cannot talk its way past. Every change is verified by execution against real data.

Passing tests prove the code runs. They do not prove the data is right. The agent never merges its own fix. A person signs the release.

Context is the knowledge of both the code and the data: the source map, the business rules, the source quirks, the decisions that explain why the pipeline looks the way it does. Cross-platform reach means the system runs above the platform, so the context does not have to be rebuilt for every estate. Vibedata runs on DuckDB, Microsoft Fabric and MotherDuck today.

None of that is a prompt. Each part is engineering you would have to do once per platform, then keep doing as the platform changes. That is the bill the platform does not show you when it makes the build cheap.

Your data engineering agent sits at the centre, with the four things that have to be true around it. Take any one away and the agent still runs. It just stops being safe to trust.

The argument rests on the shape of the work, not the headline rate

The obvious objection deserves a fair hearing. CrewAI's 7% includes weekend experiments and demos that were never meant to last, so it is not a production failure rate. A better model or a better platform will lift it. Building the agent yourself gives you control and keeps you out of a vendor's roadmap.

All of that can be true. The argument rests on CrewAI's own split between predictable work and open-ended work, which is a property of the work. The current model does not change it.

A better model raises the ceiling on each run. It does not supply your context, your isolation or your verification. A better platform makes the build cheaper, which puts more builds into the world that then need a system around them. The rate can rise and the structural problem can stay exactly where it is.

CrewAI's article points readers to a Build, Buy, or Skip decision tree. It does not make the buy case for data engineering. That case is ours, and it rests on what the system around your agent has to contain.

Source: CrewAI 1, The Context Window, 23 October 2026.

Buying the system does not mean buying the model, the cloud or the data path

The buy side is narrower than it sounds. You are buying the part that knows your estate, runs the checks and carries the context from one change to the next. The model stays a routing decision under your keys. The cloud and the data path stay yours.

The output is dbt, dlt and SQL, so the code your team reviews is the code it already reads. If you turn the system off, the pipelines it wrote keep running as those files. That is the walk-away test for the code, and it is the answer to the lock-in objection.

The productivity claim is that the same team ships more with Vibedata. The two measures are time to PR and time per PR, counting the total human effort per accepted change, including specification, review and rework. The gain is conditional on changing what the team focuses on and how it builds, and review compounds as each accepted change shrinks the next review.

Building the system yourself does not avoid that work. It moves the work onto your team, one platform at a time, and it starts the maintenance clock on the day the first agent ships. The cheap build is the down payment. The system is the bill.

Ask what your agent knows before you ask what it can build

Four questions will tell you whether you are looking at a build or a system. Ask them before the next platform demo, and ask them of your own team if it has already started.

1. What does the agent know about the data that is not in the schema: which table is the source of truth, what the grain is, which business rule wins when two definitions disagree?
2. Where can it be wrong without touching shared code, shared compute or shared data?
3. What stops a wrong change from reaching production when the agent is confident and the tests pass?
4. Which platforms must it work on, and who rebuilds the context for the next one?

An agent platform can help with none of those questions by itself. A system answers all four, and the answers are the product. The model is the part you can rent. The system is the part that makes the rental safe to use.

The next agent your team builds will start from a request that contains less than it needs. The question is whether the system arrives with it. Look at the last agent you shipped. Who maintains it now, and what does it still know about your data?

References

• CrewAI, The Context Window, 23 October 2026.