Build something that is not there yet
Load a recurring file drop into bronze with file-completeness and schema controls
- When a folder of files lands on a schedule and nobody checks whether a file is missing before the build runs.
- When the file drop's columns move around and the load has silently coerced a type more than once.
Build a dlt pipeline on the filesystem source that lands a recurring file drop into bronze, with a completeness check against the expected file set and the inferred schema frozen after the first load. Covers the landed files and their completeness proof, not the file source's own reliability.
Sample This Recipe has not been materialized in the Cookbook repository yet. Its trigger, description, prompt, agent guidance and acceptance conditions, and the explanation below, are prototype drafts. Its name, job, area and readiness come from the reconciled Cookbook seed snapshot. Readiness is a separate question from this one: it says whether the capability exists, not whether the writing has been reviewed.
Use this Recipe
Use the VibeData Recipe `file-drop-to-bronze` at https://getvibedata.ai/cookbook/file-drop-to-bronze Read the Recipe and execute it in the context of the current Intent.
Recipe id file-drop-to-bronze · Not yet materialized in the Cookbook repository, so the pointer addresses this page.
Verified by
What has to be observably true before this Recipe is finished.
- a run with a missing expected file is reported, not silently skipped
- landed row counts match the files present for the load window
- the frozen schema is committed and a moved or renamed column is visible in the diff
- a second run is incremental on file modification time
Recipe promptThe task specification the agent reads. Reference only — it is not what you copy.
Deliver: Load a recurring file drop into bronze with file-completeness and schema controls. Build a dlt pipeline on the filesystem source that lands a recurring file drop into bronze, with a completeness check against the expected file set and the inferred schema frozen after the first load. Covers the landed files and their completeness proof, not the file source's own reliability. Execute inside the current Intent. Its Domain, repository, platform, environment and attached sources are the context for this work — read them rather than asking for them. The work is done when: - a run with a missing expected file is reported, not silently skipped - landed row counts match the files present for the load window - the frozen schema is committed and a moved or renamed column is visible in the diff - a second run is incremental on file modification time Report the evidence for each condition above with the result. A condition you cannot meet is something to say, not something to work around.
Agent guidanceHow the agent approaches the work, and what it will not do.
Profile the inputs the grain, joins and measures actually depend on before proposing a model. Put the design up for review — grain first — then build in an isolated copy with tests and documentation landing beside the model rather than after it.
Composes
- source profiling and incremental load
- dlt in the project
- isolated-copy execution and gate verification
Guardrails
- Build in an isolated copy. Production is read, never written.
What you need
- Microsoft Fabric Lakehouse, MotherDuck, or DuckDB.
- Read access to the source. Not write access to production.
- dlt in the project, or the intent to add it.
How it goes
- State the outcome in one sentence, in the language the request arrived in.
- Let it profile the inputs the grain, the joins and the measures actually depend on.
- Review the design. Disagreeing about grain here costs a sentence; after the model exists it costs a rewrite.
- Let it build in an isolated copy, with the tests and the documentation landing beside the model rather than after it.
- Read the acceptance conditions against the run.
What you end up with
The deliverable, in your own repository, as ingestion work a reviewer who knows the project reads as native to it. Alongside it, the evidence for every one of the acceptance conditions above — which is the part that is still there in three weeks when somebody asks.