Convert a full-refresh dbt model to incremental and show the runtime delta
A dbt model is correct but a full refresh is too slow or too expensive to run routinely, and gets slower as history grows.
- Transformation
- Any domain
Larger outcome
5 Recipesdbt performance and maintainability
Recipes for improving dbt performance and maintainability while preserving required behaviour.
Refactor dbt models without changing consumer output
A model has grown into one long query with logic packed inline, and nobody dares change it because dashboards and reports depend on its output.
- Transformation
- Any domain
Port a dbt macro to each approved warehouse without changing its callers
Part of a dbt project is moving to another warehouse, and a macro its models call only compiles, or only returns correct values, on the old one.
- Transformation
- Any domain
Author an incremental dbt model with its contract and prove idempotency
A new dbt model must be brought up to date incrementally on each run, and nobody can show that a rerun or a re-delivery of records already applied leaves it unchanged.
- Transformation
- Any domain
Backfill a bounded date range after a logic change without a full refresh
A business rule in an incremental model was corrected, and only a known date range of published history should be restated.
- Transformation
- Any domain