Repository navigation
Make Prod deploys aware of runs affecting same models #5825
Description
Prod deploys should coordinate with scheduled runs at logical-model granularity.
Current behavior:
A prod deploy can build/materialize a replacement snapshot, then fail promotion with:
"Detected missing intervals for model ..., interrupting your current plan. Please re-apply your plan."
This happens when the scheduler advances the existing prod snapshot for the same logical model while the deploy is still running. The new snapshot is then missing intervals that were added after the deploy plan was computed.
Desired behavior:
For affected incremental models, deploy should either:
- acquire per-logical-model locks that block scheduled runs for those models during the final catch-up + promotion phase, or
- automatically replan/retry when this exact no-gaps race is detected.
Ideal flow:
- Build and materialize the plan normally.
- Before promotion, acquire locks for affected logical model names.
- Recompute missing intervals against current prod state while holding those locks.
- Backfill any newly-created intervals.
- Promote the new snapshots.
- Release locks.
This avoids blocking unrelated scheduled models and prevents manual re-apply loops for hourly/high-frequency models.
Activity
gjesse commentedon Sep 10, 2026
I’m working on an opt-in fix for this, with the intention of contributing it upstream.
The approach preserves the incumbent snapshot’s frontier when the plan is built. If the scheduler advances that snapshot before promotion, a configurable bound allows promotion when the candidate completed its original obligation and the only missing coverage is a small, contiguous trailing gap. Strict behavior remains the default; historical/internal gaps and conflicting plans still fail. Accepted gaps remain recorded as missing so the normal scheduler can repair them.
I’ve validated this with isolated, persisted DuckDB projects: strict mode reproduces the failure, while the opt-in mode finalizes successfully and a subsequent scheduler process repairs the gap. The tests verify stored interval coverage and actual parent/child rows. I'll test this in our prod environment and contribute a pr for discussion after local validation