You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
sqlmesh run keeps reporting "environment is being updated" after the updating plan has failed #6157
I found this issue when I accidentally did sqlmesh plan --auto-apply to update PROD environment while sqlmesh run is running, the run detects the environment change and waits for the plan to finish.
However, if the plan fails due to some reason before the environment finalization, the run continues firing warnings like below:
[WARNING] Environment 'prod' is being updated by plan '<plan_id>'. Retrying in 30 seconds...
The plan process has already exited with an error, but the run command warning message implies that plan process is still updating the environment. The run does not resume quickly or prompts that manual recovery is required.
I understand that running against a partially applied environment is unsafe. My concern is to properly handle the failure and help the developers debug the issue, not bypassing environment finalization.
Environment
SQLMesh: 0.236.3
Execution: CLI
Engine: Trino
State: PostgreSQL
Target SQLMesh environment: prod
Observed sequence
Start sqlmesh run.
While it is executing, run sqlmesh plan --auto-apply against the same environment.
Fail or abort the plan.
The run continues emitting the warning above even though the plan process has exited.
Restarting sqlmesh run does not resolve this issue (or the underlying unfinalized environment state).
Expected behavior
A failed plan should leave the environment protected, but the waiting run should provide a clear actionable outcome.
For example:
If plan failure is recorded, run should report that the updating plan failed and exit with a nonzero status if possible.
Otherwise, make the warning clear that SQLMesh is waiting for environment finalization, not necessarily an active process.
Include the timeout and recovery guidance such as inspecting the failed plan and successfully reapplying it.
Actual behavior
The run repeatedly reports that the environment "is being updated" without distinguishing an active plan from state left behind by a failed plan.
The following standalone DuckDB example intentionally fails during virtual-layer promotion, after the environment record is updated but before finalization. So that we can avoid needing a long-running concurrent query: a subsequent run command should encounter the same persisted unfinalized state.
Use a fresh, disposable directory with SQLMesh 0.236.3 and DuckDB support installed.
MODEL (
name repro.example,
kind FULL
);
SELECT1::INTAS id;
Apply the initial successful plan:
sqlmesh plan prod --auto-apply
Then replace models/example.sql with:
MODEL (
name repro.example,
kind FULL
);
SELECT2::INTAS id;
ON_VIRTUAL_UPDATE_BEGIN;
SELECT CAST('intentional_promotion_failure'ASINTEGER);
ON_VIRTUAL_UPDATE_END;
Apply the changed plan:
sqlmesh plan prod --auto-apply
The virtual-update statement intentionally raises a conversion error.
After the plan exits unsuccessfully, execute:
sqlmesh run prod --skip-janitor
Observe repeated warnings referencing the failed plan:
Environment 'prod' is being updated by plan '<failed_plan_id>'. Retrying in 2 seconds...
With this config, the run should eventually fail after approximately 20 seconds. Without the override, the default maximum wait is 6 hours.
Activity
lafirm commentedon Oct 21, 2026
MRE
The following standalone DuckDB example intentionally fails during virtual-layer promotion, after the environment record is updated but before finalization. So that we can avoid needing a long-running concurrent query: a subsequent
runcommand should encounter the same persisted unfinalized state.Use a fresh, disposable directory with SQLMesh 0.236.3 and DuckDB support installed.
Create
config.yaml:Create
models/example.sql:Apply the initial successful plan:
Then replace
models/example.sqlwith:Apply the changed plan:
The virtual-update statement intentionally raises a conversion error.
After the plan exits unsuccessfully, execute:
Observe repeated warnings referencing the failed plan:
With this config, the run should eventually fail after approximately 20 seconds. Without the override, the default maximum wait is 6 hours.