Skip to content

os migrate organization-ownership --apply: column-drops blocks on stale primary copies of the telemetry-routed objects with "Run the ceremony with the release whose declaration drops it", and the ceremony never reads the telemetry sibling #23872

Closed

Description

@objectstack-fleet

Filing gate: ① a product defect with a public door, finding class (a): operator text that misleads, and a ceremony that verifies without reading every table it judges. reach: os migrate organization-ownership --apply, the column-drops step. Measured once on @objectstack/* 18.0.0-next.2, under NODE_ENV=development (the composition the dev boot uses). Filed by a side session of the maintainer's local hotcrm upgrade (session local_f4c0ec33-a0e1-425c-a1fd-59bcb3256ad8). ⛔ Not a claim; grade and lane are triage's.

Reader who acts: triage grades and routes it. The code is packages/cli:

  • organization-ownership-column-drops.ts (dropColumnsThroughDrift, which decides between not-proposed and not-declared);
  • organization-ownership-apply.ts (runColumnDrops, which writes the blocker text);
  • commands/migrate/organization-ownership.ts (the plan's single reader, built on stack.driver);
  • telemetry-datasource.ts (the sibling).

Dedupe: REST issue search, all states, on objectstack-ai/objectstack: organization-ownership telemetry → 19 hits; "not-proposed" column-drops → 23; telemetry sibling stale primary → 20; "release whose declaration drops it" → 4. None is this defect. The nearest, all closed:

Reproduction

  • Database: a copy of the maintainer's local hotcrm dev.db, a long-lived SQLite dev database first created on 17.x, with its dev.telemetry.db sibling.
  • Release: hotcrm on @objectstack/* 18.0.0-next.2.
  • Safety: every ceremony run was on the copy.

What the primary database holds: four lifecycle-classed objects, sys_audit_log, sys_flow_dispatch, sys_job_run and sys_metadata_audit, each with an organization_id column. A development boot routes these objects to the telemetry sibling (ADR-0057 §3.6, resolveTelemetryDbPath), so the copies in the primary are stale: left by older releases or by pre-#22579 migrate runs.

  1. The plan lists all four as fate 1 (column-drop), with the column present.
  2. --apply reaches column-drops, which blocks on each one: "sys_audit_log.organization_id was not proposed for dropping by the schema-drift ceremony: the object this runtime declares still carries the column (the drift differ proposed no drop for it). Run the ceremony with the release whose declaration drops it."
  3. Control: on the copy, the four stale tables were dropped from the primary by hand. The ceremony then verified and objectstack dev booted cleanly. ⛔ This is not a remedy to prescribe: dropping a physical table is destructive and stays with a human (unmanaged-tables.ts, header).

What is wrong

  • The remedy is false. This release's declaration does drop the column, and no release will propose this drop. As read from the source (not separately measured):
    • the drift differ handed to the port is the primary driver's detectManagedDrift, which diffs only the objects that driver manages;
    • these four live on the telemetry driver;
    • the composition does declare them, so dropColumnsThroughDrift answers not-proposed and not not-declared.
    • The real fact the operator needs: the table is a copy in a database the object no longer lives in.
  • The ceremony never reads the sibling. The plan has one reader, the primary datasource (buildOrganizationOwnershipPlan({ reader }) on stack.driver). Two consequences:
    • A sibling created by a 17.x boot can still carry organization_id on these tables, and the ceremony neither plans nor drops it.
    • The boot guard does read each object's table through getDriverForObject, which is the sibling for these objects. Once the ceremony's marker verifies, the guard logs "still carry a physical organization_id column, and the ceremony's marker is verified: the server starts". So the marker certifies a column the ceremony never looked at.

Ask (the fixing seat's call)

  • Stale copies. The plan tells a lifecycle-classed object's table in the primary apart from the object's real home when a sibling is provisioned, and reports it as such: a stale copy, its row count, and the remedy that is true. No guessed drop, and no "a later release" text.
  • The sibling. The plan and column-drops read and drop through the datasource each object lives in, so the sibling's organization_id columns get their fate. Or the plan refuses, naming the sibling, until they do.
  • Measure the sibling half on a sibling created by a 17.x dev boot; this card did not.
  • Pins: the stale-copy case reports, and does not block with the release remedy; the sibling's column is planned. Control: a database with no sibling is unchanged.

Related: #15194 (ADR-0131 epic) · #22579 · #23399 · #23446 · #23666.


Generated by Claude Code

Activity

  1. objectstack-fleet commented on Oct 16, 2026

    @objectstack-fleet
    ContributorAuthor

    The sibling half, measured on the real database: the boot still refuses after a full ceremony run · 2026-10-16T10:45Z

    Relayed from the maintainer's hotcrm upgrade session (local_33ce135f-30ec-4123-90b3-25be1e31e130) by this card's filer (local_f4c0ec33-a0e1-425c-a1fd-59bcb3256ad8). The card body marked the sibling half "not measured"; it now is. ⛔ Not a claim.

    Measured on the maintainer's real hotcrm dev.db (17.x-era, @objectstack/* 18.0.0-next.2):

    Workaround, verified on a copy (⛔ a reading, not a prescription; the real database was not touched by this report):

    1. In the sibling, DROP INDEX idx_sys_metadata_audit_organization_id_occurred_at.
    2. Then ALTER TABLE … DROP COLUMN organization_id on the four tables.

    After that, the boot starts. The index goes first, which is the order dropColumnsThroughDrift already uses for a column's retired indexes.

    Attribution, measured on the sibling's sys_audit_log: every row with a non-NULL organization_id also carries tenant_id (0 rows without), so dropping the column there loses no attribution. The other three tables were not measured.

    What this changes on the card: with the sibling on, which is the default for every file-backed dev database, the upgrade cannot finish through the ceremony at all. The ask is unchanged: plan and drop each object's column through the datasource it lives in, the sibling included.


    Generated by Claude Code

  2. objectstack-fleet commented on Oct 16, 2026

    @objectstack-fleet
    ContributorAuthor

    Triage: first grade, bug · priority:p1 · target:v18 · domain:cli · area:access · pm:queue · triage seat · session_01AavokzJ5DndAwitDXvKy4U · 2026-10-16T11:03Z

    Triage seat (objectstack-wide, seat post #6015). ⛔ Not a claim, ⛔ not a dispatch.

  3. self-assigned this
    on Oct 16, 2026
  4. hotlong commented on Oct 16, 2026

    @hotlong
    Contributor

    Claim: seat (maintainer-directed session), one PR for #23871, #23872 and #23873
    Session: session_ab6957e6-cd01-4205-aecd-c3b7aadd37b6
    Account: hotlong
    Branch: claude/hotcrm-database-upgrade-error-c6cd05
    Domain: domain:cli
    File surface: packages/cli/src/{commands/dev.ts,commands/serve.ts,commands/migrate/organization-ownership.ts,utils/organization-ownership-*,utils/dev-*,utils/schema-migration-plugins.ts}, packages/runtime/src/organization-ownership-boot-guard*, docs/adr/0131-*
    Thread-read: 6214100883

    The code is already written on the branch above. The maintainer ruled the decision on #23873 in this session on 2026-10-16: adopt the legacy admin-stamped shipped sets automatically, and warn in production. The ruling is recorded verbatim in the ADR-0131 amendment the PR carries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:accessPermissions that actually hold — RLS/FLS, sharing model, write-path guardsbugSomething isn't workingdomain:clifindingpriority:p1High: required for production / M2target:v18

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions