Repository navigation
feat(metadata-protocol): publishPackageDrafts takes an opt-in backfill of new fields' literal defaults, written as the caller - #24269
Conversation
…l of new fields' literal defaults, written as the caller Claude-Session: https://claude.ai/code/session_01DtBADZFvLxnZCFFDWCy87D Co-authored-by: Claude <noreply@anthropic.com>
…, caller, hooks) and its door; add changesets Claude-Session: https://claude.ai/code/session_01DtBADZFvLxnZCFFDWCy87D Co-authored-by: Claude <noreply@anthropic.com>
…ger count for backfillApplied Claude-Session: https://claude.ai/code/session_01DtBADZFvLxnZCFFDWCy87D Co-authored-by: Claude <noreply@anthropic.com>
… them Claude-Session: https://claude.ai/code/session_01DtBADZFvLxnZCFFDWCy87D Co-authored-by: Claude <noreply@anthropic.com>
…blish-drafts-backfill
📓 Docs Drift CheckThis PR changes 4 package(s): 16 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: ⛔ 6 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails. What this run could not see
Coarse fallback — 150 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin fb80737750734e3aa846deac70b0237879532270 && git checkout fb80737750734e3aa846deac70b0237879532270
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 53ef7524368075a1f64003b9dfb1609d3d206d69 08c87b8d08d43a5ee4971b75143df93327696246 && git checkout -B drift-repro 53ef7524368075a1f64003b9dfb1609d3d206d69 && git merge --no-ff 08c87b8d08d43a5ee4971b75143df93327696246
node scripts/docs-audit/affected-docs.mjs --json 53ef7524368075a1f64003b9dfb1609d3d206d69
|
Contract reviewServed-tier: Inputs: card #24253 (body, claim 6245957326, dispatch 6245999954, os-dev-report 6247063019), PR #24269 (body, 14-file list, the one docs-drift advisory comment 6246776048, no review comments, no reviews), the net diff Check-runs on ① Derived judgmentsRequest keys, protocol face (
Response key
Refusals, each at its path, all collected into one 400 (
Paging and the write:
As-the-caller semantics:
Caller without the key: byte-for-byte unchanged. The door sends neither key; the protocol's only new work is Route ledger: the Generated artifacts, each the generator's output for this schema change and nothing else:
Also read: the ② Semver levelThe diff widens four published packages and narrows nothing:
③ Boundary flags
Implemented-by: VERDICT: PASS Generated by Claude Code |
席位核对 ·
|
| Item | Reading |
|---|---|
| Shape | Draft PR → main. The body opens Fixes #24253 and carries Clause-②: yes (widening) at a line start. |
| Scope | 14 files (+905 / −6), claim 6245957326 as re-declared in 6247084004. publishPackageDrafts (protocol.ts, +340); the publish-drafts door (packages.ts, +8); packages.publishDrafts (client, +10/−1); PublishPackageDraftsResponseSchema (spec, +38/−1); three generated artifacts; three pin files; four changesets. Not governed (check-governed-merges.mjs --pr 24269: 0 of 14; +905/−6 is under the human-merge threshold). |
| Ruling | The card's Scope and Done when, verbatim, are met. The backfill is opt-in; nothing is written without the key. Every named entry is judged before anything is promoted: a field without a defaultValue is refused at backfill.N, loud and by name, and the draft stays pending. The write runs as the caller. A boolean with defaultValue: false over 12 rows reads 12 false, updated: 12, on the real engine over a store with no column default. |
| As the caller | Verified by the seat in the diff. The door adds backfill and context only when body.backfill is present, and context is the request's own _context.executionContext: body.context is never read. readPublishBackfillRequest refuses a non-empty backfill with no object context ("never written as the system"). The write is updateManyData with that context per row, so permissions, RLS and FLS, validation and hooks run as on the caller's own edit. The door's requireManageMetadata still guards the route. |
| No key | ...(backfillApplied ? { backfillApplied } : {}): a caller without the key gets the same bytes as before (pinned by the door's absent-key list). |
| Review face | Schema (packages/spec/src/api/protocol.zod.ts). The CONTRACT_REVIEW_TIER record 6247208221 on this head is PASS. The declared backfillApplied matches the producer member for member on every exit. |
| Generated | authorable-surface/api.json (+1), references/api/protocol.mdx (+8) and the strictness-ledger count (442 → 444) are the generators' outputs, with their gates green. Declared to domain:spec on #6017. |
| Prose face | Four minor changesets, each with Clause-②: yes (widening). Each sentence matches the diff, including the SQL-driver note: there, ADD COLUMN already carries the literal DEFAULT, so a backfill reports updated: 0. No sentence is false. |
| Pins and ablation | Producer: 10 cases (the 12-row pin; no key; four refusals in one answer; an existing value kept; paging over 205 rows at limit 200; caller context at the CRUD seam and in the hook; a permission-denied caller; a clearing hook). Producer conformance +1, door conformance +4. Ablations A1–A4 each redden only their own cases. The one ablation the spec build refused is voided and not counted. |
| Gates | Dev union at 08c87b8d0: 116 derived, 116 run, 0 NOT MEASURED. Typecheck exit 0 for spec, metadata-protocol, runtime, client and objectql. check:generated 0. |
| CI | At this read: 41 check-runs, 36 success, 5 skipped, 0 failed. The skips are Console Pin Gate, the Dogfood matrix placeholder, Dogfood Verify CLI, Packed-tarball smoke and Temporal Conformance, each on the EXPECTED_SKIPS roster. |
Disposition:
open_questions[0](H5, a failure midway): A, implemented. The field stops at its first failing row with its count so far and a namederrorandcode; the other fields still run;success: false; the promotion stands.open_questions[1](runtime-token and expression defaults): A, refused by name. There is no single value to stamp. For objectui#12894: offer 补默认值 only for a literal default.open_questions[2](where the write runs): A, in the protocol. H2 was falsified by measurement, and the door is a pass-through.open_questions[3](as the caller means the caller's visibility): A.updatedcounts what was written, never that the table is complete.deviations: accepted, re-declared in 6247084004: thecontextrequest key, the generated rows, and the producer pin inobjectql.- Named, nothing owed (record ③.8–③.11): two wording or robustness notes for a later pass, and a one-page read-to-write window disclosed in the PR body. The Turso-remote
ADD COLUMNdefault asymmetry is out of scope; it is already recorded inremote-transport.ts.
Landing: ordinary queue (pr_ready + automerge_enable through the relay). On merge this card closes, and objectui#12894 can start.
Generated by Claude Code
Fixes #24253
Clause-②: yes (widening) (a request key on a published protocol method and its door, and a response key in
packages/spec/src/**; owes oneCONTRACT_REVIEW_TIERreview before the queue)Dispatched by the
domain:engineseat 1 PM (session_01JtSQQ3EWsbNPzyYMug1tXy); implemented by the os-dev sessionsession_01DtBADZFvLxnZCFFDWCy87Don the claim in comment 6245957326.What changes
publishPackageDraftstakes an opt-inbackfill: Array of { object, field }. Each entry names a field this publish adds. Once the drafts are live, that field's literaldefaultValueis written into the object's existing rows whose value is absent. The writes run as the caller, and the response reports the rows written per field underbackfillApplied.packages/metadata-protocol/src/protocol.ts(publishPackageDraftsand two private helpers beside it):readPublishBackfillRequestjudges the request's shape before any server state is read. The shape must be an array of{ object, field }pairs, each named once, with no other keys. A non-empty backfill needs the caller'scontext.planPublishBackfillruns afterlistDraftsand before anything is promoted. An entry is honoured only when all four hold: the package holds anobjectdraft of that name; the draft declares the field; the object the engine serves today does NOT have the field; and the field declares a literaldefaultValue.NOW(),current_user) or an expression envelope is resolved for each insert against that insert's context. There is no single value to stamp into rows that other people wrote at other times, so those defaults are refused too.400 VALIDATION_ERROR, with oneissues[]entry per problem at pathbackfill.N. Nothing is promoted and no row is written.applyPublishBackfillruns after every promoted item's side effects. For each field, it reads{ field: { $null: true } }200 rows at a time as the caller, then writes that page throughupdateManyDatawith the caller's context.updateManyDatais the bulk face of the by-id PATCH, so permissions, RLS/FLS, validation and record hooks run per row, as on that caller's own edit.packages/runtime/src/domains/packages.ts: thePOST /packages/:id/publish-draftsdoor passesbody.backfillthrough as sent, together with_context.executionContext(the request's own principal, never a system one). A body without the key sends neither key.packages/client/src/index.ts:packages.publishDrafts(id, { backfill }).packages/spec/src/api/protocol.zod.ts:PublishPackageDraftsResponseSchema.backfillApplieddeclares{ success, fields: [{ object, field, updated, error?, code? }] }. Without it, the declared parse would strip the key. The generated protocol reference, the authorable-surface shard and the strictness-ledger count were regenerated bycheck:generated --fix(442 → 444 responsez.objectsites).minorchangesets:@objectstack/metadata-protocol,@objectstack/runtime,@objectstack/clientand@objectstack/spec.@objectstack/objectqlis touched by tests only, and itsfiles[]shipsdist/alone.Where the write lives, and why (H2 measured)
The dispatch's H2 hypothesis was that only the door holds the caller's context, so the door would have to perform the writes. Measurement on
fb6fb24falsified the premise behind it.The protocol's own data methods already take the caller's
ExecutionContextfrom every door:callDatahands_context.executionContexttoprotocol.updateData/updateManyData/findData. plugin-security's CRUD/RLS/FLS middleware and the record hooks read it there.So the write lives in the protocol, and the door passes the request's context with the key. This keeps one implementation for every caller of the method. A caller without a context is refused, and the write is never performed as the system. The door half is pinned (it sends
_context.executionContext, andisSystemis unset).Premise measurement: the SQL driver already fills existing rows
Measured on a live
pnpm dev:crm -- --freshbackend (SQLite,SqlDriver(better-sqlite3)) at basefb6fb24, before any change:done: boolean, defaultValue: falseand was published with no backfill.false, and?done=falsematched 12.sqlite_mastershoweddone boolean default '0', and the stored values wereinteger 0× 12.The cause is that
ALTER TABLE … ADD COLUMNcarries the literal default (applyDeclaredColumnDefault), and the database fills existing rows with it.The gap is real on the drivers without a column DEFAULT:
columnDeclarationSQLemits no DEFAULT, and its own docblock records the asymmetry.Cloud's case (cloud#2656 / #1586) is the first of these. So on the SQL driver
backfillApplied.fields[].updatedis a truthful0, and the spec describes that.The card's Done-when is therefore pinned on a real ObjectQL engine over a store without column defaults. On a SQL backend it would pass vacuously.
Mechanism hypotheses, measured
runPublishSideEffectshas run for every promoted item. That step does two things:applyRegistryWriteThrough, without which the engine would serve no new field and strip the write; andensureObjectStorage→syncObjectSchema, which adds the column. A sync refused there is logged aterrorand the publish continues. The backfill's write then fails and is reported per field. It is never skipped.beforeUpdatehook each seeuserId= the caller, withisSystemnot set, once per row.registry.getObject(object).fields, injected system fields included) and present in this batch's draft. Unknown object, unknown field, not-new, no default, and a token or expression default are each refused by path, all in one answer, before promotion. The extra case is the token/expression refusal, decided by the axes below.droppedFields).updateDatawrites by id only; no conditional by-id update exists on this face.updated(the rows written so far),error(the record and why) andcode(the refusal's ADR-0112 code, e.g.PERMISSION_DENIED). Other fields still run, andbackfillApplied.successis false. The failure reaches the caller in the 200 body, so by AGENTS.md "Degradation log levels" it is not a degradation, and the server log carries awarnline. Pinned in the test suite.Verification record (head
08c87b8d0, after mergingorigin/maina422f04fe)Tests:
packages/objectql/src/publish-package-drafts-backfill.test.ts(10 cases, realObjectStackProtocolImplementationon a realObjectQLengine):defaultValue: falseover 12 rows reads 12false, and the receipt isupdated: 12.400 VALIDATION_ERRORatbackfill.0, and the draft stays pending.updated: 0withcode: PERMISSION_DENIED, and the publish is kept.objectql/src/publish-package-drafts-response-conformance.test.ts) gains one case:backfillAppliedparses unstripped.runtime/src/domains/packages-publish-drafts-response-conformance.test.ts) gains four cases:error/codeconform.VALIDATION_ERRORwithbackfill.0, with no flip and no announce.backfillAppliedalso joins the byte-stability absent-key list.08c87b8d0, every exit code 0:pnpm --filter @objectstack/metadata-protocol test: 244 files, 20641 passed.@objectstack/client: 52 files, 671 passed.@objectstack/runtime: 416 files, 6549 passed.@objectstack/objectql: 438 files, 8255 passed.@objectstack/spec: 693 files, 21002 passed.pnpm --filter … typecheckfor spec, metadata-protocol, runtime, client and objectql: each exit 0. objectql's and runtime'stypecheckreach the test files throughcheck:test-typecheck.pnpm --filter @objectstack/spec check:generated: exit 0, after--fixregenerated the two stale artifacts named above.Live, over HTTP through the door (a
pnpm dev:crm -- --freshbackend built from this branch, SQLite):flagged(no default) andtitle(not new) answered400 VALIDATION_ERROR.details.issuesnamedbackfill.0("declares no defaultValue") andbackfill.1("already exists on the published object"), and the live object's fields were unchanged.backfill: [done, score]answered200 published,backfillApplied: { success: true, fields: [{ done, updated: 0 }, { score, updated: 0 }] }. The 12 rows readfalse/7: the column DEFAULT had filled them, as measured above.Ablations, one per mechanism. The mutations went through
scripts/ablation-replace.mjs(wrap mode), and the dist legs were proven withscripts/ablation-dist-preflight.mjs(marker present, then--absentandtree cleanon restore):applyPublishBackfillgated off): the backfill suite went red in 6 cases, the card case included.contextdropped from the pass-through, runtime source): red in 1 case, "reaches the protocol … with THIS caller's execution context".backfillApplied.fields[].updatedwas re-typed to a string. The producer conformance went red in 1 case and the door conformance in 2.authorable-surfacereportsapi/PublishPackageDraftsResponse:backfillAppliedremoved), so nothing reacheddist/and that leg was void. The objectql leg ran green against the olddist/and is not counted. The preflight caught it, and the leg was redone as the type mutation above.Gates:
node scripts/pm/dispatch-gates.mjs --commands(no paths) on08c87b8d0derived 116 commands. Every line was run, and each exit code was captured before any pipe. The reconciliation is in the os-dev report on metadata-protocol:publishPackageDraftstakes an opt-in per-field backfill of existing rows with the new default, so a copilot draft that adds a field can fill existing records on publish (ruling A on cloud#2924) #24253.check:query-options-erasure. The new test cast engine optionsas any(test surface 236 → 237). It was fixed by typing them, and the gate is green.check:engine-split-ratioandcheck-plugin-teardown-shape --self-testfirst refused on the shallow clone. Aftergit fetch --shallow-since=2026-07-13, both exit 0.Decisions settled by the four axes (recorded for review)
NOW()or the publisher's id into everyone's rows.Acceptance notes
remote-transport.tscolumnDeclarationSQL's docblock and inturso-remote-not-null.test.ts. Carrier: none — not filed (a recorded, known asymmetry, outside this card).updatedsays what was written, never that the table is complete.discardPackageDraftsand every other verb are untouched.Generated by Claude Code