QA run · automation (18/22) · 5b14b7f3 · 2026-10-19 · 15 PASS / 2 PARTIAL / 1 FAIL / 0 BLOCKED / 4 NOT-RUN #24545

Run record for the 18.0 release QA wave (dispatch session session_01CfRDy7SuTfUqaTu2AEH2Hi; maintainer-approved plan B: Tier 1 + Tier 2, Tier 3 deferred). Runner session session_016CErRJhdMYZdRg4pe8iQfD. Protocol: .claude/skills/checklist-test/SKILL.md + docs/qa/platform-checklist/RUNNER.md.

Environment fingerprint

• framework: 5b14b7f3ad8e2e0786d228772f473d07b36b2477 (pinned; pnpm install --frozen-lockfile + pnpm build, 76/76 tasks green)
• .objectui-sha: 4016370af3c9c9dafec2bc5299afd8e8079701b8; console built with pnpm objectui:build (shallow clone into .cache/objectui-4016370af3c9), packages/console/dist/.objectui-sha matches, pnpm check:console-sha green
• resolver: node scripts/checklist-select.mjs area:automation --json → 22 runnable · 0 blocked hidden · 0 planned
• app: showcase · boot OS_PORT=P objectstack dev --ui --seed-admin -p P -d file:…/ITEM.db · per-runner port range and per-item file DB · seeded admin + demo personas; staged configs only in scratch copies of the app
• browser: Playwright on /opt/pw-browsers/chromium, headless, one context per persona
• timestamp: 2026-10-19 (UTC) · execution: parallel fan-out, one sub-agent runner per item group:
  • U1: ports [41380,41381,41382,41383] · env 4 min / test 15 min
  • U2: ports [41390,41391] · env 3 min / test 10 min
  • U3: ports [41400,41401] · env 10 min / test 22 min
  • U4: ports [41410] · env 2 min / test 13 min

Scope

Selector area:automation · 22 runnable item(s) (revision = the one each verdict is valid for):

• automation.flow-run-step-nesting r5 · P1 · mixed · Tier 2 → pass (runner U2)
• automation.time-relative-trigger r3 · P2 · mixed · Tier 3 → not-run
• automation.rollup-summary-filter r4 · P1 · mixed · Tier 2 → pass (runner U2)
• automation.flow-node-type-matrix r6 · P1 · mixed · Tier 1 → pass (runner U4)
• automation.trigger-type-matrix r4 · P1 · mixed · Tier 1 → pass (runner U4)
• automation.flow-error-handling r4 · P1 · mixed · Tier 2 → pass (runner U2)
• automation.screen-flow-roundtrip r4 · P1 · browser · Tier 1 → pass (runner U4)
• automation.durable-suspend-restart r1 · P1 · api · Tier 1 → pass (runner U3)
• automation.paused-run-trigger-record-masked r1 · P2 · api · Tier 1 → pass (runner U3)
• automation.connector-dispatch-matrix r3 · P2 · mixed · Tier 3 → not-run
• automation.flow-runs-page-test-trigger r2 · P2 · mixed · Tier 3 → not-run
• automation.flow-toggle-kill-switch r3 · P2 · api · Tier 3 → not-run
• automation.trigger-status-contract r2 · P1 · api · Tier 2 → partial (runner U1)
• automation.packaged-flow-disable-durable r3 · P0 · api · Tier 2 → pass (runner U1)
• automation.packaged-flow-subflow-disable-refusal r3 · P1 · api · Tier 2 → pass (runner U1)
• automation.packaged-flow-clone-contract r3 · P1 · api · Tier 2 → pass (runner U1)
• automation.setup-packaged-automation-board r3 · P1 · browser · Tier 1 → partial (runner U4)
• automation.flow-registration-verdict-on-read r1 · P1 · api · Tier 1 → pass (runner U3)
• automation.flow-authoring-refusals-v18 r1 · P1 · cli · Tier 1 → fail (runner U3)
• automation.flow-run-semantics-v18 r1 · P1 · api · Tier 1 → pass (runner U3)
• automation.inbound-hook-timestamped-signature r1 · P1 · api · Tier 1 → pass (runner U3)
• automation.scheduled-work-opt-in r1 · P1 · api · Tier 1 → pass (runner U3)

Per-clause verdicts

Reproduction rules (fails)

• automation.flow-authoring-refusals-v18 acceptance[2] — disposition: stale-spec · stale-spec: the runtime PUT gate deliberately stands down on the CEL root-scope judgment (whole-stack, owned by os validate — which still refuses both arms, so negative clauses hold). Re-point the clause to expect 200 for the record.x and user.id arms at PUT. Per this wave plan the re-point is parked on the wave anchor via the Checklist-accuracy section; no product card.
  • repro: boot scratch showcase with OS_METADATA_WRITABLE=flow; as admin PUT /api/v1/meta/flow/showcase_qa_arm_user_id_cel {autolaunched, record-after-update start on showcase_project, edge e1 condition 'user.id == "u1"', notify node} → 200, GET → 200. Same for an autolaunched flow with no record entrance whose edge condition reads 'record.x == 1'. Reproduced twice on fresh boots. The same bodies fail os validate with expression-invalid.
  • expected vs actual: expected 422 INVALID_METADATA + read-back 404; actual 200 + stored (the user.id flow faults at run time with 'Unknown variable: user')
  • mechanism: By design: packages/lint/src/flow-cel-root-scope.ts:138-157 + :489 perWriteSnapshotEntrance() — the CEL root-scope judgment (automation: a flow CEL expression may name the run user as user, ctx.user or os.user; objectstack validate passes it and the run faults Unknown variable, because flow CEL binds only current_user #22565/lint(flow CEL roots): record is still in ENGINE_BOUND_ROOTS, so objectstack validate passes a record.X read in a flow with no record entrance, which faults at run time once #22642 lands; and two texts describe record as always bound #22677) stands down at the runtime publish gate because the per-write snapshot carries no actions or other flows (rulings 5791822697 item 3 and 6104584601 on lint(runtime gate): a flow write's per-write snapshot carries no actions and no other flows, so a flow CEL root rule cannot see the entrances that hand a flow its record (W, from #22565) #22636, letter B; it is a whole-stack judgment that belongs to objectstack validate). The item's PUT clause should leave out the record.x and user.id arms, or expect 200 for them.

Derived item verdicts

15 pass · 2 partial · 1 fail · 0 blocked · 4 not-run (of 22).

Tier 1 pin runs (RUNNER rule 6, one vitest batch per package, all at the subject sha):

• packages/spec: 8 files, 582 passed (flow-value-slot-template, flow-text-slot-template, flow-builtin-node-config-keys, flow-builtin-node-config-values, flow-bound-variable-name, flow, retry-policy, anonymous-elevated-flow-gate)
• packages/runtime: 3 files, 71 passed (action-flow-elevated-door, endpoint-flow-elevated-door, domains/automation-hooks)
• packages/services/service-automation: 5 files, 70 passed (flow-cel-current-user, flow-cel-record-binding, decision-overlapping-edge-conditions.pin, flow-refused-registration-read.integration, per-kernel-scheduled-work-policy)
• packages/lint: 1 file, 80 passed (validate-expressions.flow-cel-root)
• packages/core: 1 file, 16 passed (security/http-signature)
• packages/platform-objects: 1 file, 4 passed (apps/setup-packaged-automation-nav)
• packages/triggers/trigger-api: 1 file, 12 passed (inbound-hook-member); packages/triggers/trigger-schedule: 1 file, 10 passed (per-kernel-scheduled-work-policy)
• packages/rest: 1 file, 3 passed (meta-flow-cached-registration)
• packages/qa/dogfood: 4 files, 22 passed (flow-durable-suspend, flow-node, flow-trigger-conformance, flow-trigger-record-credential-mask)
• objectui-only refs (setup-packaged-automation-board component tests, screen-flow-roundtrip e2e/live/screen-flow.spec.ts) were NOT run from this checkout — both items were driven BY HAND in the browser against the live boot with the 4016370af3c9 console (RUNNER option (a))

Partial / gap notes:

• automation.trigger-status-contract: acceptance[4] 409 RESUME_IN_PROGRESS arm not-run (3-way concurrent resume answered 200 + 404 + 404; no deterministic stock trigger) and 503 STORE_UNAVAILABLE arm blocked(fixture); every other arm passed
• automation.setup-packaged-automation-board: acceptance[5] scored blocked(dependency) by the parent — the admin cell passed (409 text verbatim in role=alert), but the member cell's dom oracle is unreachable because a plain member cannot open Setup (knownGaps[2]); server-side the member toggle answers 403 with the exact manage_metadata sentence

Handover — items with no verdict this run

• automation.time-relative-trigger r3 · P2 · mixed — Tier 3 deferred by the 18.0 release-sweep plan
• automation.connector-dispatch-matrix r3 · P2 · mixed — Tier 3 deferred by the 18.0 release-sweep plan
• automation.flow-runs-page-test-trigger r2 · P2 · mixed — Tier 3 deferred by the 18.0 release-sweep plan
• automation.flow-toggle-kill-switch r3 · P2 · api — Tier 3 deferred by the 18.0 release-sweep plan

Checklist-accuracy findings / fixture gaps

Recorded for the wave anchor (collected by the dispatch session); not extracted, no checklist edit in this run.

Checklist accuracy:

• automation.packaged-flow-clone-contract · acceptance[1] · FLOW_CLONE_DROPPED_KEYS is 10 today, not 9: flow-clone.ts:149 appends 'protection' to the 2 read decorations + 7 protection fields; the clause's enumerated list omits 'protection'
• automation.packaged-flow-subflow-disable-refusal · acceptance[0] · remedy text is caller-specific ('Disable the calling flow '‹name›' first — or leave this one armed.'), not the quoted generic 'Disable the calling flow(s) first, or leave this one armed' — substring match on the quote fails although meaning is identical
• automation.trigger-status-contract · steps[2] · 'a flow that succeeds' is unnamed; stock showcase_resilient_sync triggered bare answers 400 FLOW_FAILED (its CEL condition reads record status), so name showcase_notify_owner with {params:{ownerId,message}} (200) as the success specimen
• automation.trigger-status-contract · steps[5] · INVALID_SIGNAL arm: the route maps inputs ?? variables, so a body carrying valid inputs plus variables:{$x} silently drops variables and resumes; a $ key inside inputs answers INVALID_SCREEN_INPUT first. The reachable spelling is output:{'$x':1} (→ 'collect.$x'); also the pause must be triggered with recordId+objectName or the valid resume fails 400 FLOW_FAILED (update_record needs recordId)
• automation.trigger-status-contract · steps[5] · the 403 persona must be admin-created and must change password first (otherwise 403 PASSWORD_RESET_REQUIRED, which looks like the arm but is not)
• automation.flow-error-handling · step 5 / fixtures.requires · says to author the probe flow 'in the scratch package' via POST /api/v1/automation, but that door reads no package selector (automation-flow-door-package-address.test.ts) — the flow lands package-unbound ('Organization flows', /_console/studio/~org/automations). Either drop 'in the scratch package' or name PUT /api/v1/meta/flow/:name?package=
• automation.flow-run-step-nesting · acceptance[1] · names an 'end' step that 'carries no parentNodeId', but the engine logs no step for the end node (both showcase_batch_reminders and showcase_resilient_sync runs end at the last real node) — the clause holds vacuously for end
• automation.rollup-summary-filter · step 7 / acceptance[4] · should say that objects authored into the scratch base must carry the package namespace prefix (POST /packages derives namespace from the id's last segment, e.g. com.u2.rollup → 'rollup_'), otherwise the Studio edit cannot be published (publish-drafts outcome=refused NAMESPACE_PREFIX) and the persisted-metadata half cannot be reached
• automation.flow-authoring-refusals-v18 · acceptance[2] · the runtime PUT gate stands down on the CEL root-scope judgment by design (flow-cel-root-scope.ts:138-157, perWriteSnapshotEntrance; rulings 5791822697/6104584601 on lint(runtime gate): a flow write's per-write snapshot carries no actions and no other flows, so a flow CEL root rule cannot see the entrances that hand a flow its record (W, from #22565) #22636), so the record.x and user.id arms answer 200 at PUT /meta/flow. The clause should exclude those two arms (they are build-door-only).
• automation.flow-authoring-refusals-v18 · steps · 'job retryPolicy:{maxRetry:3}' cannot be PUT to /meta/flow. The flow-shaped equivalents are try_catch config.retry.maxRetry (422) or errorHandling.maxRetry (422, but no did-you-mean). The steps should name which one.
• automation.flow-authoring-refusals-v18 · steps · the anonymous-endpoint arm at PUT is judged only in the flow direction: it needs a code-declared authRequired:false endpoint, and then PUT of a runAs:'system' target flow answers 422 at path runAs. PUT /meta/api is NOT_CREATABLE by default. With OS_METADATA_WRITABLE=api, an anonymous elevated endpoint row is saved 200 without the D2b judgment, but it is never served (POST to its path → 404 ENDPOINT_NOT_FOUND), as documented at metadata-protocol/src/protocol.ts:22265-22297.
• automation.flow-authoring-refusals-v18 · acceptance[1] · schema-layer arms surface from os validate as code STACK_SCHEMA_INVALID with a contract sentence ('refused at bogusKey by the notify contract'), not as a rule id 'node-config-refused-by-contract'. Match on path + sentence, not on the rule id.
• automation.flow-registration-verdict-on-read · fixtures · storing the refused flow also needs OS_METADATA_WRITABLE=flow (the flow type is not overlay-writable on stock showcase), and it needs a NAME NEVER REGISTERED: a refused re-registration keeps the previous registration and reads 'registered'. Working recipe: an api-type flow with no config.secret, PUT under the OS_ALLOW_UNLINTED_METADATA_WRITES=1 hatch.
• automation.paused-run-trigger-record-masked · fixtures · the member's read grant is applied by POST /data/sys_user_permission_set {user_id, permission_set, organization_id} as admin. Admin create-user answered membershipCreated:false, and the member still resolved positions org_member/everyone.
• automation.trigger-type-matrix · acceptance[4] / steps[5] · says 'capture the trigger POST'; since objectui#12037 the declared flow action launches through POST /api/v1/actions/showcase_task/showcase_bulk_reassign (screen-flow-roundtrip rev4 already re-pointed this) — re-word to 'the action-door launch POST'
• automation.setup-packaged-automation-board · acceptance[1] / steps[3] · 'default all On' / 'confirm every switch reads On' — showcase_approver_bindings is authored status:'obsolete', so its switch correctly reads Off (31 On / 1 Off); the clause should exempt obsolete-status flows
• automation.flow-node-type-matrix · acceptance[1] · 'end logs no step' — an end node on an UNTAKEN branch is logged as a step with status 'skipped' (showcase_inquiry_purge run_f8a7bc80: node end nodeType end status skipped; also exec_review approval skipped in budget_approval); a taken end logs nothing. Harmless, but the wording 'end logs no step' is not literally true
• automation.flow-node-type-matrix · steps[1] · 'open the Studio flow-designer palette' — on stock showcase the package is read-only, so the canvas has no Add node; the palette is reachable only on a writable scope (here: an org-scoped clone made via Setup › Packaged Automation, Studio ~org scope). Name that route in the step
• automation.setup-packaged-automation-board · steps[1] · the sidebar entry lands on /_console/apps/com.objectstack.setup/component/automation/packaged (package-id path segment), not /apps/setup/component/automation/packaged as the fallback URL is spelled

Fixture gaps:

• automation.trigger-status-contract · 409 RESUME_IN_PROGRESS has no deterministic stock trigger (3-way concurrent resume race lost) — record as knownGap or provision a slow resume path
• automation.trigger-status-contract · qa-flow-status-doors recipe lacks a success-row specimen; recipe itself now proven live (all steps 2xx)
• automation.paused-run-trigger-record-masked · stock showcase has no active pausing record-change flow on a credential object and no run-reader permission set; authored in the scratch copy (known gap, still open)
• automation.flow-run-semantics-v18 · stock showcase has no current_user-stamping record flow, no autolaunched runAs:'system' flow and no overlapping-edge decision; authored in the scratch copy
• automation.flow-registration-verdict-on-read · stock showcase ships no refused flow; one was planted at runtime through the lint hatch
• automation.flow-node-type-matrix · showcase_fan_out_notify / showcase_resilient_sync http steps need outbound egress (fetch failed here); covered by the self-ping scratch flow per knownGaps[1]
• automation.trigger-type-matrix · stock showcase_task_due_reminder only sweeps daily 08:00 UTC; in-session proof needs a runtime cadence-override copy (method recorded in notes)

Incidental product findings (outside any clause)

Both seen while driving items in this area, independently re-verified on a fresh boot, and extracted as their own defect cards at close-out (see the close-out comment):

• while driving automation.trigger-status-contract steps[5]: POST /api/v1/automation/:name/runs/:runId/resume silently drops variables when inputs is also present (the route reads inputs ?? variables), so a body carrying both resumes 200 and the second bag is never checked.
• while driving automation.rollup-summary-filter step 7: an active PUT /api/v1/meta/object/:name?package= saves an object name WITHOUT the package namespace prefix (200, the object is live), while publish-drafts for the same object in the same package refuses it with NAMESPACE_PREFIX.