QA run · tier1:automated-pins (77 items) · e4e5c6e3 · 2026-08-17 · 35 PASS / 39 PARTIAL / 0 FAIL #9401

Round R3-T1 of the #9296 wave — Tier-1 automated-pin sweep. Scope: the 77 active, non-blocked checklist items that carry an automated.ref, run per RUNNER rule 6 ("don't re-prove what automation pins"). Verdicts below are pinned-clause verdicts only — the unpinned clauses of these items are Tier-2's, and nothing here was hand-driven.

35 PASS · 39 PARTIAL · 0 FAIL · 3 BLOCKED. No clause failed as a product defect, and no access-control hole was found. The headline is the PARTIAL column: the pins are real and green, but on 39 of 77 items the pin reaches only a subset of the item's acceptance clauses. Every gap is named in the "Coverage gaps" section — that list is the round's most reusable output.

Environment fingerprint

Pin inventory was checked before running: all 35 framework refs and all 25 objectui refs resolve to real files. Two refs name companion tests by bare name only — access-security.scope-depth-asymmetry (showcase-scope-depth-write, -fallback) and records-forms.import-wizard-encoding-and-hints (importMissingRequiredHint, importLegacyReferenceGuard, which actually live in plugin-grid/src/__tests__/*.tsx). Both resolve; noting them only because a bare name costs the next runner a search.

Batch commands and real output

1 — dogfood (33 pinned files, one invocation) · packages/qa/dogfood · npx vitest run test/<33 files>

Test Files 33 passed (33) · Tests 283 passed | 1 skipped (284) · Duration 169.90s

The single skip is attachments-permission-matrix > attachments cross-tenant isolation (g), gated by describe.skipIf(!organizationsAvailable). That probe resolves the cloud-private enterprise package @objectstack/organizations through the production resolver; it is absent in this OSS workspace, so the block skips by design and loudly (OS_TEST_MULTI_ORG_ENABLED=1 turns the skip into a hard failure for runs that ship the package). This is blocked(dependency), not a defect — but it does mean every cross-tenant attachment clause is unproven here.

2 — objectql · npx vitest run src/{engine,engine-lookup-referential-integrity,engine-autonumber-runtime-owned,secret-fields,overlay-precedence}.test.ts src/validation/rule-validator.option-visibility.test.ts

262 passed, 0 failed — engine 135/135 · autonumber 32/32 · secret-fields 36/36 · overlay-precedence 26/26 · lookup-RI 13/13 · option-visibility 20/20

3 — mcp / cli / services / connectors (5 invocations, sequential)

mcp 37 passed (plugin 17, skill-prompts 12, validate-expression 8) · cli 11 passed (verify-tenancy-posture 7, format.exit-code 4) · service-automation 31 passed (resume-authority-gate 22, connector-descriptor-audit 9) · connector-mcp 13 passed · service-datasource 54 passed (admin-routes 30, envelope.conformance 24)

4 — scope-depth companions · npx vitest run test/showcase-scope-depth-{write,fallback}.dogfood.test.ts

13 passed (write 9/9, fallback 4/4)

5 — objectui unit pins (25 files, one invocation, run in the pinned-sha cache)

249 passed, 0 failed, 25 files — celAuthoring 34 · CelPredicateField 24 · importParsers 30 · UndoManager 28 · bulkAction* 30 · gantt 22 · discardGuard 14 · RecordPicker/Lookup 18 · occSave 8 · calendar dnd 8 · grid personalization 10 · PermissionMatrixEditor 10 · others

6 — objectstack verify --rls · examples/app-showcase · node packages/cli/bin/run.js verify --rls

✓ verify passed — no runtime failures
CRUD: 15 verified, 0 gaps, 0 FAILED, 1 needs-fixture, 7 skipped (0 mismatches)
RLS: 20 PROVEN (20 consistent, 0 HOLES) · 3 NOT PROVEN (0 member-visible, 0 probe-blocked, 3 skipped)
position personas: 9 of 9 declared position(s) probed · all personas: 35 PROVEN (35 consistent, 0 HOLES)

Confirms R1 finding #1 — --rls is a separate invocation and bare verify emits no RLS section. The run's own ⚠ 3 of 23 objects were NOT proven is a coverage caveat (2 external federated read-only objects + 1 with no plain-text probe field), not a hole. Two personas (exec, auditor) proved nothing — every object was member-visible, probe-blocked or skipped; worth a Tier-2 look, since a persona that proves nothing is indistinguishable from a persona with no policy.

7 — framework playwright smoke · examples/app-showcase · playwright test --config=/tmp/.../pw-showcase.config.ts

35 passed (2.2m) — with the pinned chromium. Confirms R1 finding #2; the scratch config lives in /tmp so the subject tree stays clean.

8 — objectui live e2e (18 pinned specs) · playwright test --config=/tmp/.../pw-live.config.ts against os serve --dev on :3000 + console vite dev on :5180

18 passed, 11 failed (7.8m) — and re-run in isolation: the same 11, deterministic, not load (#9311 ruled out).

Why the 11 live-e2e reds are NOT scored as product failures

RUNNER rule 2 (reproduce, self-check, rule out the traps) and rule 5 (severe findings are hypotheses) both point the same way here. Four independent pieces of evidence:

1. objectui's own CI runs only 4 of these specs. test:e2e:live:ci is exactly screen-flow · action-modal · master-detail · saved-view-filter, and all four passed in this run. The workflow header is explicit that the lane is continue-on-error: true, "INFORMATIONAL, NON-REQUIRED", and warns: "do not switch all 20+ live specs on at once and inherit whatever flake exists". The 14 specs outside that allowlist have never been proven green by the repo that owns them.
2. The pinned pairing is a published backend, not this source build. e2e/live/ci/backend.env pins OBJECTSTACK_VERSION=17.0.0-rc.2 / OBJECTSTACK_REF=89d2a4eb…, and its own header says testing a mismatched pair "proves nothing". I ran the pinned console against source-built e4e5c6e3.
3. Several reds are demonstrably assertion bugs, not product bugs. form-view-subforms and grid-file-upload fail with a strict-mode violation in which the locator matched two elements — one of them the real <h3>Line Items</h3>. The surface rendered; the assertion is ambiguous because a Radix sr-only dialog description now also contains the phrase. user-filters asserts Timeline is absent from the viz whitelist while the app offers it — a product change the spec never caught up with.
4. The ledger already predicted two of them. platform-core.metadata-authoring-roundtrip and studio-authoring.record-page-roundtrip both carry a stale flag saying studio-record-page.spec.ts is "STALE, NOT RED — that spec fills the Object control as an input when it is now a role=combobox button", with a runnerRule of "do NOT score their failure as a FAIL of this item". This run reproduced that failure byte-for-byte: locator.fill: Error: Element is not an <input>, <textarea> or [contenteditable] element. Independent corroboration of the ledger's own note.

Additionally, meta/app on stock showcase returns exactly showcase_app, setup, account — the Studio app is not installed, confirming #9296 finding #4. That is why studio-object-designer F1 ("a new package appears in the switcher") cannot pass here, while F2/F3/F4 of the same spec do.

So these are recorded as blocked/partial with the reason, never as fail. Tier-2 should still look at them — a deterministic red is worth a matched-pairing re-run before anyone calls it clean.

Per-item verdicts

Verdict scope is the pinned clauses. Scoring rule applied: pass only where the pin's executed tests span the item's acceptance clauses; partial where the pin demonstrably reaches a subset (gap named below); blocked where the pin could not be consulted in a valid pairing.

access-security

ai

api-backend

approvals

attachments-storage

automation

cli

dashboards

identity-auth

integration-system

platform-core

records-forms

search

studio-authoring

Coverage gaps — every partial / blocked, with what the pin did not reach

• access-security.crud-permission-matrix (rev1, partial) — the ref itself says persona-grained cells remain manual — verify proves object-level CRUD + cross-owner RLS, never the persona x CRUD-cell matrix
• access-security.fls-mask-and-strip (rev1, partial) — ref is explicit: write half only. The read-side clauses (masked value in the GET payload, field stripped from the response shape) have no pin
• ai.mcp-http-surface (rev2, partial) — pin is the identity-admission block only: C0 (opted-out boot 404 on /mcp and /mcp/skill), C1 (GET /mcp/skill serves SKILL.md unauthenticated) and C2 (405 + Allow: GET) are unpinned
• ai.mcp-stdio-fail-closed (rev3, partial) — covers C0–C2 + C7 (keyless refusal, unresolvable key, principal binding, no key demanded when disabled). C3–C6 — member-keyed RLS/FLS read parity, aggregate_records parity, FLS aggregate fail-closed, revocation honored on next call — are unpinned
• api-backend.query-contract-matrix (rev1, partial) — the ref scopes the pin to "[Security: $expand bypasses RLS/FLS on the referenced object (data leak on lookups) #2850] expand sub-read through the secured find path" = C4 only. C0–C3 and C5–C8 (operator answer sets, $top/$skip windowing, $select narrowing, sort + INVALID_SORT, INVALID_FILTER, UNSUPPORTED_QUERY_PARAM, $in/$nin empty-list boundary) are REST-contract clauses this engine unit suite does not reach. filter-comparand-shape.ts is a source module, not a test
• attachments-storage.presigned-upload-roundtrip (rev1, partial) — pin reaches C4 (anonymous presign 401) and the authenticated upload. C0 (presign envelope {uploadUrl, method, fileId, expiresIn, downloadUrl}), C1 (status pending->committed on complete), C2 (GET /files/:id/url envelope) and C3 (302 redirect) are unpinned
• attachments-storage.download-authz-both-sides (rev2, partial) — C3 (acl=public_read re-opens the stable anonymous capability URL) and C4 (browser renders friendly denial copy) are unpinned; the cross-tenant case is skipped — see the enterprise-gate note
• attachments-storage.read-inherits-parent-rls (rev2, partial) — C1 (COUNT filtered identically to rows) and C3 (filter fails CLOSED on compute failure / past the 2000-row pre-scan cap) are unpinned
• attachments-storage.attach-requires-parent-edit (rev2, partial) — C3 (unscoped multi-delete with neither id nor where is refused outright, installAttachmentAccessHooks does not authorize an UNSCOPED multi-delete: no id + no where reads as "nothing to authorize" and deleteMany runs over the whole table #4757) is unpinned
• attachments-storage.sys-file-status-pipeline (rev1, partial) — C0 (every status enum variant), C1 (a pending file is not downloadable) and C4 (only scope=attachments committed files tombstone via these hooks) are unpinned
• attachments-storage.orphan-tombstone-reap (rev1, partial) — C1 (bytes reclaimed BEFORE the row delete — the ordering, not just the outcome) is not separately asserted
• attachments-storage.upload-session-abort (rev3, partial) — C0 (the chunked round-trip over the real routes: init -> chunk PUTs -> progress) is unpinned
• attachments-storage.inline-grid-receipt-cells (rev2, blocked) — not consultable — see the live-e2e pairing note; the spec is outside objectui's own CI allowlist and its failure is an ambiguous-locator assertion, not a product signal
• automation.flow-node-type-matrix (rev3, partial) — two tests against a 6-clause node-type matrix — the pin exercises the node-type registry, not each declared node type's runtime semantics
• automation.trigger-type-matrix (rev1, partial) — two tests against 6 clauses; the ledger conformance proves declared-vs-registered parity, not each trigger type firing end-to-end
• cli.migrate-plan-apply-json (rev1, partial) — pin is the exit-code formatter only = C2. C0 (bare os migrate never mutates), C1 (--json shape), C3 (destructive skipped until --allow-destructive), C4 (idempotent re-run in_sync), C5 (busy-DB asymmetry), C6 (--json non-interactive confirmation_required) are unpinned
• cli.verify-verdict-exit-mapping (rev1, partial) — the pin decides which SUITE runs, not how a verdict maps to an exit code; the verdict->exit clauses are only indirectly covered
• cli.scaffold-first-run (rev2, partial) — the workflow cannot be executed locally (registry install + docker) and was never run against the subject sha e4e5c6e; the nightly published-package canary job has no run in the window
• dashboards.dataset-report-authoring (rev1, partial) — three tests against 8 clauses; the dataset/report authoring clauses beyond save->publish->read are unpinned
• identity-auth.auth-method-matrix (rev2, partial) — C7 says outright that only the OIDC flow is pinned. C0–C6 and C9 (advertisement parity, email+password round trip, phone+password, phone-OTP NOT_SUPPORTED, login-page per-variant presence/absence, server-side refusal of a disabled method, 2FA gate, change-email/delete-user) are unpinned
• identity-auth.invitation-scope-gates (rev3, partial) — C4 (UI invite affordances, both sides), C5 (lifecycle vocabulary pending->accepted/rejected/expired, cancel/resend) and C6 (/security/my-delegable-scope is self-scoped) are unpinned
• identity-auth.admin-lifecycle-operations (rev3, partial) — C8 is the only pinned clause. C0–C7 and C9 (ban persistence/reversal, password rotation, explicit-password-wins, set-role flipping gate outcomes, revoke-user-sessions mid-flight, owner_id FK clear on user removal, impersonation attribution + console visibility, non-admin refusal of every /admin/ route) are unpinned
• integration-system.connector-declarative-boot (rev2, partial) — two tests against 6 clauses — the declarative MCP connector boots, but the wider connector-boot matrix is unpinned
• integration-system.webhook-lifecycle (rev4, partial) — one test against 6 clauses; delivery, retry, signature and teardown clauses are unpinned
• integration-system.email-template-render (rev4, partial) — two tests against 8 clauses; requireVars fail-fast, TEMPLATE_INACTIVE, (name, locale) fallback, fromOverride/replyTo/derived bodyText, customized-row survival, stray-key alias prescription and the raw /email/send door are unpinned
• integration-system.datasource-admin-lifecycle (rev1, partial) — the ref is explicit: this pins route behavior + envelope; the LIVE-mount half is not pinned — a real os dev mount of the datasource is not exercised
• platform-core.nav-surfaces-render (rev3, partial) — C2 (a bad route renders a NAMED not-found/error inside a live shell) and C3 (route-level ErrorBoundary fallback, recoverable via Try Again) have no spec in the smoke — its 35 cases are clean-render and gated-inline-def cases only
• platform-core.metadata-registry-serving (rev2, partial) — six tests against 7 clauses — the create/seed path is pinned; the wider registry-serving surface is not fully reached
• platform-core.metadata-authoring-roundtrip (rev2, partial) — the item carries a ledger stale flag: its studio-record-page.spec.ts ref is STALE, NOT RED (the Object control is now a role=combobox, not an input). This run independently reproduced exactly that — locator.fill: Element is not an <input>. Per the item's own runnerRule the Studio clause (C3, the PUT /api/v1/meta/page/<name>) must be hand-driven and its spec failure must NOT be scored here
• records-forms.list-view-capabilities (rev1, partial) — the user-filters and inline-edit-polish-2572 halves are not consultable in this pairing — see the live-e2e note
• records-forms.conditional-rules-header (rev1, partial) — the field-conditional-rules half (visibleWhen/readonlyWhen/requiredWhen reacting to Status) is not consultable in this pairing
• records-forms.conditional-rules-grid (rev1, partial) — the grid-file-upload half is not consultable in this pairing
• records-forms.cascading-options (rev1, partial) — the client half ("province options re-filter live as country changes, and the stale value clears") is not consultable in this pairing
• records-forms.master-detail-atomic-save (rev2, partial) — the form-view-subforms half is not consultable — and note its failure is a strict-mode locator ambiguity in which the Line Items heading WAS found, i.e. the surface rendered
• records-forms.related-list-server-pagination (rev3, blocked) — not consultable — outside objectui's CI allowlist, unpinned backend pairing
• records-forms.adhoc-filter-sort-builder (rev1, partial) — the user-filters half is not consultable in this pairing
• records-forms.concurrent-edit-conflict (rev1, partial) — the inline-edit-polish-2572 half (single save bar / atomic inline save) is not consultable in this pairing
• records-forms.object-hook-lifecycle (rev1, partial) — the ref is explicit: the transition audit / over-budget warn / priority-ordering LOG oracles and the abort/log/priority SCRATCH variants are NOT dogfood-pinned and must be hand-driven
• records-forms.cascading-multilevel-and-clear (rev1, partial) — the client clear-on-change half (cascading-options.spec.ts) is not consultable in this pairing
• studio-authoring.object-designer-roundtrip (rev1, partial) — F1 ("a newly created package appears in the switcher without a reload") is red; the Studio APP is not installed on stock showcase (meta/app = showcase_app, setup, account), so the switcher clause is not scoreable here
• studio-authoring.record-page-roundtrip (rev2, blocked) — the item carries a ledger stale flag whose runnerRule says outright: do NOT satisfy this item by citing these specs, and do NOT score their failure as a FAIL. This run reproduced the documented staleness exactly (the Object control is now a role=combobox button). Hand-drive until objectui re-points the spec
• studio-authoring.draft-publish-lifecycle (rev4, partial) — three tests against 12 clauses; If-Match 409 metadata_conflict, package-wide publish-drafts atomicity, invalid-draft gate, Studio chrome badges, history/diff/rollback/audit and X-Actor inertness are unpinned

Checklist-accuracy findings (for the ledger, not for code)

1. The pins are narrower than the items on 39 of 77. Not a defect in any pin — but an item scored pass on a pin that reaches 1 of 10 clauses would be a false green. The gap list above is the raw material for either tightening automated.ref scope notes (as fls-mask-and-strip, crud-permission-matrix, datasource-admin-lifecycle and object-hook-lifecycle already do well) or promoting the unpinned clauses to real tests per RUNNER rule 6's reverse direction.
2. 11 items pin objectui e2e/live specs that objectui itself does not gate. Only 4 live specs are in test:e2e:live:ci; the rest are informational and unproven. The ledger should either pin to the allowlist, or record the required pairing (backend.env) so a runner knows a green needs the published backend.
3. e2e/live/ci/backend.env violates its own stated invariant in the pinned objectui tree: OBJECTSTACK_VERSION=17.0.0-rc.2 while this repo's lock resolves @objectstack/spec ^17.0.0-rc.6, though the file says the two "MUST stay aligned … bump both in the same PR". That is an objectui repo issue, filed here only as a pointer.
4. api-backend.query-contract-matrix cites a source module as a pin — packages/objectql/src/filter-comparand-shape.ts is not a test.
5. Cross-tenant clauses are structurally unprovable in the OSS workspace (@objectstack/organizations is cloud-private). The dogfood probe handles this honestly; the checklist should mark such clauses so runners stop rediscovering it.

Not re-derived (already known, confirmed again)

• objectstack verify --rls is a separate invocation — confirmed, 20 PROVEN / 0 HOLES / 9-of-9 personas.
• Playwright needs the explicit executablePath — confirmed, 35 passed with it.
• Studio is not installed on stock showcase — confirmed via meta/app.

Nothing landed in the repo: no commits, no PR, no screenshots. Scratch artifacts stayed in /tmp/qa-wave/**.

Parent: #9296