Skip to content

test(runtime): keep module loads out of three slow test files' clocked windows (runtime row of #23181) - #23204

Merged
objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-23181-s1-runtime-test-budget
Oct 13, 2026
Merged

objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-23181-s1-runtime-test-budget

Conversation

@objectstack-fleet

@objectstack-fleet objectstack-fleet Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Part of #23181
Clause-②: no

The @objectstack/runtime row of #23181 (slice 1): three test files the hourly full battery read over 10 s of wall. This PR edits those three test files only: no runtime source, no vitest config, no timeout value, no test removed or moved.

Rows

file card's CI reading this box, isolated, median of 5 (before to after) this box, whole-package run (before to after)
src/declared-capability-absence-warn-once.test.ts 19.4 s (38325245250) 23145 ms to 101 ms 10990 ms to 91 ms
src/stored-metadata-reader-contexts.pin.test.ts 20.7 s (38314183560) 5756 ms to 5085 ms 7241 ms to 8207 ms (one run each, see notes)
src/stored-metadata-body-boundary.pin.test.ts 13.7 s (38210849670) 4976 ms to 4149 ms 6012 ms to 7085 ms (one run each, see notes)
  • warn-once: this PR completes that file's row. Its clocked work goes from seven module-graph evaluations to none (margin to 10 s: about 9.9 s on this box, isolated and in the whole-package run).
  • The two pin files: this PR removes the one test-side cost found and does not claim the rows. Both were already under 10 s on this box before the change (isolated and in the whole-package run); their CI readings are the slowest of 20 contended hourly runs. What remains in their clocked windows is CPU-bound boot and sandbox work (measured below), not a sleep, retry or poll. The next hourly full battery's timing refresh (shard-timings-refresh.yml, measure-test-shard-timings --diff) reads all three; if it names a pin file again, the remaining cost is outside the test (Acceptance notes).

"Isolated" is the file run alone, five times, with the agent-detection variables unset so vitest picks the default reporter CI reads; the number is the file line's result.duration (hooks plus tests, collection excluded). The pin files' before and after runs were interleaved (base, after, base, after), with the base bytes run from an untracked copy byte-identical to the merge base's blob (9423d04d44 for body-boundary). Every after run sat below every before run. This is a shared 4-core box, so read the ratios as well as the absolutes.

Where the time went, and the change

declared-capability-absence-warn-once.test.ts

  • Before: each of the 7 tests called vi.resetModules() and re-imported the dispatcher and sendError, re-evaluating the same 733 modules (spec/src 422, core/src 93, platform-objects/src 71, runtime/src 64, rest/src 24, types/src 22, plus the inlined objectql and metadata-protocol dist bundles): about 0.9 s per test isolated, 1.1 to 1.4 s per test in the whole-package run. In isolation the first test also paid the graph's cold transform inside its clock (17.0 s; vitest transform 15.5 to 17.0 s).
  • Change: one module graph, imported at module top so the transform is paid during collection (the clocked-window rule, scripts/check-test-source-alias.mjs). The per-test isolation the reset bought is kept by the route key instead. The dedupe registry is keyed on METHOD plus the mounted route pattern (parked by observability/instrument.ts), so each boot() mounts the dispatcher under its own prefix (/t1/api/v1, /t2/api/v1, and so on). Every key a test touches is one the registry has never reported, the same precondition a fresh process gives those keys. Handler lookups go through route('GET /notifications'); asserted paths through path('/notifications').
  • Whole-file wall with collection included also fell: vitest Duration 22.2 to 23.5 s before, 17.4 to 19.7 s after (isolated).

stored-metadata-reader-contexts.pin.test.ts and stored-metadata-body-boundary.pin.test.ts

  • Where it goes: one boot per file in beforeAll (already shared), then the tests. On the base bytes of the reader file (3 runs), beforeAll took 2.7 to 3.1 s wall against 3.1 to 3.4 s CPU, and the tests 2.6 to 2.9 s wall against 5.5 to 6.2 s CPU. CPU at or above wall in both windows means no idle wait to remove. bootstrap() is about 2.0 s, of which the objectql plugin's start() is 0.54 to 0.58 s and init/ready the rest. The two waitFor polls (100 ms interval, for a /meta-authored action or hook to bind through the event-driven resync) settle in 153 to 274 ms per test, and in 826 ms for the body-boundary file's hook bind, whose probe runs three hook bodies per attempt.
  • The test-side cost found: bootstrap() first-evaluates two workspace modules inside beforeAll, driver-sqlite-wasm/dist and driver-sql/dist, reached through service-datasource's lazy import(). These are the transform-queue-exposed kind of load the clocked-window rule is about. Every other first load in the boot is a native external (better-auth, knex, sql.js, nanoid), and better-auth is not a runtime dependency, so a module-top import of it would be a phantom dependency.
  • Change: module-top import '@objectstack/driver-sqlite-wasm' and import '@objectstack/driver-sql', the shape datasource-visibility.test.ts already carries. Both pairs were already registered for runtime in KNOWN_UNALIASED_TEST_IMPORTS. Measured: bootstrap() 2.0 to 2.1 s became 1.5 to 1.8 s in a probe; the files went 12% and 17% faster isolated.

Every assertion kept

  • All three files: it/describe title lines are hash-identical at the merge base and head (reader 21 it sites / 27 tests, body-boundary 5 / 7, warn-once 7 / 7). The two pin files gained two import lines and a comment; no test body changed.
  • warn-once: 25 expect( calls before and after, and each test proves the same thing:
    • reports a declared 501 ONCE across N requests, at warn, naming the missing service: five requests to a route the registry never saw, one warn, zero error. The asserted path is this boot's mounted pattern.
    • a SECOND, DIFFERENT route still reports — the dedupe key is the route: two never-seen routes, one line each, same code and prose.
    • an UNDECLARED 500 on the same door is still loud, once per request: three faults, three error lines, no warn.
    • a declared absence and a fault coexist in one process without either changing the other: one warn and two error on one mount.
    • the envelope the dispatcher answers is classified identically by the REST envelope writer: the dispatcher and sendError still share one @objectstack/types registry (one module graph), and both lines name the service and the suppression.
    • answers the same bytes it answered before this change, on every request: request 1 is still the reported occurrence on a never-seen route, so the first-versus-suppressed byte comparison still holds.
    • answers the same bytes for a fault, too: unchanged.

One-shot proofs (run from the committed state, not kept as tests)

Both went through scripts/ablation-replace.mjs (anchor hit counted, blob hash moved, restore proven by the head blob hash and an empty git diff HEAD), against the warn-once file at 8d5fab4bdc:

  1. Isolation removed (every boot on the shared /api/v1 prefix): 3 failed, 4 passed. a SECOND, DIFFERENT route… got 1 line where 2 were expected, …coexist… 0 where 1 was, and the envelope… (door 1) 0 where 1 was. That is the order-dependence the reset used to prevent, so the prefix is load-bearing. Blob a643630be43d to e35018939f4d, restored to a643630be43d.
  2. Dedupe removed (packages/types/src/server-fault-log.ts, the has(key) early return; aliased to source for this suite, so no build leg): 3 failed. reports … ONCE… got 5 lines where 1 was expected, a SECOND… 4 where 2 were, and …coexist… 2 where 1 was. The file still pins the dedupe itself. Blob 7ed86cc374e8 to 758f68960025, restored.

Verification at 8d5fab4bdc (branch merged with main at d4aaeaf0e2)

  • pnpm --filter @objectstack/runtime test: Test Files 364 passed (364), Tests 5178 passed | 19 skipped (5197), lock verdict command-exit 0.
  • pnpm --filter @objectstack/runtime typecheck: exit 0. tsc --noEmit is clean, and check:test-typecheck: OK. The three files are in the tsconfig.test.json program (3 of 367 runtime test files) and carry no entry in test-typecheck-debt.json.
  • Lint, narrowed to the touched files with its proof:
    • eslint, invoked as pnpm lint invokes it with --no-inline-config --format json, over the 3 files: 3 files, 0 errors, 0 warnings, 0 fatal.
    • --print-config resolves each file to an active config (not ignored).
    • eslint.config.mjs enables no type-aware linting (no parserOptions.project, no typed rules; its own comment says so), so this diff cannot move any untouched file's verdict.
  • node scripts/pm/dispatch-gates.mjs --commands: 50 commands, run as printed with exit codes captured before any pipe. 49 exit 0 with their own pass lines. pnpm check:dual-build-cjs-loads first answered PREREQUISITE NOT MET (exit 3: 38 packages had no dist/). After a full build (74/74 tasks), the same command answered exit 0: 108 require entry points across 67 packages load. --ran: 50 derived, 50 run, 0 NOT-MEASURED, 0 UNRUN. (Corrected by the seat from the dev's report; the PR body had been written before that run.)
  • No changeset: runtime's files is dist, README.md, CHANGELOG.md. A string unique to this change has 0 hits in them; a control string from dispatcher-plugin.ts has 1 hit in dist/index.js. Label skip-changeset.

Acceptance notes

  • The pin files' residual, if a refresh names them again. Their clocked work is the composed boot (about 1.5 s after this change) plus one sandboxed body per assertion. The sandbox instantiates a fresh WASM module per invocation by design (ADR-0102 D2), which shows up as about 2x CPU over wall in the test window. A 4-vCPU shard running turbo --concurrency=4 multiplies exactly that. The card's readings sit 2.3x to 2.9x over this box's whole-package run. Neither lever is a fixed delay or poll the test waits on, so neither is in this slice: a cheaper boot, or reusing compiled WASM across invocations while keeping separate instances. Observation only; carrier: none.
  • Whole-package before/after for the pin files is not an A/B. It is one run each, on different trees (merge base versus merged main), on a shared box (the after run queued 187 s behind sibling work), and with a changed file order (vitest schedules by previous duration, and warn-once is no longer the slowest file). The module-top imports cannot add clocked time, since they move evaluation into collection. The interleaved isolated runs are the comparison.
  • warn-once's vi.setConfig({ testTimeout: 30_000 }) was raised for the per-test reload this PR removes. It is left as it stands, because this slice's claim excludes timeout values. Its comment now says why it was raised, so whoever next owns the file can drop it.
  • @objectstack/runtime CI reads (Test Core) will be the first contended reading of the new shapes. The hourly timing refresh is the pin.

Generated by Claude Code

…ree slow files' clocked windows

declared-capability-absence-warn-once: one module graph at module top; each
boot mounts the dispatcher under its own route prefix instead of
vi.resetModules() plus a fresh import per test, so each test still meets a
registry that never saw its route keys without re-evaluating ~730 modules.

stored-metadata-reader-contexts.pin / stored-metadata-body-boundary.pin: pay
the two default-datasource drivers' first transform at module load, as
datasource-visibility.test.ts does, instead of inside beforeAll's bootstrap.

Claude-Session: https://claude.ai/code/session_01VoSxBQujKLZKPwK2u5ehQ6
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 44e2d505c7e6f5da8efc4419fc9de40bbee9ebc0 → packageMentionDocs.

@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 13, 2026
@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 13, 2026 08:56
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 13, 2026
Merged via the queue into main with commit 75cada0 Oct 13, 2026
52 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-23181-s1-runtime-test-budget branch October 13, 2026 09:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants