Repository navigation
test(spec): eight scripts/ test-budget rows -- three trimmed, five declare their measured CI budget - #23752
Conversation
…heir measured CI budget strictness-ledger-doc loads the tree's ledger once per file instead of once per case; conversions-major18-merge masks each distinct registry text once; openapi-self-consistency generates each distinct fixture once (the unmutated source for five cases, the dangling component ref for two) and spawns the package's own tsx instead of going through npx. Every expect line is unchanged. dist-freshness-adoption, def-key-collisions, dist-freshness, upgrade-guide-docs and projection-cli spawn the real producer once per single-cause fixture, so each carries its measured budget line under ruling 6181973363 (slowest of three executed hourly runs x 1.25, rounded up). Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HjR8RUwTPQ3TgbfKihmxp9
📓 Docs Drift CheckNothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs. What this run could not see
Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): |
席位核对 · ACCEPT · PR #23752 @
|
| file | slowest CI (run 38798542648) |
× 1.25 | declared |
|---|---|---|---|
dist-freshness-adoption.test.ts |
71.4 | 89.2 | 90 |
def-key-collisions.test.ts |
51.3 | 64.1 | 70 |
dist-freshness.test.ts |
24.2 | 30.2 | 40 |
upgrade-guide-docs.test.ts |
19.7 | 24.6 | 30 |
projection-cli.test.ts |
11.6 | 14.6 | 20 |
The class is "spawns the real producer once per single-cause fixture" for all five. Each REASON names the producer spawned, and the per-case floors in the report put more than 98 % of each file's time in those spawns. The seat ran readTestBudget itself on the eight head blobs (fetched to a seat-owned ref): the five declared files parse (90 / 70 / 40 / 30 / 20 s, with the class above), and the three trimmed files read null. The two refused controls are the dev's reading.
- Three trims (verdict a), with every
expectline unchanged in the diff:strictness-ledger-doc: oneloadLedgerper file.conversions-major18-merge:maskCommentsmemoised per distinct text.openapi-self-consistency: one generator run per distinct fixture, using the package's owntsx.- The memoised inputs are pure reads that no case writes to.
- No row is counted off. The next refresh list decides each one, per the card's pin.
openapi-self-consistencyis left undeclared after its trim. That is right: declaring on its pre-trim 48.6 s would budget it at about 5× its new cost. - Out of scope, carried on this card (test budget (domain:spec): 26 test file(s) over 10 s of wall in the hourly full battery (Part of #23151) #23180): two earlier declarations read above their budgets in executed run
38798542648:build-schemas-check-mode.test.ts983.3 s against 940, andcheck-liveness.test.ts425.6 s against 410. Ruling item 2 says a budget "rises only by a PR that re-measures and says why in the line". That is this card's next slice for the seat that takes it. The two@objectstack/lintrows and thesrc/rows are also open. - CI on this head at this act: in progress. The seat readies and arms auto-merge once every check is green.
Generated by Claude Code
Part of #23180
Clause-②: no
The eight
packages/spec/scripts/rows of the test-budget census, measured first. Three rows are trimmed in the test (everyexpectline unchanged). Five rows spawn the real producer once per single-cause fixture, so each carries its measured// test-budget:line under ruling record6181973363. This PR counts no row off the card: the next refresh list decides each one.Current CI readings (executed hourly runs, not replays)
Read from the
Test Core (1/6)shard logs of the three hourly schedule runs on 2026-10-15 whose aggregator lists@objectstack/specas "measured here" (it is in none of their cache-replayed lists):38702741098(08:01, headf7cd580bff),38731551915(12:01, head57994c311f),38798542648(20:01, head9e0945cf45). The newest run,38814162165(22:01), replayed spec and is not used. Seconds, vitest module line:dist-freshness-adoption.test.tsdef-key-collisions.test.tsstrictness-ledger-doc.test.tsopenapi-self-consistency.test.tsdist-freshness.test.tsupgrade-guide-docs.test.tsconversions-major18-merge.test.tsprojection-cli.test.tsEvery row is still over 10 s in at least two of the three runs, so none had dropped off since the card's refresh run
38340176409.(a) Trimmed: what each one waited on
strictness-ledger-doc.test.ts: eleven cases each calledloadLedger, which parses every*.zod.tsundersrc/into a TypeScript AST (about 400 ms locally per call).loadLedgeris a pure read of the checkout and no case writes to what it returns, so the file now loads it once (treeLedger()). The independentanalyzeTreewalk in "measures the same tree the AST counter does" stays its own walk.conversions-major18-merge.test.ts:entriesOfanddefinitionsOfranmaskCommentsover the 750 KBsrc/conversions/registry.tsabout 55 times (about 40 ms per pass, measured), re-deriving the same dozen texts. The mask is a pure function of its input, so the file now masks each distinct text once (maskOnce, keyed by the text).openapi-self-consistency.test.ts: nine spawns of the realbuild-openapi.ts, five of them the identical unmutated run and two the identical dangling-component-ref mutation. Each distinct fixture is now generated once and read by the cases that assert on it (4 spawns). The spawn calls the package's ownnode_modules/.bin/tsx, the same binarynpx tsxfinds in the sandbox, without npx's own start-up:build-openapi.tsran in 1342 / 1399 / 1519 ms through npx against 886 / 811 / 924 ms direct.Local module line, interleaved runs under
os-verify-lock.sh, BASE534ca13ebagainstd2790bd96(shared-box seconds):strictness-ledger-docopenapi-self-consistencyconversions-major18-mergeAt each file's own CI-to-local ratio (8.3x, 4.1x, 5.9x; slowest CI reading over the first local before-reading), the after-readings project to about 8 s, 15 s and 7 s in the battery. That is NOT MEASURED on CI.
openapi-self-consistencyis projected to stay over 10 s. Its remaining floor is four single-cause spawns of the real producer, one of the ruled classes, so if the next refresh still lists it, its declaration follows with the post-trim CI reading. This PR does not declare it with the pre-trim number.(b) Declared: the floor is one producer spawn per single-cause fixture
Per-case local readings show where each file's time goes. Each number is the slowest of the three CI readings above x 1.25, rounded up to the next 10 s:
dist-freshness-adoptiondef-key-collisionssrcwith the one re-declared def keydist-freshnessbuild-api-surface.tsupgrade-guide-docsbuild-upgrade-guide.tsprojection-cliWhat one spawn costs, measured locally: a bare
tsx -e 1takes 424 to 587 ms, and abuild-upgrade-guide.tsusage error that exits 2 before writing anything takes 733 to 901 ms, which is about the same as a full--check(805 to 914 ms). The cost is tsx start-up plus importing the registries, and generation itself is negligible. The sandbox copies are not the cost either:fs.cpSyncofscripts/takes 4 to 14 ms, and ofsrc/101 to 379 ms.Each line is at line 2, under the copyright line, as the four declarations on
mainare.readTestBudgetanddeclaredBudgetInTree('@objectstack/spec', FILE)fromscripts/measure-test-shard-timings.mjsread 90, 70, 40, 30 and 20, andnullfor the three trimmed files. Two controls are refused: a misspelled class ("the class ... is neither class the ruling names") and a 15 s budget ("not on the 10 s grid").Ablations: the trimmed paths still go red
All three ran through
scripts/ablation-replace.mjsin wrap mode. Expected direction: red. Observed: red. Each restore was proven (blob equals HEAD,git diff HEADempty), and the tree was clean after. Nodist/is in any path: tsx and vitest read source.scripts/build-openapi.tswithout itsBaseResponsecomponent: blob04db95741ff4tobe6bcfae2f9c. 3 of 21 failed, including both shared-run cases ("emits all nine components": expected output to containComponents: 9; "writes a document whose components are complete": length 9, got 8). The second shared case took 6 ms, which shows it read the shared run.ui/posture row indocs/audits/2026-07-unknown-key-strictness-ledger.counts/ui.mdedited: bloba95f31caf2ceto5ddf40dc3f32. 2 of 24 failed: "is checked in current" and "the read-time totals equal the sums of the shards' own rows" ([478, 352, 6, 1, 119]against[478, 353, 6, 1, 118]). Both read the memoised ledger.MAJOR_18_CONVERSIONSentries swapped insrc/conversions/registry.ts: blob3e1bfb1b04f9toe3de633149bf. 4 of 12 failed, including "the entries are kept SORTED by identifier", all through the memoised mask.Verification
c0ca73fd2(merged withorigin/main3b46855fb, spec rebuilt): all eight files pass. Tests per file: 10, 17, 24, 21, 24, 15, 12 and 12, 135 in all.pnpm --filter @objectstack/spec typecheckatc0ca73fd2: VERDICT command-exit 0. The eight files are compiled bytsconfig.scripts.json(check:scripts-typecheck), which lists 8 of 8.node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackatc0ca73fd2derived 59 commands. All ran, each exit recorded before any pipe.--ranreports "59 derived famil(ies) accounted for — 57 run, 2 NOT-MEASURED (2 DERIVED from a recorded exit 3)". The two arecheck:dual-build-cjs-loadsandcheck:lean-entry-closure: PREREQUISITE NOT MET, because the dist of other packages was not built here. Neither reads a test file underpackages/spec/scripts/. CI runs both.npm pack --dry-runof@objectstack/spec18.0.0-next.1 lists 2092 files, with 0 underscripts/, 0*.test.tsand none of the eight. Positive controlssrc/data/field.zod.tsanddist/index.jsare present. Soskip-changeset.Test Coreruns both spec projects.Acceptance notes
38798542648:build-schemas-check-mode.test.tsread 983.27 s against its declared 940 s, andcheck-liveness.test.tsread 425.63 s against its declared 410 s. In run38731551915they read 925.47 s and 402.21 s. Neither file is in this round. The next refresh's list decides whether either needs a re-measured line.def-key-collisions.test.tsstill spawns throughnpx tsx, which costs about 0.5 s of its 8.4 s locally. It is left unchanged, so the declared number is measured on the code it is declared for.conversions-major18-merge's module line was 2866 ms beside an import of 3.38 s in the same run. The census and these budgets measure the module line only.Generated by Claude Code