Skip to content

test(spec): eight scripts/ test-budget rows -- three trimmed, five declare their measured CI budget - #23752

Merged
objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-23180-spec-scripts-rows
Oct 16, 2026
Merged

objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-23180-spec-scripts-rows

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Part of #23180
Clause-②: no

The eight packages/spec/scripts/ rows of the test-budget census, measured first. Three rows are trimmed in the test (every expect line unchanged). Five rows spawn the real producer once per single-cause fixture, so each carries its measured // test-budget: line under ruling record 6181973363. This PR counts no row off the card: the next refresh list decides each one.

Current CI readings (executed hourly runs, not replays)

Read from the Test Core (1/6) shard logs of the three hourly schedule runs on 2026-10-15 whose aggregator lists @objectstack/spec as "measured here" (it is in none of their cache-replayed lists): 38702741098 (08:01, head f7cd580bff), 38731551915 (12:01, head 57994c311f), 38798542648 (20:01, head 9e0945cf45). The newest run, 38814162165 (22:01), replayed spec and is not used. Seconds, vitest module line:

file project 08:01 12:01 20:01 verdict
dist-freshness-adoption.test.ts repo 56.7 71.3 71.4 (b) declared 90
def-key-collisions.test.ts repo 37.6 47.4 51.3 (b) declared 70
strictness-ledger-doc.test.ts repo 32.1 39.1 43.9 (a) trimmed
openapi-self-consistency.test.ts local 37.9 45.2 48.6 (a) trimmed
dist-freshness.test.ts repo 18.1 23.1 24.2 (b) declared 40
upgrade-guide-docs.test.ts local 15.1 19.1 19.7 (b) declared 30
conversions-major18-merge.test.ts repo 10.2 14.0 16.8 (a) trimmed
projection-cli.test.ts local 9.4 11.6 11.6 (b) declared 20

Every row is still over 10 s in at least two of the three runs, so none had dropped off since the card's refresh run 38340176409.

(a) Trimmed: what each one waited on

  • strictness-ledger-doc.test.ts: eleven cases each called loadLedger, which parses every *.zod.ts under src/ into a TypeScript AST (about 400 ms locally per call). loadLedger is a pure read of the checkout and no case writes to what it returns, so the file now loads it once (treeLedger()). The independent analyzeTree walk in "measures the same tree the AST counter does" stays its own walk.
  • conversions-major18-merge.test.ts: entriesOf and definitionsOf ran maskComments over the 750 KB src/conversions/registry.ts about 55 times (about 40 ms per pass, measured), re-deriving the same dozen texts. The mask is a pure function of its input, so the file now masks each distinct text once (maskOnce, keyed by the text).
  • openapi-self-consistency.test.ts: nine spawns of the real build-openapi.ts, five of them the identical unmutated run and two the identical dangling-component-ref mutation. Each distinct fixture is now generated once and read by the cases that assert on it (4 spawns). The spawn calls the package's own node_modules/.bin/tsx, the same binary npx tsx finds in the sandbox, without npx's own start-up: build-openapi.ts ran in 1342 / 1399 / 1519 ms through npx against 886 / 811 / 924 ms direct.

Local module line, interleaved runs under os-verify-lock.sh, BASE 534ca13eb against d2790bd96 (shared-box seconds):

file before (ms) after (ms)
strictness-ledger-doc 5450 / 5145 / 5413 975 / 960 / 1053
openapi-self-consistency 11587 / 10687 / 10898 3465 / 3479 / 3871
conversions-major18-merge 2778 / 2872 / 3036 1150 / 1303 / 1231

At each file's own CI-to-local ratio (8.3x, 4.1x, 5.9x; slowest CI reading over the first local before-reading), the after-readings project to about 8 s, 15 s and 7 s in the battery. That is NOT MEASURED on CI. openapi-self-consistency is projected to stay over 10 s. Its remaining floor is four single-cause spawns of the real producer, one of the ruled classes, so if the next refresh still lists it, its declaration follows with the post-trim CI reading. This PR does not declare it with the pre-trim number.

(b) Declared: the floor is one producer spawn per single-cause fixture

Per-case local readings show where each file's time goes. Each number is the slowest of the three CI readings above x 1.25, rounded up to the next 10 s:

file where the time goes (local) arithmetic
dist-freshness-adoption all 10 cases, one spawn each of a real dist-reading gate in its own sandbox (0.74 to 2.31 s per case) 71.362 x 1.25 = 89.2, so 90
def-key-collisions one case, 8425 of 8430 ms: the real schema generator on a sandbox copy of src with the one re-declared def key 51.268 x 1.25 = 64.1, so 70
dist-freshness 4 spawn cases, 4531 of 4595 ms: 5 runs of the real build-api-surface.ts 24.165 x 1.25 = 30.2, so 40
upgrade-guide-docs 4 spawn cases, 7077 of 7106 ms: 9 runs of the real build-upgrade-guide.ts 19.673 x 1.25 = 24.6, so 30
projection-cli 4 spawn cases, 4790 of 4819 ms: 6 runs of the two real projection generators 11.642 x 1.25 = 14.6, so 20

What one spawn costs, measured locally: a bare tsx -e 1 takes 424 to 587 ms, and a build-upgrade-guide.ts usage error that exits 2 before writing anything takes 733 to 901 ms, which is about the same as a full --check (805 to 914 ms). The cost is tsx start-up plus importing the registries, and generation itself is negligible. The sandbox copies are not the cost either: fs.cpSync of scripts/ takes 4 to 14 ms, and of src/ 101 to 379 ms.

Each line is at line 2, under the copyright line, as the four declarations on main are. readTestBudget and declaredBudgetInTree('@objectstack/spec', FILE) from scripts/measure-test-shard-timings.mjs read 90, 70, 40, 30 and 20, and null for the three trimmed files. Two controls are refused: a misspelled class ("the class ... is neither class the ruling names") and a 15 s budget ("not on the 10 s grid").

Ablations: the trimmed paths still go red

All three ran through scripts/ablation-replace.mjs in wrap mode. Expected direction: red. Observed: red. Each restore was proven (blob equals HEAD, git diff HEAD empty), and the tree was clean after. No dist/ is in any path: tsx and vitest read source.

  • A1, scripts/build-openapi.ts without its BaseResponse component: blob 04db95741ff4 to be6bcfae2f9c. 3 of 21 failed, including both shared-run cases ("emits all nine components": expected output to contain Components: 9; "writes a document whose components are complete": length 9, got 8). The second shared case took 6 ms, which shows it read the shared run.
  • A2, the ui/ posture row in docs/audits/2026-07-unknown-key-strictness-ledger.counts/ui.md edited: blob a95f31caf2ce to 5ddf40dc3f32. 2 of 24 failed: "is checked in current" and "the read-time totals equal the sums of the shards' own rows" ([478, 352, 6, 1, 119] against [478, 353, 6, 1, 118]). Both read the memoised ledger.
  • A3, two adjacent MAJOR_18_CONVERSIONS entries swapped in src/conversions/registry.ts: blob 3e1bfb1b04f9 to e3de633149bf. 4 of 12 failed, including "the entries are kept SORTED by identifier", all through the memoised mask.

Verification

  • Final head c0ca73fd2 (merged with origin/main 3b46855fb, spec rebuilt): all eight files pass. Tests per file: 10, 17, 24, 21, 24, 15, 12 and 12, 135 in all.
  • pnpm --filter @objectstack/spec typecheck at c0ca73fd2: VERDICT command-exit 0. The eight files are compiled by tsconfig.scripts.json (check:scripts-typecheck), which lists 8 of 8.
  • Gates: node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack at c0ca73fd2 derived 59 commands. All ran, each exit recorded before any pipe. --ran reports "59 derived famil(ies) accounted for — 57 run, 2 NOT-MEASURED (2 DERIVED from a recorded exit 3)". The two are check:dual-build-cjs-loads and check:lean-entry-closure: PREREQUISITE NOT MET, because the dist of other packages was not built here. Neither reads a test file under packages/spec/scripts/. CI runs both.
  • No publish: npm pack --dry-run of @objectstack/spec 18.0.0-next.1 lists 2092 files, with 0 under scripts/, 0 *.test.ts and none of the eight. Positive controls src/data/field.zod.ts and dist/index.js are present. So skip-changeset.
  • Local test scope: only the eight changed files ran here. The diff is test files only, and no other test imports them. CI's Test Core runs both spec projects.

Acceptance notes

  • Two declared rows from the earlier round read above their budgets in executed run 38798542648: build-schemas-check-mode.test.ts read 983.27 s against its declared 940 s, and check-liveness.test.ts read 425.63 s against its declared 410 s. In run 38731551915 they read 925.47 s and 402.21 s. Neither file is in this round. The next refresh's list decides whether either needs a re-measured line.
  • def-key-collisions.test.ts still spawns through npx tsx, which costs about 0.5 s of its 8.4 s locally. It is left unchanged, so the declared number is measured on the code it is declared for.
  • The vitest module line excludes the file's import time. For example, conversions-major18-merge's module line was 2866 ms beside an import of 3.38 s in the same run. The census and these budgets measure the module line only.

Generated by Claude Code

…heir measured CI budget

strictness-ledger-doc loads the tree's ledger once per file instead of once
per case; conversions-major18-merge masks each distinct registry text once;
openapi-self-consistency generates each distinct fixture once (the unmutated
source for five cases, the dangling component ref for two) and spawns the
package's own tsx instead of going through npx. Every expect line is
unchanged.

dist-freshness-adoption, def-key-collisions, dist-freshness,
upgrade-guide-docs and projection-cli spawn the real producer once per
single-cause fixture, so each carries its measured budget line under ruling
6181973363 (slowest of three executed hourly runs x 1.25, rounded up).

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HjR8RUwTPQ3TgbfKihmxp9
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 2906cd8d2e568054a4894f01c801a35b83470692 → packageMentionDocs.

@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 15, 2026
@objectstack-fleet

objectstack-fleet Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor Author

席位核对 · ACCEPT · PR #23752 @ c0ca73fd2 (#23180, the eight packages/spec/scripts/ rows) · domain:spec seat 1 · os-elon-musk · session session_01HjR8RUwTPQ3TgbfKihmxp9 · 2026-10-16T00:06Z

Reviewer of record. Every item below was checked against GitHub, not against the report (6204837006).

  • Shape:
    • Draft. The first line is Part of #23180, with no closing keyword anywhere, and the card stays open. Clause-②: no.
    • 8 files, all packages/spec/scripts/*.test.ts, +65/−23. No governed path, no published schema, no Clause-②: yes, so no contract review is owed. CI plus this read decide it.
    • skip-changeset: the files ship in no tarball (npm pack --dry-run: 0 of 2,092 entries under scripts/, with controls present).
  • Ruling 6181973363, the five declarations: each line was read in the diff. The spelling is // test-budget: SECONDS — CLASS: REASON, and the arithmetic was re-done by the seat from the cited reading (CI s × 1.25, then up to the next 10):
file slowest CI (run 38798542648) × 1.25 declared
dist-freshness-adoption.test.ts 71.4 89.2 90
def-key-collisions.test.ts 51.3 64.1 70
dist-freshness.test.ts 24.2 30.2 40
upgrade-guide-docs.test.ts 19.7 24.6 30
projection-cli.test.ts 11.6 14.6 20

The class is "spawns the real producer once per single-cause fixture" for all five. Each REASON names the producer spawned, and the per-case floors in the report put more than 98 % of each file's time in those spawns. The seat ran readTestBudget itself on the eight head blobs (fetched to a seat-owned ref): the five declared files parse (90 / 70 / 40 / 30 / 20 s, with the class above), and the three trimmed files read null. The two refused controls are the dev's reading.

  • Three trims (verdict a), with every expect line unchanged in the diff:
    • strictness-ledger-doc: one loadLedger per file.
    • conversions-major18-merge: maskComments memoised per distinct text.
    • openapi-self-consistency: one generator run per distinct fixture, using the package's own tsx.
    • The memoised inputs are pure reads that no case writes to.
  • No row is counted off. The next refresh list decides each one, per the card's pin. openapi-self-consistency is left undeclared after its trim. That is right: declaring on its pre-trim 48.6 s would budget it at about 5× its new cost.
  • Out of scope, carried on this card (test budget (domain:spec): 26 test file(s) over 10 s of wall in the hourly full battery (Part of #23151) #23180): two earlier declarations read above their budgets in executed run 38798542648: build-schemas-check-mode.test.ts 983.3 s against 940, and check-liveness.test.ts 425.6 s against 410. Ruling item 2 says a budget "rises only by a PR that re-measures and says why in the line". That is this card's next slice for the seat that takes it. The two @objectstack/lint rows and the src/ rows are also open.
  • CI on this head at this act: in progress. The seat readies and arms auto-merge once every check is green.

Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants