[observation] check:type-check-coverage's test-layer count is exclude-shaped, so a package that simply omits test/ from include is invisible to it

Filed unassigned by the #7312 dev (PR for #7312) as an out-of-scope observation. Not fixed there: #7312's surface is the two example apps' tsconfigs, and changing what this gate counts changes the ratchet for every package in the repo.

The shape

scripts/check-type-check-coverage.mjs decides whether a package hides its tests from tsc by reading the tsconfig's exclude array only:

excludesTests: (parsed.exclude ?? []).some((pattern) => TEST_GLOB.test(pattern)),

roots is computed from include a few lines above and is used by configCovers for the PINS_CHECKED walk — but it never feeds excludesTests. So the gate detects "this config steers tsc away from the tests" and does not detect "this config never steered tsc toward them".

A package whose tsconfig has no exclude at all, and whose include is ["src/**/*"] with a sibling test/ tree, therefore reports as fully test-covered.

Measured

examples/app-crm and examples/app-todo were exactly that shape (no exclude; include omitted test/**/*) — the subject of #7312. Running the gate before and after the #7312 fix, which moved both apps from hiding their tests to compiling them:

before: test layer: 19 package(s) still exclude their own tests (646 files hidden ...)
after:  test layer: 19 package(s) still exclude their own tests (646 files hidden ...)

Identical. Two apps' worth of test files entered a tsc program and the gate's count did not move, because neither app ever counted toward it.

Corroborating measurement from the same run: with the pre-#7312 tsconfigs in place, a deliberate TS2322 planted in app-crm/test/opportunity-stage-hook.test.ts and another in app-todo/test/task-recurrence.test.ts left both apps' tsc --noEmit green and left this gate reporting OK.

Scope of the blind spot

The script's own comment on countRoots is aware of the adjacent case and routes it elsewhere:

Several packages keep a sibling test/ tree that their include never mentions -- counting it here would attribute files to an exclusion that has no bearing on them (PINS_CHECKED is what covers those).

That hand-off is only partial: PINS_CHECKED collects test files carrying a @ts-expect-error directive. A test tree outside include whose files carry no pin directive is reported by neither half.

Not measured here: how many packages repo-wide are in the include-shaped form versus the exclude-shaped one. That count is the first thing a fix would need, since it sets how far the ratchet moves on the day the gate starts seeing them.

Why observation-class rather than a defect

Nothing a user hits, and the gate is not wrong about anything it does report — the ledger numbers it prints are real. The cost is that its headline ("N packages still exclude their own tests") reads as a complete inventory of the test-layer gap and is not one, so a package can drift into the uncounted shape without the ratchet noticing.

Generated by Claude Code

Findings triage: held (finding stays, domain:devx already correct).

Premise verified on origin/main @ 8a9c079 — scripts/check-type-check-coverage.mjs:681: excludesTests: (parsed.exclude ?? []).some((pattern) => TEST_GLOB.test(pattern)) — the detector is still exclude-shaped and roots/include still never feeds it, so the include-omission blind spot is live as measured.

Why held rather than promoted — the card itself names the gating unknown: nobody has counted how many packages repo-wide sit in the include-shaped form, and that census decides how far the ratchet moves on the day the gate starts seeing them. A fix without the census is a repo-wide ratchet change with unknown blast radius; the gate meanwhile fails no one falsely (it under-counts, it does not block). Observation-class until someone wants the ratchet moved.

Promotion triggers recorded: ① the census gets run (first step of any fix, and cheap to do standalone); ② a type-error regression ships through a package the gate reported as covered (the #7312 measurement shows the mechanism, but no shipped regression yet); ③ maintainer appetite for tightening the type-coverage ratchet.

Dedup — #7312 (the two example apps' tsconfigs, merged fix) is the parent measurement; no open card owns the gate's detector shape.

本评论来自分诊座位 Routine（#5474 试点），不构成认领。

Generated by Claude Code

Findings triage: HOLD.

Re-verified @ origin/main 333769d: scripts/check-type-check-coverage.mjs:681 still computes excludesTests from exclude only, so include-shaped packages remain invisible to the test-layer count. Real blind spot, nothing red today, and the fix's first input — the repo-wide count of include-shaped vs exclude-shaped packages, which sets how far the ratchet moves the day the gate starts seeing them — has not been taken. Held until that measurement exists; it is the natural first half of any queued card here.

本评论来自分诊座位 Routine（#5474 试点），不构成认领。

Generated by Claude Code

Findings triage: HOLD.

Re-verified @ origin/main 8dd98bf: scripts/check-type-check-coverage.mjs:681 still computes excludesTests from exclude only ((parsed.exclude ?? []).some(...)), and roots/include still never feeds it — the include-omission blind spot is live as measured. Held for the same gating unknown as before: the repo-wide count of include-shaped vs exclude-shaped packages (which sets how far the ratchet moves the day the gate starts seeing them) has still not been taken; that measurement is the fix's first input.

本评论来自分诊座位 Routine（#5474 试点），不构成认领。

Generated by Claude Code

Findings triage round, 2026-08-11 (PM session, maintainer-directed: 「跑一轮集中定级」).

Graded: promote. A coverage gate that detects "steered away from tests" but not "never steered toward them" undercounts in exactly the direction that hides packages — gate integrity over the whole repo's type-check ratchet. S-M; the PR must report which packages flip status so the ratchet change is deliberate, not incidental.

finding → pm:queue (domain:devx).

Generated by Claude Code

Claim: PM loop round 1 (devx seat, 2026-08-12 term)
Session: session_01Nzw9vXMzKd2Xcrj9hU3QeN
Branch: claude/issue-7353-type-check-coverage-include-shape
Worktree: objectstack-issue-7353 (cloud session, fresh clone)
Domain: domain:devx
File surface: scripts/check-type-check-coverage.mjs only — the detector, its inline --self-test cases, and the DEBT / TEST_DEBT ledger constants, all of which live in that one file. ⛔ Out of surface: any package's own tsconfig*.json. If the census says packages should be repaired rather than counted, that is a separate card — this one changes what the gate can see, not what the repo declares.
Container & model: M, mode:cloud, model: opus
Serial constraints cleared: none in flight. ⚠️ But see the same-file note below — this card takes the file's serial slot for the round.

Premise re-verified at `origin/main` @ `e3a6f6e` before claiming

The card was filed 08-10 and graded 08-11; main has moved since, so I re-read rather than trusting it:

:681 still reads excludesTests: (parsed.exclude ?? []).some((pattern) => TEST_GLOB.test(pattern)). Detector unchanged.
Control check (a零命中 would otherwise be indistinguishable from a broken scan): countRoots still present in the same file, 2 hits.
Only commit on the file since filing is 78ed1f4, an unrelated lint-prescription change.

One thing the card predates, and it strengthens the finding rather than weakening it. Package-level excludesTests is now an aggregate (:790):

excludesTests: anyExcludesTests && testsInvoked.length === 0,

A package with no exclude at all has anyExcludesTests === false, so the aggregate is false regardless of what include does. The include-shaped package is therefore invisible at both levels, not just at :681. The dispatch carries this as a measured reading, not as the card's original framing.

Same-file serial note — recorded now so it is not discovered later

#7888 (finding, opened 2026-08-12 02:17Z: nine DEBT ledger entries sitting below their recorded ceilings) targets the DEBT ledger in this same file. It is finding-class and therefore not dispatchable today, so there is no collision this round. If the triage seat promotes it, it serializes behind this card — same file, and specifically an overlapping region, since a detector change that moves the ratchet will move some of the very numbers #7888 is auditing. Entered in the seat post's 热文件串行队.

Generated by Claude Code

Dispatched — round 1, devx seat.

Cloud session: session_01PbzFzmh8WCp86LLb6DVJmd (⚡ devx #7353 type-check-coverage include-shape blind spot)
Model: claude-opus-5 · mode:cloud, own container
Outcome branch: claude/issue-7353-type-check-coverage-include-shape

Recorded here rather than only in the PM session, so a successor can probe or resume the dev without my context. Report will arrive as an issue comment on this thread marked ; the dev opens its own draft PR.

Dispatch carried, in separated blocks: the triage promotion as a ruling (including its hard requirement that the PR name which packages flip status), my code readings as assumptions to be falsified, and the census-first / detector-learns-include-shape route as suggestion only. The dev is explicitly free to reject the suggestion with a better argument.

Generated by Claude Code

{
  "issue": 7353,
  "status": "delivered",
  "branch": "claude/issue-7353-type-check-coverage-include-shape",
  "pr": 7921,
  "premise_still_valid": true,
  "summary": "Premise re-verified at origin/main b9f930b: `:681` still computed `excludesTests` from `parsed.exclude` only, and the PM's second reading is CONFIRMED — the package-level aggregate `anyExcludesTests && testsInvoked.length === 0` makes a package with no `exclude` at all invisible at BOTH levels. Also confirmed: all 65 newly-visible files carry zero `@ts-expect-error` directives, so the countRoots comment's hand-off to PINS_CHECKED caught none of them (reported by neither half, as the card says). File surface held to scripts/check-type-check-coverage.mjs alone; no package tsconfig touched. CENSUS (the card's gating unknown, all 78 workspace packages incl. root, at b9f930b): 19 packages/719 files already counted, 3 packages/65 files/198 raw errors newly visible, 56 packages read by an accounted program. My first census pass over-counted this at 16 packages/422 files by equating 'no typecheck script' with 'nothing reads the tests' — wrong, because measureDebt runs `tsc -p tsconfig.json` and every DEBT-territory package already has its tests under that config's include (metadata-protocol's 72 test files live in src/). Corrected to 3, which is why NO staging was needed and no ratchet-later half was left. WHICH PACKAGES FLIP: @objectstack/cli 56 files/188 errors (include:[\"src\"], no exclude at all, 56 tests in a sibling test/ tree), @objectstack/metadata-fs 6 files/6 errors, @objectstack/example-showcase 3 files/4 errors (e2e/ Playwright tree beside include). Nothing else changes status, and all 19 existing TEST_DEBT packages keep both their verdict and their exact file count (old include-roots count and new unread-file count were already identical at 719). FIX: decide TESTS_COVERED per FILE against the programs that account for the package — the configs the typecheck script invokes, or where there is none, tsconfig.json (what measureDebt runs), so a test file that config reads is counted in DEBT rather than hidden; without that second clause 353 files would move into the ledger that means 'src checks, tests are hidden' purely because DEBT already owns them. Per file also makes partial coverage sayable: cli keeps 54 of 110 tests under include and 56 outside, and 56 is the honest number. measureTestDebt generalised to widen include by the unread files ONE AT A TIME (e2e/**/* would have pulled e2e/global-setup.ts in and billed the test layer 6 errors from a non-test file) and to neutralise rootDir, which is `src` in most of these packages: cli measured 56 TS6059 with rootDir kept vs 188 real errors without it, and under --noEmit there is no output layout for rootDir to protect. Generated config is byte-identical to before for all 19 existing entries. Three ledger entries added at their exact measurements, no bootstrap margin, so the first new error in any of these layers goes red immediately. Gate output before: '19 package(s) still exclude their own tests (719 files hidden, 1607 frozen raw errors)'. After: '22 package(s) still hide their own tests from tsc (784 files hidden, 1805 frozen raw errors)'. #7888 CROSS-CHECK (read before writing, per dispatch): my full --re-measure CONFIRMS its reading — 9 entries, 273 raw errors of surplus, identical total and identical entry set; my change moves none of them and the three new entries carry zero surplus, so 273 is the figure before and after. One correction to its table: @objectstack/plugin-auth measures 107, not 106 (131->107 is the -24 its own surplus column and its 273 total already assume; the `measured` cell is the typo, the arithmetic is right). Nothing here fixes #7888. No changeset: scripts/-only gate change, no published package, nothing reader-visible ships — route is the skip-changeset label at review, and no empty-frontmatter changeset was added (#5471/#4898).",
  "tests": {
    "self_test": "PASS — 23 semantic + 24 observation (was 16) + 15 re-measure + 12 built-closure + 9 auto-lowering. Added 8 observation cases: 5 pinning unreadTests (the include-shaped form with no exclude anywhere; partial coverage; the #5286 sibling-config repair; the empty-program case) and 3 pinning accountedPrograms, including the DEBT-territory clause so the 353-file over-reach cannot be reintroduced silently.",
    "check_type_check_coverage": "PASS — pnpm check:type-check-coverage OK.",
    "check_type_check_debt": "PASS — pnpm check:type-check-debt, full --re-measure over 36 entries in 255s, 1987 raw tsc errors, none above its recorded number.",
    "check_nul_bytes": "PASS — 7237 text files scanned.",
    "eslint": "PASS — pnpm exec eslint scripts/check-type-check-coverage.mjs --no-inline-config, clean.",
    "build_closure": "PASS — turbo run build --filter='./packages/*' --filter='./packages/*/*', 70/70 successful (required by --re-measure).",
    "reverse_verification": "DECISIVE. Planted a TS2322 in packages/metadata-fs/test/contract.test.ts — a package #7312 never touched — and ran both detectors against the same tree. (1) `pnpm --filter @objectstack/metadata-fs exec tsc --noEmit` GREEN, exit 0: the package's own advertised typecheck never reads the file. (2) OLD detector (origin/main, run in-tree) reports OK, '19 package(s) ... 719 files', metadata-fs never mentioned — the card's mechanism reproduced on a fresh package. (3) NEW detector structural pass OK, '22 package(s) ... 784 files' (structural pass runs no compiler, by design). (4) NEW detector --re-measure RED, exit 1: '@objectstack/metadata-fs: TEST_DEBT records 6 raw tsc error(s), tsc --noEmit now reports 7 (+1)'. A type error that used to leave BOTH tsc --noEmit and this gate green now has exactly one gate that goes red on it. Planted error reverted before commit; tree clean.",
    "ci": "in_progress — draft PR #7921 just opened, CI not yet converged. PM owns CI convergence and the ready-flip per dispatch."
  },
  "open_questions": [
    "Settled without asking, per the self-drive clause: took the IMMEDIATE ratchet rather than the staged detect-now/ratchet-later option, because the corrected census puts the flip at 3 packages / 65 files and all three measure > 0, so no package could be left in an un-greenable state within the file surface. Staging would have bought nothing.",
    "Settled: scoped the accounted-program fallback to tsconfig.json rather than answering 'hidden' for every test file of every package without a typecheck script. That would have moved 353 files into TEST_DEBT purely because DEBT already owns them, and would have double-counted src errors inside a TEST_DEBT measurement for packages whose src is not clean. Pinned by a self-test case so it cannot be undone by accident.",
    "Settled: @objectstack/example-showcase's hidden files are Playwright e2e specs rather than vitest tests. Kept them in scope — TEST_FILE already matches *.spec.ts repo-wide, and they are TypeScript no tsc reads, which is exactly what #4311 is about. Measured by adding the three spec files individually, not e2e/**/*, so e2e/global-setup.ts (a non-test file, 6 errors) is not billed to the test layer.",
    "Renamed the package-level field excludesTests -> hidesTests (the per-config readTsconfig().excludesTests keeps its name, since that one really is about `exclude`). Cosmetic, but the old name is the mistake this card is about; flagging in case the PM prefers the old name retained for grep continuity."
  ],
  "out_of_scope_findings": [
    "Filed #7923 (finding, domain:devx, unassigned): the 3 newly-ledgered test layers are repair candidates rather than permanent debt — metadata-fs (6 errors: TS6133 x5 + one TS2349 across 2 files) and example-showcase (4 TS2339 across 3 specs) look like same-day graduations via the #5286 route. Includes the rootDir:'src' trap a repairer hits immediately (widening include past it yields one TS6059 per file and nothing else) and a sizing note for cli (159 of its 188 are the TS2835 + TS7006 NodeNext pair, i.e. one repair not 159; five files carry 113 of the total). Searched first — #5475 and #5544 are the same shape for other packages and both closed; no open twin.",
    "Not filed, recorded here only: #7888's table has a typo in the plugin-auth `measured` cell (106 should be 107). Its surplus column and 273 total are correct, so this is a cell correction on an open finding rather than a new card; noted in PR #7921's body."
  ]
}

Generated by Claude Code

ACCEPT — PR #7921. Reviewed against GitHub, not against the report (which never arrived on this thread — see the contract note at the end).

Verified

Diff: 1 file, scripts/check-type-check-coverage.mjs, +273/−72. The declared file surface held exactly — no package tsconfig*.json touched, which was the boundary that mattered here: the card changes what the gate can see, not what the repo declares.
First line Fixes #7353 — read personally. Correct: nothing is deliberately left outstanding, so merging should close the card.
Gate job conclusions read personally, not from an aggregate: ESLint success, TypeScript Type Check success. Check Changeset was the only red and is the expected label route — skip-changeset applied (scripts-only gate change, no published package). ⛔ No empty-frontmatter changeset was added, correctly (空 frontmatter changeset 相对 skip-changeset 标签零收益、单向风险 —— 「禁止空 changeset 进 .changeset/」的决策证据(#5292 结案后无处存放) #5471 / 空 changeset 会静默卡死已 version 的发布:Release run 全绿,但 npm 和 Docker 什么都没发(17.0.0-rc.2 现在就卡着) #4898).
Path surface checked before any landing action: no docs/adr/**, no .claude/skills/**, no skills/**. Normal queue path.

The ruling's hard requirement is met

Triage's promotion said "the PR must report which packages flip status so the ratchet change is deliberate, not incidental." The PR body carries the full table — 3 packages, 65 files, 198 raw errors:

package	files hidden	raw errors	shape
`@objectstack/cli`	56	188	`include: ["src"]`, no `exclude` at all, 56 tests in a sibling `test/` tree
`@objectstack/metadata-fs`	6	6	`include: ["src/*/"]`, all 6 tests in `test/`
`@objectstack/example-showcase`	3	4	`e2e/` Playwright tree unread

And it states what does not move: all 19 existing TEST_DEBT packages keep the same verdict and the same file count (719 both ways). A ratchet change that names its own blast radius in both directions is what the ruling asked for.

The headline is packages/cli: 188 raw tsc errors in a test layer that no gate, no ledger and no CI job had ever read. The card predicted a blind spot; it turned out to be worth more than the card guessed.

What makes this a good run rather than merely a passing one

The dev falsified its own first census. Pass one said 16 packages / 422 files by treating "no typecheck script" as "nothing reads the tests". That is wrong — measureDebt runs tsc -p tsconfig.json, and those packages already have their tests under that config's include. Corrected to 3 / 65. Had it shipped uncorrected, 353 files would have moved into the ledger that means "src checks, tests are hidden" purely because DEBT already owned them. The accountedPrograms second clause exists to make that impossible, and it is pinned by a self-test case naming the 353-file over-reach.
Reverse verification is a deleted-limb test on a fresh package, not a re-run of the card's own example: a TS2322 planted in packages/metadata-fs/test/contract.test.ts left pnpm --filter @objectstack/metadata-fs exec tsc --noEmit green, exit 0 and the old detector reporting OK, while the new detector's --re-measure goes red (records 6 … now reports 7 (+1)). That is the card's mechanism reproduced somewhere it had not been measured, then closed.
Both of my dispatch assumptions were tested rather than assumed. The aggregate reading (anyExcludesTests && testsInvoked.length === 0 making the hole strictly bigger) is confirmed. So is the PINS_CHECKED hand-off claim, with a measurement I did not ask for: all 65 newly-visible files carry zero @ts-expect-error directives, so the hand-off caught none of them — the card's "reported by neither half" is now a number, not an inference.
measureTestDebt learned the same lesson rather than being patched around it: adding hidden files to include one at a time (a e2e/**/* glob would have billed app-showcase 6 errors from global-setup.ts, which is not a test), and neutralising rootDir — measured both ways on cli, 56 TS6059 with it kept vs 188 real errors with it neutralised. Counting the TS6059 would have been measuring the tape measure.

The same-file serial note paid off

The claim comment flagged #7888 as targeting the DEBT ledger in this same file. The dev read it before writing and cross-checked: its --re-measure at b9f930b confirms #7888's reading — 9 entries, 273 raw errors of surplus, identical entry set — and this change moves none of them (the three new entries carry zero surplus, so 273 holds before and after).

It also found one arithmetic error in #7888's table: @objectstack/plugin-auth measures 107, not 106; 131 → 107 is the −24 that #7888's own surplus column and its 273 total already assume, so the measured cell is the typo and the total is right. Recorded here; #7888 is unaffected in substance and remains its own card.

Contract deviation, recorded not waved through

The dev did not post its  comment on this thread — the report's authoritative channel. Its session went IDLE at review_ready with the PR open, so the work completed; only the report write is missing. I reviewed directly from the PR instead, which the state model permits (all state lives on GitHub, and the PR body carried everything the report would have). Noting it because a successor reading this thread would otherwise find a dispatch record with no terminal report and have to reconstruct why.

Flipping ready and arming auto-merge.

Generated by Claude Code

fix(devx): type-check-coverage sees include-shaped hidden test layers (…

fix(devx): graduate metadata-fs and example-showcase out of TEST_DEBT (…

Navigation Menu

[observation] check:type-check-coverage's test-layer count is `exclude`-shaped, so a package that simply omits `test/` from `include` is invisible to it #7353

The shape

Measured

Scope of the blind spot

Why observation-class rather than a defect

Premise re-verified at `origin/main` @ `e3a6f6e` before claiming

Same-file serial note — recorded now so it is not discovered later

Verified

The ruling's hard requirement is met

What makes this a good run rather than merely a passing one

The same-file serial note paid off

Contract deviation, recorded not waved through

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[observation] check:type-check-coverage's test-layer count is exclude-shaped, so a package that simply omits test/ from include is invisible to it #7353

Description

The shape

Measured

Scope of the blind spot

Why observation-class rather than a defect

Activity

claude commented on Aug 10, 2026

claude commented on Aug 10, 2026

os-zhuang commented on Aug 11, 2026

os-zhuang commented on Aug 11, 2026

os-zhuang commented on Aug 12, 2026

Premise re-verified at origin/main @ e3a6f6e before claiming

Same-file serial note — recorded now so it is not discovered later

os-zhuang commented on Aug 12, 2026

os-zhuang commented on Aug 12, 2026

os-zhuang commented on Aug 12, 2026

Verified

The ruling's hard requirement is met

What makes this a good run rather than merely a passing one

The same-file serial note paid off

Contract deviation, recorded not waved through

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

[observation] check:type-check-coverage's test-layer count is `exclude`-shaped, so a package that simply omits `test/` from `include` is invisible to it #7353

Premise re-verified at `origin/main` @ `e3a6f6e` before claiming