Skip to content

docs(checklist-test): a verification re-run files no qa-run record; sweeps still file one per run - #24172

Merged
objectstack-fleet[bot] merged 1 commit into
mainfrom
claude/issue-24127-qa-run-record-on-problem
Oct 17, 2026
Merged

objectstack-fleet[bot] merged 1 commit into
mainfrom
claude/issue-24127-qa-run-record-on-problem

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #24127
Clause-②: no

Ruling-ref: 6230593290 (the maintainer's 「C 有问题才开卡 (Recommended)」, adopting the director's 口径 6229123659 items 1–4). Tier S governed surface (.claude/**): this PR stays draft until its contract-review record.

What this changes

One file, .claude/skills/checklist-test/SKILL.md, in the three places the ruling names:

  • §4 (结果 issue). The opening sentence now names the whole-round run (release candidate, area or all, maintainer-named target). It still files exactly one qa-run record per run, pass or fail alike, and it names §5's verification re-run as the one run that files none. This is ruling item 2; RUNNER.md's title grammar is unchanged.
  • §5 (收口, beside the three 重测触发). Defines 「验证重测」 once: a run whose trigger is the same PR's ledger edit of that item (the edit §3 describes). It is not a fourth trigger and it files no qa-run record. The per-item table goes in the PR body, and the item's history row (ref) names the PR. A partial / blocked gap goes in the PR body and in the item's fixtures.knownGaps. A product-defect fail is extracted as today, its source line naming the PR. A stale-clause fail is fixed in the same diff. This is ruling item 1 with its sub-point.
  • Guardrails. 「一个选择器、一次运行、一张 issue」 gains its complement 「验证重测不立 issue,证据进 PR 正文(§5)」.
  • §5's health-signal sentence 「开着的 qa-run 数 = 未清的收口债」 is unchanged. It becomes true again by construction.

No gate, label or script is added.

Reader check (ruling item 3, the card's confidence gap)

Measured at bd4276a91 with git grep -n qa-run, reading every non-markdown hit:

  • scripts/qa/qa-rollup.mjs fetches GET /issues?labels=qa-run&state=all (L762) and parses the canonical TITLE only (L269) into an area x latest-verdict x staleness matrix. It has no re-run category. A verification re-run that files no record is simply absent from it. Consequence, by design of the ruling: the roll-up's per-selector latest verdict is not refreshed by a verification re-run; that evidence lives in the item's history row and the PR body.
  • scripts/pm/check-half-states.mjs L3608 H13_EXEMPT_LABELS only exempts qa-run from the domain-without-pm-state flag. It counts nothing.
  • .claude/skills/pm-dispatch/references/triage-duties.md L16 excludes qa-run from the sweep.
  • .claude/skills/pm-dispatch/references/lanes/skills.md L11 says a qa-run record is a protocol carrier, not work. It counts nothing.
  • The remaining code hits are qa-runner, a spec/core family name, not the label.

No reader counts verification re-runs. The dispatch's RUNNER.md line numbers were stale: the record and label discipline sit at L460–L461 and L555 at bd4276a91.

Line budget

  • .claude/skills/checklist-test/SKILL.md: 238 → 238 lines (CEILINGS 238, headroom 0). Widest table row 221 → 221 (pin 221). No CEILINGS row moved, so no ruled raise is taken.
  • The ruled text costs +4 lines (the §5 block, 3 lines plus a blank line). §4's opening and the Guardrails bullet carry their additions at +0.
  • Paid by deleting content in the same sections, not by re-wrapping:
    • §4 label bullet: the drift narrative 「实测漂移形态:记录事后被补上 tests / tracking / bug,每一个都把协议载体推回排除清单本要挡住的 sweep」. Its two extra labels move into the ⛔ list. Its reason is already the bullet's own 「run 记录是协议载体、不入分诊 sweep」. −3 lines.
    • §4 title bullet: the restated fraction code span, now 「分数强制」 (the fraction stays visible in the title template). −1 line.
    • §4 opening: 「JSON 不进,截图不进」, already said by 「运行产物一概不进仓库树」 and the next paragraph. This makes room for the sweep enumeration at 3 → 3 lines.
  • Every new line is within 120 bytes (largest 119 B).

Verification (HEAD 296f877d0)

Local subset, derived by node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack after the final commit (16 families), plus the two the dispatch named. Exit codes captured before any pipe:

node scripts/check-closing-keyword-parity.mjs :: exit 0
node scripts/check-closing-keyword-parity.mjs --self-test :: exit 0
node scripts/check-comment-mask-corpus.mjs :: exit 0
node scripts/pm/check-harness-current.mjs --self-test :: exit 0
pnpm --filter @objectstack/lint run check:doc-formula-expressions :: exit 0
pnpm check:agent-test-spelling :: exit 0
pnpm check:cross-package-test-inputs :: exit 0
pnpm check:doc-authoring :: exit 0
pnpm check:driver-memory-census :: exit 0
pnpm check:gitlink-declared :: exit 0
pnpm check:nul-bytes :: exit 0
pnpm check:pm-governed-merges :: exit 0
pnpm check:pm-skill-ratchet :: exit 0
pnpm check:refd-timer-probe :: exit 0
pnpm check:skill-frame-sync :: exit 0
pnpm check:watch-hint-literal :: exit 0
pnpm check:pm-skill-id-lint :: exit 0
pnpm check:pm-governed-prose :: exit 0
  • check:doc-formula-expressions first exited 3 (PREREQUISITE NOT MET: @objectstack/formula and @objectstack/lint were unbuilt). It was re-run after turbo run build for those two packages and exited 0. The exit 3 measured nothing.
  • check:pm-skill-ratchet: 「.claude/skills/checklist-test/SKILL.md is 238 lines (ceiling 238; headroom 0)」 and 「widest table row is 221 bytes (pin 221; headroom 0)」.
  • --ran reconciliation: 「16 derived famil(ies) accounted for — 16 run, 0 NOT-MEASURED (a DERIVED zero — all 16 recorded an exit code and none of them is 3)」.
  • No ablation: the diff changes no tested behaviour. skip-changeset: .claude/** publishes nothing.

Acceptance notes

  • Out of scope, reported and not edited (domain:devx text): three docs still say every completed run files a record. docs/qa/platform-checklist/RUNNER.md L460–L461 says "Every completed run — pass or fail alike — files one GitHub issue as its durable record". docs/qa/platform-checklist/runs/README.md L11 says "every completed run files one qa-run GitHub issue". docs/qa/platform-checklist/README.md L466–L467 says "every run files one qa-run GitHub issue as its record". Each needs the verification re-run exception.

维护者速读(草稿)

  • 改了什么: 只改 checklist-test 技能一个文件。整轮测试(发版候选、整区或全量、你点名的目标)仍然每轮开一张 qa-run 记录。为验证清单修订而做的重测(验证重测)不再开记录:逐项结果写进 PR 正文,条目的 history 行指向这个 PR;没测全的缺口写进 PR 正文和条目的 knownGaps;真缺陷照常开缺陷卡。
  • 为什么改: 你裁决选了 C「有问题才开卡」。之前每次验证重测都开一张记录,一天开出几十张无用的卡,「开着的 qa-run 数」也就不再代表真实的收口债。
  • 风险与代价(含回滚): 读 qa-run 的工具(qa-rollup、分诊排除、半态检查)都已核过,没有一个统计验证重测,所以不受影响。代价是 qa-rollup 的「最新判定」不再反映验证重测,这部分证据在条目 history 和 PR 里。文件行数不变(238/238),不加门禁、标签或脚本。回滚就是还原这一个提交。RUNNER.md 等三份 devx 文档还写着「每次运行都开记录」,已作为范围外发现上报,本 PR 不改。
  • 席位意见:
  • 你要做的: 无需操作。这是 Tier S 改动,由技能席位做合同级复核后落地。

Generated by Claude Code

…weeps still file one per run

A run whose trigger is a ledger edit the same PR makes is a verification
re-run, not a fourth retest trigger: it files no `qa-run` record whatever
its outcome. Its per-item table goes in the PR body and the item's
`history` row names the PR; a `partial` / `blocked` gap goes in the PR body
and the item's `fixtures.knownGaps`; a product-defect `fail` is extracted
as today with its source line naming the PR, and a stale-clause `fail` is
fixed in the same diff. Sweeps (release candidate, area or `all`, a
maintainer-named target) still file one record per run.

Line-neutral (238/238): paid by deleting the label bullet's drift
narrative (its labels kept in the prohibition list), the title bullet's
restated fraction, and the opening paragraph's restated artefact list.

Claude-Session: https://claude.ai/code/session_012PxNHU3dfov2hbpg9ycZ8q
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 296f877d02ee62a68c87aa8ffe94380ddddbd65e
Local-runs: none
Scope: .claude/skills/checklist-test/SKILL.md + docs/qa/platform-checklist/README.md (the item shape: history[].ref, fixtures.knownGaps), docs/qa/platform-checklist/areas/ai.json (a live knownGaps row), scripts/qa/qa-rollup.mjs, scripts/pm/check-half-states.mjs, .claude/skills/pm-dispatch/references/triage-duties.md (the qa-run readers the hunk's ruling-item-3 claim is about), the ruling 6230593290 and the director's 口径 6229123659

Read 2026-10-17T13:33Z in seat by session_012PxNHU3dfov2hbpg9ycZ8q (marchtian, domain:skills#1), whose served model is the CONTRACT_REVIEW_TIER value (get_session last_served_model). Inputs: card #24127 (body + 5 comments: 6229123659, 6230593290, 6231437747, 6234201532, 6234257518, and the report 6234454492), PR #24172 (body, 1-file list, diff +15/−15 against base bd4276a91), the head's check-runs at this write, and origin/main bd4276a91. Read-only: nothing built, run or re-run; the dev's local gate runs are its evidence, not repeated.

① Derived judgments

The accept-set change: governed rule text in one skill, no code, three places. Each judged against the ruling's letter (6230593290 items 1–3, which adopt 6229123659 items 1–4):

  • §4 title and opening. 「一次整轮运行一张 GitHub issue」; the opening now enumerates the whole-round runs — 「发版候选、区或 all、维护者点名目标」 — files one record each, 「通过与失败同等」, and names §5's 验证重测 as the run that files none. Right: ruling item 2 verbatim in substance (release candidate, area or all, maintainer-named target). The dropped 「JSON 不进,截图不进」 survives as 「运行产物一概不进仓库树」 in the same sentence; lossless.
  • §4 title bullet. 「唯一形状、严格解析、分数强制」 replaces the restated (<已判定>/<总数>) span; the template line immediately above still shows the fraction, and RUNNER.md 「The record title」 stays the grammar's home. Right; lossless.
  • §4 label bullet. The incident narrative 「实测漂移形态:记录事后被补上 tests / tracking / bug…」 is deleted and its two labels (tests, tracking) move into the ⛔ list beside bug/priority; the lifecycle clause stays. Right: AGENTS.md's rules-only rule (「no incident narrative」) is served, and the prohibition is wider than before, not narrower.
  • §5 new block (the ruling's item 1 with its sub-point). 「验证重测不是第四条触发:由同一 PR 对该项的清单编辑(§3)引发,⛔ 不立 qa-run 记录。逐项判定表进 PR 正文,该项 history 行的 ref 指向该 PR;partial/blocked 的缺口进 PR 正文与 fixtures.knownGaps;产品缺陷 fail 照常抽卡、溯源指向该 PR;条款过期在同一 diff 修。」 Clause by clause: trigger = the same PR's ledger edit (6230593290 item 1 「a run whose trigger is a ledger edit the same PR makes」) — right, and 「(§3)」 points at the section that defines that edit (L110–117, 「那是一次清单编辑… 追加一条 history」) — right; 「不是第四条触发」 — right, the ruling keeps the three triggers and the ⛔ line above; no record on all-pass — right; per-item table in the PR body and the item's history row names the PR — right, and the key path holds: README.md L79 declares "history": [ { "revision", "date", "change", "ref": "#PR" } ]; partial/blocked → PR body + fixtures.knownGaps — right, the sub-point ruled with the letter; the path fixtures.knownGaps is the ledger's own (areas/ai.json L22 carries a knownGaps list under the item's fixtures block; FOLLOW-UPS.md §8d/§9g name 「clauses recorded as knownGaps」); product-defect fail extracted as today with its provenance line naming the PR — right (the ruling's 「a defect card, as today」; §5's QA-source: line gains the PR as the source where no record exists); stale-clause fail fixed in the same diff — right.
  • Guardrails. 「一个选择器、一次运行、一张 issue;验证重测不立 issue,证据进 PR 正文(§5)。」 — the complement, pointing back to the one definition. Right.
  • Item 3 (the confidence gap) answered in the PR body and re-read by the seat: scripts/qa/qa-rollup.mjs fetches labels=qa-run and parses the canonical title only (header L5–L53, parseRunTitle L291); it has no re-run category, so a run that files nothing is absent, not miscounted. check-half-states.mjs L3608 only exempts the label (H13_EXEMPT_LABELS). triage-duties.md L16 excludes the label from the sweep. No reader counts verification re-runs. The dev's one consequence — the roll-up's latest verdict is not refreshed by a verification re-run — is the ruling's own design (item 3: 「qa-rollup keeps reading records only」), stated, not hidden.
  • The health-signal sentence (L165 「开着的 qa-run 数 = 未清的收口债」) is untouched and becomes true again by construction. Right.

Ratchet. 238 → 238 lines (CEILINGS 238, headroom 0, no raise taken, so no ruling id is owed in the body for a raise); the widest table row 221/221 untouched; the seat's own read of the head file: 238 lines, longest line 221 bytes (a table row), the §5 block's lines under 120 bytes. The ratchet script is not in the diff. Paid by content deletion inside §4 (the narrative, the restated span, the restated artefact clause), not by re-wrapping: each deleted clause's survivor is named above.

Main drift. Base bd4276a91 is origin/main's tip at this write; checklist-test/SKILL.md's last touch on main is 57cfedd24 (2026-09-19); git merge-tree --write-tree origin/main FETCH_HEAD exits 0.

Value density (a skill loaded by every checklist runner): +4 lines of rule that remove a write per verification re-run and restore the one health signal §5 defines; the deletions remove a narrative and two restatements. Right.

Check-runs on 296f877d0 at this write: 26 success · 12 skipped · 0 in_progress · 0 other. Gate-carrying lanes still running are read at landing, not here; this record judges the diff.

② Semver level

.claude/** publishes nothing: no changeset owed; skip-changeset present on the PR (the dev's label-write, read back). The PR body's line 2 is Clause-②: no at line start, copied from the claim (6234201532): governed rule text, no published accept set or public surface touched. Matched and well-formed. No model identifier in the title, body (session-URL footer), commit (model-free trailer pair) or comments.

③ Boundary flags

Dev report 6234454492: open_questions: []; two out_of_scope_findings and six deviations:

  • OOS-1 (class b, named producer): three domain:devx docs still state the pre-ruling 「every completed run files a record」 — RUNNER.md L460–461, runs/README.md L11, README.md L466–467 at bd4276a91; the seat re-read all three lines. Filed by the seat as platform-checklist docs: RUNNER.md, README.md and runs/README.md still say every completed run files a qa-run issue; ruling C on the checklist-test skill exempts verification re-runs (PR #24172) — add the exception in the three devx docs #24173 (domain:devx, documentation), with the three lines quoted and a re-check per line; the dev's 「carrier: devx seat · noted, not filed」 was the right posture for a dev and is now a card under the filing gate's 「已落地工作欠的后续照立卡」.
  • D1 base 697301491 → bd4276a91 under the order; the touched files are byte-identical between them. Answered.
  • D2 RUNNER.md line numbers in the order (L425/L519) were stale; the reader conclusion holds at L460–461 / L555. Answered; the order's citation error, not the dev's.
  • D3 the §4 deletions touch the title and label bullets beyond the opening sentence — inside §4, which the order names, and declared. Accepted.
  • D4 model-free trailer pair kept against the harness reminder. Right.
  • D5 check:doc-formula-expressions exit 3 → built under the verify lock → exit 0. Answered.
  • D6 worktree removed after push confirmed. Right.

Reviewer's own flags: none. mcp_calls 0; api_writes three relay strokes (pr_create with assignee, labels_add, the report) plus two git push; no PATCH/PUT/DELETE.

Implemented-by: claude/issue-24127-qa-run-record-on-problem
Reviewed-by: session_012PxNHU3dfov2hbpg9ycZ8q

VERDICT: PASS

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

维护者速读

改了什么:checklist-test 技能一个文件,三处:§4 把「每次运行一张 qa-run issue」收窄到整轮运行(发版候选、整区或 all、你点名的目标);§5 在三条重测触发旁定义「验证重测」(由同一 PR 的清单编辑引发):不开记录,逐项结果进 PR 正文、条目 history 行指向该 PR,缺口进 PR 正文与 fixtures.knownGaps,真缺陷照常抽卡;Guardrails 加上对应的一句。238/238 行不变,不加门禁、标签或脚本。

为什么改:你在 #24127 裁了 C「有问题才开卡」(6230593290)。此前每次验证重测都开一张记录,一天几十张,「开着的 qa-run 数」失去健康指标的意义。

风险与代价(含回滚):读 qa-run 的三个读者(qa-rollup、半态巡查、分诊排除)都核过,没有一个统计验证重测;代价是 qa-rollup 的「最新判定」不再被验证重测刷新,证据在条目 history 与 PR 正文 —— 这是裁决第 3 项的设计。三份 devx 文档(RUNNER.md、README.md、runs/README.md)仍写旧规则,已立 #24173 给 devx 席。回滚 = revert 这一个 commit。

席位意见:ACCEPT。逐句对照裁决三项核过(见上方 ## Contract review PASS),头 296f877d0 38 个 check 26 绿 12 预期跳过 0 红。Tier S,席位落地。

你要做的:无。

席位核对

Seat domain:skills#1 · session_012PxNHU3dfov2hbpg9ycZ8q (marchtian) · read 2026-10-17T13:35Z · head 296f877d02ee62a68c87aa8ffe94380ddddbd65e · verdict: ACCEPT (Tier S — the seat lands it after the two pre-landing checks)

Implemented-by: claude/issue-24127-qa-run-record-on-problem
Reviewed-by: session_012PxNHU3dfov2hbpg9ycZ8q

  1. Shape. Draft PR against main; body line 1 Fixes #24127, line 2 Clause-②: no (from claim 6234201532); Ruling-ref: 6230593290 on line 4. The only closing keyword is the Fixes line; no other open card number sits beside a closing verb (the body names [Decision] checklist-test files one open qa-run issue per re-run: 40 of 42 open qa-run issues are re-runs that verified ledger edits. Keep, one record per batch, or file only on a problem? #24127 only). File list: .claude/skills/checklist-test/SKILL.md (+15/−15). Base bd4276a91 = origin/main tip; merge-tree exit 0.
  2. Ruling and acceptance, read by the seat. Items 1–3 of 6230593290 are each present once (§5 block, §4 opening, Guardrails) and the reader check for item 3 holds (contract review above). The ratchet: 238/238 lines, widest row 221/221, no CEILINGS move; paid by content deletion with each survivor named. RUNNER.md untouched (item 2).
  3. Gates — check-runs pinned to head 296f877d0. 38 completed: 26 success (Lint & Repo Gates, TypeScript Type Check, Test Core, Governed Surface Queue Guard, Check Changeset, every Lint · / Type Check · lane, the three claim checks, Dogfood Regression Gate), 12 skipped — roster verdict: VERDICT check-expected-skips: OK — 12 skipped check-run(s), every one in the roster (exit 0) (exit 0) — 0 failed. check-governed-merges.mjs --pr 24172: GOVERNED, Tier S, .claude/** ×1, +15/−15 under the threshold. PR object at this read: draft=true mergeable=true state=clean.
  4. Changeset. None owed (.claude/** unpublished); skip-changeset present, Check Changeset success; the seat concurs.
  5. Report hygiene. mcp_calls 0; api_writes three relay strokes + two pushes; deviations D1–D6 answered in the contract review; D2 was the order's stale line numbers, corrected there.
  6. Out of scope. OOS-1 filed as platform-checklist docs: RUNNER.md, README.md and runs/README.md still say every completed run files a qa-run issue; ruling C on the checklist-test skill exempts verification re-runs (PR #24172) — add the exception in the three devx docs #24173 (domain:devx); nothing else.
  7. Landing (Tier S). Pre-landing ①: PASS record 6234499645 on this head. ②: every check green or an expected skip. Next act: relay pr_ready then automerge_enable; the probe-table landing comment follows when the queue ref is gone and origin/main carries the commit. subscribe_pr_activity live on this PR.

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 17, 2026 13:36
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 17, 2026 13:36
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 17, 2026
Merged via the queue into main with commit b0fe9fa Oct 17, 2026
39 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-24127-qa-run-record-on-problem branch October 17, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants