Skip to content

Read classic required checks for a token without admin rights - #186

Merged
mchwang merged 1 commit into
mainfrom
fix/classic-checks-read-access
Oct 21, 2026
Merged

mchwang merged 1 commit into
mainfrom
fix/classic-checks-read-access

Conversation

@mchwang

@mchwang mchwang commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

A token without admin rights got 404 "Not Found" from branches/<b>/protection on every protected branch checked. So GhMergeGateway#rules (github/merge.ts) read its merge rules as unknown and blocked every merge. The branch read (branches/<b>) shows such a token protection.enabled and protection.required_status_checks (enforcement_level, contexts, checks with app_id).

On a "Not Found" answer, #rules now uses the branch read:

  • enabled: false with enforcement off and empty lists: no classic protection.
  • enabled: true: the listed checks are required (enforcement off means none). They are held to the protection read's rules: both lists present and naming the same checks, every check with an app_id.
  • The branch read has no strict, so it never gives the atomic base guard. For such a token, a branch that only classic strict guards stays blocked by base-guard.
  • Every other answer still fails closed. Two answers that were read as no classic protection now fail closed too: classic protection off with enforced or listed checks, and classic protection off without a readable required-checks object.

The admin path (protection read answers 200) is unchanged.

Evidence

Recorded with scripts/record-merge-gh.ts and gh 2.102.0 on 2026-10-20, from public repositories this token may only read:

  • nonadmin-ruleset: vitejs/vite PR 23837 into main (ruleset only).
  • nonadmin-classic: cli/cli PR 14716 into trunk (classic checks only).
  • nonadmin-mixed: sveltejs/svelte PR 19050 into main (both require one check).

With main's code, all three read as rulesKnown: false. With this change they read the correct required checks (test/merge-recorded.test.ts). The vite and svelte checks are bound to an app, so they read as missing against gh's app-less rollup: #176, unchanged and fail-closed.

Other shapes seen live without admin rights, used by the hand-written fakes: withastro/astro (enabled: true, off), biomejs/biome (non_admins, no checks), python/cpython (everyone).

Not checked:

  • That the branch read's checks match the protection read's for one branch (no recording has both).
  • A token with write access but no admin rights. This is the access codeboost needs to merge. If such a token gets another answer (for example 403), merging stays blocked.
  • Fine-grained or GitHub App tokens.
  • Private repositories read without admin rights.
  • GitHub Enterprise Server.

Validation (head 12b8cee, base aded16e)

  • Focused: test/merge.test.ts and test/merge-recorded.test.ts, 229 passed.
  • Typecheck: tsc --noEmit clean.
  • Design system and plan format #1 rule: with the github/merge.ts change reverted, 9 tests fail. A mutation that takes strict from the branch read fails the base-guard test.
  • Full non-Docker suite, run locally on 4b9f051. That head has the same github/merge.ts as this one; only tests and docs changed after it. Result: 2452 passed, 4 failed, 1 skipped.
    • All 4 failures are 15 s timeouts in test/pre-merge.test.ts, which this change does not touch. The load average was about 121.
    • Rerun alone, a different test in that file failed (a timing assertion). I record these as machine-load failures, not product failures.
  • Docker suites and real-docker: not run locally (another session owns local Docker). CI is the gate.

Review record

Independent-agent reviews ran in fresh subagents on the full diff against aded16e:

Round Head Findings Changes
1 3529b69 enabled: false never checked the branch read's required checks (contradictory answer read as known); overclaims ("every protected branch", "any reader"); a fine-grained token was claimed to get "Not Found"; misleading test title; docstring overclaim Classic-off now needs enforcement off and empty lists; wording fixed; test renamed
2 7efe327 Doc scope (protected branches only); unverifiable private-repo evidence; commit message missed a case; no row for enforced with empty lists while off Fixed; evidence clause removed
3 0dc0d68 Two missing rows (enforced and empty while enabled; "Not Found" with no object) Rows added (biome evidence)
4 6a49466 No 403 row; commit message case count; stale "no recording has an app-pinned check" Fixed
5 20a44bb README still required reading classic protection; no evidence note on the "Branch not found" fake; misleading row labels Fixed
6 4b9f051 Guard test could not catch strict read from the branch; misplaced evidence comment Row with branch strict: true added; mutation confirmed
7 602dede "automatic merge" could be read as GitHub auto-merge; note on the fakes overclaimed what is read Wording fixed
8 9e29263 Write-but-not-admin access not checked, but claims were unqualified Added to "Not checked"; claims narrowed
9 12b8cee No findings —

Copilot review

Requested on 12b8cee (base aded16e), authorized by the clean independent round 9 on that pair. Main moved to a56bbdc (#185) after round 9; it touches none of this PR's files, and the PR's merge base is still aded16e.

  • Copilot round 1 (review 5592342474, commit 12b8cee): no open issues. Its summary raised two concerns:

Review lessons audit

  • The contradictory enabled: false answer (round 1), the missing rows (rounds 2–4, 6) and the 403 row: covered by "Treat every behavioural claim… give it a test that fails if the claim is false" (CLAUDE.md Review readiness) and the Design system and plan format #1 rule.
  • Overclaims and unchecked token types (rounds 1, 2, 7, 8): covered by "Treat every behavioural claim… as something to verify".
  • Fake evidence notes (rounds 5, 6): covered by the fakes rule ("name the version checked beside the fake").
  • Stale doc sentences (rounds 4, 5): covered by the same claim rule.
  • No new rule is needed.

🤖 Generated with Claude Code

A token without admin rights got 404 "Not Found" from
branches/<b>/protection on every protected branch checked, so its
merge rules read as unknown and merging was blocked. The branch read
(branches/<b>) showed such a token protection.enabled and
protection.required_status_checks (enforcement_level, contexts,
checks) on the public repositories checked. Use it on a "Not Found"
answer: enabled false with enforcement off and no checks is no
classic protection; enabled true takes its checks, held to the
protection read's rules. It has no `strict`, so it never gives the
atomic base guard. Every other answer still fails closed. Classic
protection off whose required-checks object is missing or
contradicts itself (enforced, or listing checks) was read as no
classic protection and now fails closed too.

Recorded with gh 2.102.0 from public repositories read without admin
rights: vitejs/vite (ruleset only), cli/cli (classic only) and
sveltejs/svelte (both).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Exact-head full-suite and CI validation remain unconfirmed, and write-capable non-admin token behavior is unverified for this merge-safety change.

No open issues 🎉

What changed in this PR

Adds a branch-read fallback for classic required checks when GitHub returns 404 “Not Found” to a non-admin token, while preserving fail-closed merge safeguards.

Changes:

  • Validates fallback check lists without granting an atomic base guard.
  • Adds recorded responses and regression tests for valid, malformed, and contradictory responses.
  • Documents supported observations and unverified token scenarios; app-pinned check handling remains tracked in #176.
File Description
test/​merge.test.ts Tests fallback validation and base-guard preservation.
test/​merge-recorded.test.ts Replays non-admin rule and check responses.
test/​fixtures/​gh/​nonadmin-ruleset.json Records a ruleset-only branch.
test/​fixtures/​gh/​nonadmin-mixed.json Records combined classic and ruleset checks.
test/​fixtures/​gh/​nonadmin-classic.json Records classic required checks.
README.md Clarifies access requirements and fallback limitations.
github/​merge.ts Implements validated classic-check fallback.
docs/​implementation/​guarded-merge.md Documents fallback behavior and evidence limits.
docs/​designs/​codeboost-plan-indexed-review.md Updates recorded-test evidence.
docs/​architecture.md Summarizes non-admin check support.

Review effort: Balanced

@mchwang
mchwang merged commit aced300 into main Oct 21, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants