Repository navigation
ci: prune each Test Core Turbo seed to its own run's tasks before saving it - #25523
Merged
Merged
Conversation
…ing it The Test Core restore takes the newest seed by key prefix and turbo never evicts its local cache, so every saved seed was the previous seed plus one more run, and the bare `<os>-turbo-test-` fallback carried that lineage from shard to shard. On push run 39982654692 each of the six shard seeds was 2.41-2.44 GB (Build Core's full 76-task build: 0.048 GB), about 14.6 GB per merge into the repository's 10 GB Actions cache pool. scripts/ci/prune-turbo-cache.mjs keeps only the cache entries named by the test step's run summary and the closure build's summary, drops the rest, and reports save=false when the test step left no summary (a cancelled run or an empty shard) so such a run no longer saves a seed. The save step reads that output. The self-test runs once in Build Core. Claude-Session: https://claude.ai/code/session_01SawdJgiuoz8deqVdM51jAE Co-authored-by: Claude <noreply@anthropic.com>
This was referenced Oct 23, 2026
os-zhuang
approved these changes
Oct 24, 2026
os-zhuang
marked this pull request as ready for review
October 24, 2026 00:18
This was referenced Oct 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Elon · project thread
Fixes #25452
Clause-②: no
Metadata-protocol: none
Before each Test Core shard saves its Turbo seed on a push to
main, the seed is now pruned to the cache entries its own run used. The Test Core seeds were not 2.4 GB of useful cache. They were one seed lineage that grew by one run per merge and was never trimmed. Pruning fixes this at the source, so the card's change (b) is the only change here. Change (a), dropping the Build Core / Type Check seeds, and change (c), the Next.js cache, are left alone: the measurement below shows those seeds are 0.05 GB each, not where the pool goes.Step 1: the measurement (save-step and restore-step logs; the pool API is refused from the cloud container)
Bytes saved per merge (push run 39982654692,
mainat05cc3ee953, a completed run; read from each save step'sSent N of N (100.0%)line):Linux-turbo-test-<1..6>-main-Linux-turbo-build-core-main-Linux-turbo-dogfood-<1..3>-main-Linux-turbo-typecheck-main-(lint.yml run 39982654477)Each merge writes about 1.5× the repository's 10 GB default pool. The jobs API shows every one of the six Test Core save steps succeeding on five more push runs from 16:23Z to 17:56Z (39986349408, 39989699728, 39994571067, 39996442855, 39999263178), including the cancelled runs.
Eviction, seen from a PR restore. On PR run 39999049930 (17:55Z),
Test Core (1/6)(job 120570291429) restoredLinux-turbo-test-3-main-fa06169c…(2.50 GB). That is shard 3's seed, reached through the bareLinux-turbo-test-restore key. The push at 17:36Z (run 39996442855) had saved a shard-1 seed, but noLinux-turbo-test-1-*entry was left 19 minutes later.Why 2.4 GB: the seed snowballs. A full 76-task workspace build is 0.048 GB in Build Core's seed, and a shard's closure is a subset of that build. The restore step takes the newest seed by prefix, turbo never evicts its local cache, and the save step stored the whole directory. So every seed was the previous seed plus one run, and the bare-prefix fallback carried the lineage from shard to shard. That is why all six shards are the same size. The sizes keep growing: shard 3 was 2.441 GB at
05cc3ee9(16:20Z) and 2.50 GB atfa06169c(17:36Z). The same growth reproduces on a two-package turbo 2.11.5 workspace: one input change took.turbo/cachefrom 9 files to 18, and pruning to the run summary took it back to 9 with a full3 cached, 3 totalreplay.Pool total and largest entries: pending.
GET /actions/cache/usageand/actions/cachesanswer 403 from a cloud container ("Access to this GitHub Actions path is not permitted through this proxy"). The devx seat has asked the maintainer for the two reads, and they will be added here when they arrive.What changed
scripts/ci/prune-turbo-cache.mjs(new). It reads every run summary (--summarize) in--run-summaries, plus any--extra-summaries, and collects each task'shash. It then deletes every<hash>.tar.zst/<hash>-meta.json/<hash>-manifest.jsonin.turbo/cachewhose hash no summary names, and appendssave=true|falsetoGITHUB_OUTPUT.save=false. That covers a cancelled test step or an empty shard. An extra summary alone (the closure build's) does not count, because saving builds without test results would make the newest seed one that replays no tests.--self-testcovers 25 cases across 5 batteries with a pinned battery floor and the verdict handshake.ci.yml, Test Core: a new step,Prune the Turbo cache to this run's tasks (main only)(if: always() && github.event_name == 'push'), runs beforeSave Turbo cache (main only). It reads.turbo/runs(the test step's summary) plus$RUNNER_TEMP/turbo-remote-cache(the closure build's summary, moved there by the remote-cache assertion). The save step'sif:gains&& steps.prune-turbo-cache.outputs.save == 'true'.ci.yml, Build Core: a newprune-turbo-cache self-teststep next toassert-turbo-remote-cache self-test, and the script is added to thecorefilter's guard rows (check-ci-filter-parityandcheck-self-test-wiredboth require it).Kept on purpose: the bare
Linux-turbo-test-restore key. Once seeds are pruned, a cross-shard restore brings at most another shard's pruned seed, about one closure's worth. The shared build entries still replay, and the prune removes the rest before the next save, so the fallback can no longer carry a lineage forward. Removing it would also take away the warm start its comment describes for a new shard number.Expected after
Each Test Core seed should hold its shard's closure builds plus its test results, which is well under Build Core's 0.048 GB full build plus test logs. A merge would then write about 0.3–0.6 GB in total instead of about 14.85 GB. A side effect: PR shards stop downloading about 2.4 GB in
Restore Turbo cache, which measured 15–46 s per shard on the PR runs above. The first push after landing still restores the old snowball once, prunes it, and saves the small seed. The old 2.4 GB entries then age out of the pool.Acceptance (from the card)
gh apireads arrive.mainpost-landing: the Test Core save-step sizes, the pool total (target ≤ 8 GB at steady state), and two readings an hour apart. Over the next 10 PR runs whose closure matchesmain: the median of the shard "Build this shard's dependency closure" step ≤ 1.7 min and Build Core's build step ≤ 2.5 min. Theassert-turbo-remote-cachehit counts will be recorded before and after.Risks and rollback
maindepends on this seed.test/test:repoare off the remote cache (Build Core's PINS), so a pushed shard replays unchanged packages' tests only from the Actions seed. The prune keeps every hash the test step's summary names, which covers every test task and every build it depended on, so the replay set is unchanged. A test task that failed has no entry either way.Local verification (head fcae6e3)
node scripts/ci/prune-turbo-cache.mjs --self-test: OK, 25 cases across 5 batteries. Firing proof: with the keep rule ablated to keep everything (a temporary copy of the script), the self-test exits 1 with 5 failures (every file of an unnamed hash is dropped,an inherited entry no summary names is deleted, …). The copy was deleted afterwards.dist/and the summaries, re-run, and the result isCached: 3 cached, 3 totalwith outputs restored.check-self-test-wired(+--self-test),check-ci-filter-parity(+--self-test),check-step-collectors(+--self-test),check-self-test-workflow-commands,check-published-list-mirrors,check-declaration-mirrors,check-platform-checklist-watchdog,check-empty-changeset --base origin/main,check-adr-0087-registration --base origin/main,check-aggregator-roster,check-closing-keyword-parity,check-comment-mask-corpus,check-shard-attestation --self-test, andpnpm check:workflow-status-functions,workflow-step-name-quoting,nul-bytes,merged-result,pm-governed-merges,pm-expected-skips,single-claim-paths,pnpm-filter-targets,pnpm-flag-spellings,scripts-symbol-anchors,runner-env-posture,node-version,pnpm-acquisition,select-shard-packages,shard-attestation,required-contexts,stall-guard-budget,stall-guard-headroom,issue-citations,optional-error-sink,doc-authoring. The rest of the farm is left to CI.ci.ymlparses (yaml); eslint is clean on the new script.check-governed-merges.mjs --test .github/workflows/ci.yml scripts/ci/prune-turbo-cache.mjs: GOVERNED, landing tier H (.github/workflows/**×1), 331 changed lines.skip-changeset: no published package is touched (.github/workflows/**,scripts/ci/**).Acceptance notes
claude/project-thread-53z7nt, the name the session was given. The claim (comment 6371859957) names it.维护者速读(草稿)
改了什么:main 每次合并后,Test Core 的 6 个分片会各存一份 Turbo 缓存种子。现在存之前先把种子裁剪到「这次运行真正用到的任务」,其余旧条目删掉。如果测试步骤没有跑完(被取消或空分片),这次就不存。
为什么改:实测每份种子 2.4 GB,每次合并共写约 14.6 GB,而仓库的 Actions 缓存池只有 10 GB,结果是每合并一次,上一次的种子就被挤掉。原因是种子在滚雪球:每次都在恢复的旧种子上再加一轮,从不清理。完整构建一次的产物其实只有 0.05 GB。裁剪后预计每次合并只写 0.3–0.6 GB,PR 运行也不用再下载 2.4 GB 的种子。
风险与代价(含回滚):main 上测试结果的复用只靠这份种子,所以本次运行用到的条目(含全部测试结果)一律保留,复用范围不变。没跑完的运行不存种子,旧种子继续可用。回滚只需 revert,不动任何包和恢复键。缓存池总量的读数还在等,拿到后补进正文。
席位意见:同意落地(devx 席位已在 head fcae6e3 复核,ACCEPT 见 #25452)。日志已证实种子在滚雪球:每次合并约 14.6 GB,远超 10 GB 的缓存池。修剪只保留本次运行用过的任务,失败模式偏保守:认不出的文件保留,没有运行摘要就不存。改动前的缓存池总量等你本机读出后补进正文,不影响审批。
你要做的:本 PR 改了
.github/workflows/**,属于 Tier H,需要 GOVERNED_APPROVERS 中的一位批准后,席位才会翻 ready 并落地。Generated by Claude Code
🤖 Generated with Claude Code
https://claude.ai/code/session_01SawdJgiuoz8deqVdM51jAE