Repository navigation
Option for 1-hour prompt cache TTL for Claude models (5-minute cache expires during normal interactive use) #5220
Description
Describe the feature or problem you'd like to solve
Claude prompt caching in Copilot CLI appears to use Anthropic's default 5-minute cache TTL. In normal interactive use (read the answer, think, write the next prompt) gaps of more than 5 minutes are common, so the full conversation gets re-written to cache on almost every turn instead of being read from cache.
I measured this with copilot -p / --resume in fresh sessions (~35k-token context), sending a follow-up after different idle gaps, and reading cache_read_tokens / cache_write_tokens from the local session usage data. Two independent runs gave identical results:
| Idle gap | Claude Opus/Sonnet/Haiku 5.5 | GPT-6.1-sol / GPT-6-sol / GPT-5.6-sol |
|---|---|---|
| 2 min | ~95% cache read | ~95% cache read |
| 11 min | 0% cache read (~100% cache write) | ~95% cache read |
| ~85 min | 0% | 0% |
My broader session history (Claude 5.x, CLI) shows the same pattern: 98% cache read for gaps < 4 min vs ~1% for gaps of 6–60 min.
Anthropic supports a 1-hour TTL (cache_control: {"type": "ephemeral", "ttl": "1h"}). VS Code Copilot Chat already has experimental settings for this (github.copilot.chat.anthropic.promptCaching.extendedTtl / extendedTtlMessages), though they currently miss Claude 5.x models (microsoft/vscode#340830, fix in microsoft/vscode#340831). Copilot CLI has no equivalent option.
Proposed solution
Add an option in Copilot CLI (config setting and/or env var) to use the 1-hour cache TTL for Claude models, covering the Claude 5.x family. Consider making it the default for interactive sessions, since the higher write cost (2x vs 1.25x) is usually recovered after one or two follow-up turns with a pause longer than 5 minutes.
Example prompts or workflows
Any interactive session with a large context where the user pauses longer than 5 minutes between prompts (reviewing diffs, testing changes, meetings).
Additional context
- Related: Enhance the prompt caching for Claude Sonnet model to reduce latency and token costs #3808 (prompt caching for Claude), Auto-compact on idle, aligned to the model's prompt cache TTL #4724 (auto-compact on idle aligned to cache TTL)
- Triggered by this discussion: https://www.reddit.com/r/GithubCopilot/comments/1xauf3s/anthropic_caching_is_this_a_bad_joke/
- Tested with Copilot CLI 1.0.102-1 on Windows.
Activity