Repository navigation
Unknown model_tokens silently falls back to 8192 and truncates — warning goes to stderr, once per process, and never reaches the result #1121
Description
Summary
When a model is not in the token table, the graph warns once to stderr and then proceeds with an 8192-token window. For a model with a 1M-token context this silently truncates every non-trivial page, changing the answer without failing, without appearing in the returned object, and without any indication in execution_info.
Max input tokens for model google_genai/gemini-2.5-flash not found, please specify
the model_tokens parameter in the llm section of the graph configuration.
Using default token size: 8192
Why the default is the problem
A silent 8192 window is a plausible answer generator. The run succeeds, the JSON validates, the fields look reasonable — and the model simply never saw the part of the page that mattered. On a long privacy policy or a services page, the extracted answer flips based on where the truncation landed.
Two properties make it easy to miss:
- the warning is on stderr, so it vanishes in any batch/worker/async context — I hit this inside a 15-agent pipeline where nothing surfaced it
- it is emitted once per process, not once per run, so a long job warns on the first URL and stays silent for the remaining hundreds
What I'd suggest, in preference order
- Raise when
model_tokensis absent for an unknown model, with the message telling the user to set it. Loud beats quietly wrong for something that changes results. - If a hard raise is too breaking, keep the fallback but put it in the result — e.g.
execution_info["truncated"] = Trueplus the effective window — so downstream code can detect it. - Failing both, make the default large rather than small. An over-long prompt fails visibly at the provider; an over-short one fails invisibly.
- Independently:
gemini-2.5-flash/gemini-flash-latestare worth adding to the token table — Google's current flash line is 1M input.
Related
#1099 / #1002 cover model_tokens being rejected on some provider paths. This is the opposite case: it is accepted, absent, and silently defaulted.
Environment
scrapegraphai fresh from PyPI, Python 3.13, macOS, langchain-google-genai (note: not bundled — installing it is a required extra step for Google backends), Playwright chromium headless.
Happy to open a PR for (1) or (2) if you indicate which you'd accept.
Activity
dosubot commentedon Jul 28, 2026
Hi @SilverBulletNeo! I'm Dosu and I'm helping the ScrapeGraphAI, Inc team.
Your analysis is accurate. The fallback lives in
abstract_graph.pylines 207–222 — when the model/provider pair isn't in themodels_tokensregistry, alogger.warningis emitted andself.model_tokenis hard-coded to8192[1]. That token budget is then used directly as thechunk_sizefor document splitting [2], so for a 1M-context model this silently over-chunks and the model never sees the full page — exactly the failure mode you describe.On
gemini-2.5-flash: confirmed missing from the registry. Thegoogle_genaisection currently hasgemini-2.0-flash-latest(1M) andgemini-2.0-pro-exp(2M), but nothing for the 2.5 line orgemini-flash-latest[3]. No open PR adds them.On the 8192 default: your option (1) — raising on an unknown model — would be the cleanest fix. The current behavior is a plausible-but-wrong silent degradation, which is worse than a loud failure. Your observation about
logger.warningvanishing in batch/async contexts and firing only once per process (Python's logging dedup) makes this especially dangerous at scale.Workaround for now: you can set
model_tokensexplicitly in your config to bypass the lookup entirely:This takes the
elsebranch at line 221 and skips the registry entirely. Note that on v2.1.4model_tokensleaked into the LLM client call [4], but that was fixed in v2.1.5 via PR #1100 [5], so make sure you're on ≥ 2.1.5.A PR for (1) or (4) — or both — would be a solid contribution.
To reply, just mention @dosu.
Share context across your team and agents. Try Dosu.
fix(graph): expose when the 8192 token fallback was used
ci(release): 2.1.7 [skip ci]
ci(release): 2.2.0-beta.6 [skip ci]
ci(release): 2.2.0 [skip ci]
chenhz01 commentedon Sep 22, 2026
"A plausible answer generator" is exactly right - the run fails to fail. Make the unknown-token path a hard contract. (1) If
model_tokensis unknown, don't degrade to 8192 - raise a config error by default, with an explicitallow_default_token_window=Trueescape hatch for the old behavior. Open PRs already expose the effective window in execution_info; the proposal below goes further on default behavior. (2) On truncation, attachinput_truncated: true, the window used, and source length to the result andexecution_info, so the caller sees the model never saw half the page. (3) The open PR covers routing the signal into the result; what remains unclaimed is the raise-by-default contract and escape hatch, which I'd focus on. Truncation that can flip answers must be a first-class diffable fact on the result, not a log line. Want me to take a crack at it? Happy to open a PR with working code.