Skip to content

GRAPHIFY_DISABLE_THINKING is ignored by the claude backend, so thinking eats max_tokens and extraction truncates #4546

Open

Description

@tobimax

Summary

GRAPHIFY_DISABLE_THINKING is honoured by the OpenAI-compatible and Ollama paths but not by the native Claude backend (--backend claude). _call_claude in graphify/llm.py sends no thinking parameter at all, so on Claude models that think by default the thinking tokens count against the response's max_tokens (16384 from BACKENDS["claude"]), and extraction truncates or comes back hollow.

Version

graphifyy 0.9.92, anthropic 1.15.0, Python 3.14, graphify extract . --backend claude --model claude-haiku-5-5 --force (headless, in CI).

What happens

Claude Haiku 5.5 runs adaptive thinking by default. With _call_claude sending no thinking setting, a full extraction of a docs-heavy repo (347 semantic files) logged:

  • 38 chunks truncated at depth N, splitting into halves, 11 of them still truncated at recursion depth 3 (max 3) — partial result kept;
  • 3 hollow responses, one of them content=no nodes/edges, output_tokens=13347 — almost all of the 16384 spent on thinking;
  • 64/347 dispatched file(s) produced no nodes, after which the shrink guard (correctly) refused to overwrite the existing graph.

Setting GRAPHIFY_DISABLE_THINKING=1 changes nothing on this path, because _thinking_disabled_via_env() is read in the OpenAI-compatible and Ollama request builders but not in _call_claude (nor in the claude branch of the labelling call).

Expected

GRAPHIFY_DISABLE_THINKING=1 sends thinking={"type": "disabled"} on the Claude backend too, in both _call_claude and the labelling call, as it does elsewhere. Claude Haiku 5.5 accepts disabled thinking at its default effort. Optionally, a way to set output_config.effort (for example GRAPHIFY_CLAUDE_EFFORT) would let users keep thinking on but bounded.

Workaround we are using

GRAPHIFY_MAX_OUTPUT_TOKENS=21000 plus --token-budget 15000, so each smaller chunk's response, thinking included, fits. It works around the symptom; it does not stop thinking being billed on a structured-extraction task that does not need it.

Related

#730 (hardcoded max_tokens), #2880 (hollow responses), #3671 (labelling discards finish_reason='length').

Activity

  1. github-actions commented on Oct 18, 2026

    @github-actions

    Thanks for opening this issue, @tobimax. A maintainer will take a look soon.

    If you would like to discuss it in real time, come say hi on our Discord server. For longer-form questions and ideas there is also GitHub Discussions.

    To help us triage, please make sure the report includes what you expected, what actually happened, and the steps (and a small sample) to reproduce it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions