Skip to content

fix(models): honor tool_config function calling mode in AnthropicLlm - #7634

Open
Somay-kousis wants to merge 1 commit into
google:mainfrom
Somay-kousis:fix/r5-a
Open

Somay-kousis wants to merge 1 commit into
google:mainfrom
Somay-kousis:fix/r5-a

Conversation

@Somay-kousis

Copy link
Copy Markdown
Contributor

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

2. Or, if no issue exists, describe the change:

Problem:

AnthropicLlm always sent tool_choice auto, so the function calling mode in generate_content_config.tool_config never reached Claude. NONE did not stop tool calls, ANY did not force one, and the model_consult force plugin (ANY with one allowed name) had no effect on Claude.

Solution:

AnthropicLlm now maps config.tool_config.function_calling_config to Claude's tool_choice:

ADK mode Claude tool_choice
NONE {"type": "none"}
ANY with one allowed_function_names entry {"type": "tool", "name": ...}
ANY otherwise {"type": "any"}
AUTO, VALIDATED, unset, no config {"type": "auto"} (unchanged)

Two limits come from the Anthropic API:

  • Claude rejects forced tool use (any or tool) while extended thinking is on. In that case ANY falls back to auto with a warning, the same way the existing output-schema tool falls back. NONE is kept, since Claude accepts it with thinking.
  • Claude can't limit the call to a subset of several tools, so two or more allowed names map to any. integrations/openai/_openai_common.py handles the same case this way.

Requests with no tools still send no tool_choice. The output-schema path, which sets its own choice, is unchanged.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

New tests in tests/unittests/models/test_anthropic_llm.py:

  • test_tool_config_mode_reaches_claude_tool_choice: every mode, streaming and non-streaming.
  • test_forced_tool_choice_with_thinking_falls_back_to_auto: ANY and ANY with one name, with a manual budget and with adaptive thinking.
  • test_tool_config_none_mode_is_kept_with_thinking.
main: 9 failed, 193 passed
fix:  202 passed

The 10 new cases that pass on main are the ones where auto is correct. Three mutations of the fix (NONE back to auto, single-name branch disabled, thinking fallback removed) each make at least 3 tests fail. tests/unittests/models and tests/unittests/flows pass (2814 tests). isort, pyink and mypy are clean on the changed file.

Manual End-to-End (E2E) Tests:

The repro in the issue (Agent + InMemoryRunner, AsyncMessages.create faked to record the request):

before                                   after
NONE None -> {'type': 'auto'}            NONE None -> {'type': 'none'}
ANY None -> {'type': 'auto'}             ANY None -> {'type': 'any'}
ANY ['get_weather'] -> {'type': 'auto'}  ANY ['get_weather'] -> {'type': 'tool', 'name': 'get_weather'}

No live Claude call; the thinking limits come from Anthropic's tool use docs and the fallback this file already has.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

LiteLlm also ignores a single allowed function name (ANY always becomes "required", lite_llm.py:3645-3654). Not changed here.

AnthropicLlm always sent tool_choice auto, so a FunctionCallingConfig
mode of NONE or ANY (or ANY with one allowed function name) set in
generate_content_config.tool_config never reached Claude. Map the mode
to Claude's none, any and tool choices. A forced choice falls back to
auto while extended thinking is on, since Claude rejects forced tool
use there.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AnthropicLlm ignores tool_config: function calling mode NONE or ANY never reaches Claude

2 participants