Skip to content

fix: normalize tool call names in OpenAI streaming responses - #8339

Open
哈基米 (BlueX888) wants to merge 1 commit into
microsoft:mainfrom
BlueX888:fix/prep-openai-stream-tool-name-not-normalized
Open

哈基米 (BlueX888) wants to merge 1 commit into
microsoft:mainfrom
BlueX888:fix/prep-openai-stream-tool-name-not-normalized

Conversation

@BlueX888

Copy link
Copy Markdown

Why are these changes needed?

BaseOpenAIChatCompletionClient.create() passes tool-call names through normalize_name() (python/packages/autogen-ext/src/autogen_ext/models/openai/_openai_client.py:771), which replaces invalid characters with _ and truncates to 64 characters — its docstring notes that LLMs sometimes emit function names ignoring the API's format requirements. The streaming path, create_stream(), accumulated the name chunks raw (full_tool_calls[idx].name += tool_call_chunk.function.name, _openai_client.py:991) and built the final CreateResult from full_tool_calls without ever normalizing, so the identical API response produced a different FunctionCall.name depending on whether streaming was used (e.g. get weather! streamed vs get_weather_ non-streamed).

A raw name can violate the ^[a-zA-Z0-9_-]{1,64}$ pattern that function names must match, so replaying the streamed AssistantMessage in the next request sends the raw name back to the API, where the request can be rejected — while the identical non-streamed conversation works. The other sibling paths already normalize the LLM-provided name (OpenAI create(), Ollama create()/stream, Anthropic create()/stream); only OpenAI create_stream() omitted it.

This PR normalizes the assembled tool-call names in create_stream() once all name chunks have been merged, so streaming and non-streaming produce the same FunctionCall.name for the same API response.

Related issue number

Closes #8337

Changes

  • python/packages/autogen-ext/src/autogen_ext/models/openai/_openai_client.py: in create_stream(), after content = list(full_tool_calls.values()), pass each tool call's name through normalize_name(), mirroring create() at line 771.
  • python/packages/autogen-ext/tests/models/test_openai_model_client.py: added test_openai_chat_completion_client_create_stream_normalizes_tool_call_name, which mocks the same tool-call response (name="get weather!") for both create() and create_stream() and asserts the streamed CreateResult.content equals the non-streamed one and the normalized name get_weather_.

Testing

From python/, with the repo venv:

  • New regression test on the unpatched tree fails with the reported divergence (name='get weather!' streamed vs 'get_weather_' non-streamed); with the fix:
    • pytest packages/autogen-ext/tests/models/test_openai_model_client.py::test_openai_chat_completion_client_create_stream_normalizes_tool_call_name → 1 passed
  • Module's existing tests: pytest packages/autogen-ext/tests/models/test_openai_model_client.py → 53 passed, 33 skipped, 1 failed (test_reasoning_effort_parameter — pre-existing, fails identically on the base tree in this environment with ModuleNotFoundError: No module named 'azure', an optional dependency not installed locally; unrelated to this change)
  • ruff format --check . (autogen-ext) → 198 files already formatted
  • ruff check . (autogen-ext) → All checks passed!
  • mypy --config-file ../../pyproject.toml --exclude src/autogen_ext/runtimes/grpc/protos --exclude tests/protos --ignore-missing-imports src tests (autogen-ext poe task) → 0 errors in the changed files (the 446 reported errors are pre-existing missing-optional-dependency noise, identical on the base tree)
  • pyright on the changed files → 0 errors, identical on the base tree

Checks

The non-streaming path of BaseOpenAIChatCompletionClient.create already
passes tool call names through normalize_name, so that names violating
the ^[a-zA-Z0-9_-]{1,64}$ pattern are sanitized. The streaming path
(create_stream) assembled names from chunks without normalizing them,
so the same model output produced different FunctionCall.name values
depending on the mode, and replaying a streamed tool call in the next
request could be rejected by the API.

Normalize the assembled tool call names once the stream is complete,
after all name chunks have been merged.
@BlueX888

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant