Repository navigation
fix: normalize tool call names in OpenAI streaming responses - #8339
Open
哈基米 (BlueX888) wants to merge 1 commit into
Open
哈基米 (BlueX888) wants to merge 1 commit into
哈基米 (BlueX888) wants to merge 1 commit into
Conversation
The non-streaming path of BaseOpenAIChatCompletionClient.create already
passes tool call names through normalize_name, so that names violating
the ^[a-zA-Z0-9_-]{1,64}$ pattern are sanitized. The streaming path
(create_stream) assembled names from chunks without normalizing them,
so the same model output produced different FunctionCall.name values
depending on the mode, and replaying a streamed tool call in the next
request could be rejected by the API.
Normalize the assembled tool call names once the stream is complete,
after all name chunks have been merged.
Author
|
@microsoft-github-policy-service agree |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
BaseOpenAIChatCompletionClient.create()passes tool-call names throughnormalize_name()(python/packages/autogen-ext/src/autogen_ext/models/openai/_openai_client.py:771), which replaces invalid characters with_and truncates to 64 characters — its docstring notes that LLMs sometimes emit function names ignoring the API's format requirements. The streaming path,create_stream(), accumulated the name chunks raw (full_tool_calls[idx].name += tool_call_chunk.function.name,_openai_client.py:991) and built the finalCreateResultfromfull_tool_callswithout ever normalizing, so the identical API response produced a differentFunctionCall.namedepending on whether streaming was used (e.g.get weather!streamed vsget_weather_non-streamed).A raw name can violate the
^[a-zA-Z0-9_-]{1,64}$pattern that function names must match, so replaying the streamedAssistantMessagein the next request sends the raw name back to the API, where the request can be rejected — while the identical non-streamed conversation works. The other sibling paths already normalize the LLM-provided name (OpenAIcreate(), Ollamacreate()/stream, Anthropiccreate()/stream); only OpenAIcreate_stream()omitted it.This PR normalizes the assembled tool-call names in
create_stream()once all name chunks have been merged, so streaming and non-streaming produce the sameFunctionCall.namefor the same API response.Related issue number
Closes #8337
Changes
python/packages/autogen-ext/src/autogen_ext/models/openai/_openai_client.py: increate_stream(), aftercontent = list(full_tool_calls.values()), pass each tool call'snamethroughnormalize_name(), mirroringcreate()at line 771.python/packages/autogen-ext/tests/models/test_openai_model_client.py: addedtest_openai_chat_completion_client_create_stream_normalizes_tool_call_name, which mocks the same tool-call response (name="get weather!") for bothcreate()andcreate_stream()and asserts the streamedCreateResult.contentequals the non-streamed one and the normalized nameget_weather_.Testing
From
python/, with the repo venv:name='get weather!'streamed vs'get_weather_'non-streamed); with the fix:pytest packages/autogen-ext/tests/models/test_openai_model_client.py::test_openai_chat_completion_client_create_stream_normalizes_tool_call_name→ 1 passedpytest packages/autogen-ext/tests/models/test_openai_model_client.py→ 53 passed, 33 skipped, 1 failed (test_reasoning_effort_parameter— pre-existing, fails identically on the base tree in this environment withModuleNotFoundError: No module named 'azure', an optional dependency not installed locally; unrelated to this change)ruff format --check .(autogen-ext) → 198 files already formattedruff check .(autogen-ext) → All checks passed!mypy --config-file ../../pyproject.toml --exclude src/autogen_ext/runtimes/grpc/protos --exclude tests/protos --ignore-missing-imports src tests(autogen-ext poe task) → 0 errors in the changed files (the 446 reported errors are pre-existing missing-optional-dependency noise, identical on the base tree)pyrighton the changed files → 0 errors, identical on the base treeChecks