Repository navigation
Conversation
The model call generator is suspended at its yield while the flow post-processes the response (runs tools), and call_llm was only ended in the generator's finally, so its duration included downstream tool time. End call_llm before yielding the final non-partial response that reports a finish reason, after trace_call_llm and the after-model callback. The finally still ends it on early close or when no finish reason is reported. Fixes google#7625
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
Problem:
_run_one_step_asyncruns_postprocess_async, and so the tools, inside theasync forover_call_llm_async. The model-call generator is suspended at itsyieldwhile the tools run.call_llmis opened in_model_call.pyand was only ended in the generator'sfinally, which runs after the tools, so its duration included all of the downstream tool time.generate_content(schema v2) already ends at the response when a finish reason is reported (6364060). This makescall_llm(schema v1) behave the same way.Solution:
call_llmbefore yielding the final non-partial response that reports a finish reason, aftertrace_call_llmand the after-model callback have run.finallynow checksis_recording(), soaclose()/ early termination, and responses without a finish reason, still end the span exactly once at teardown.Parentage is unchanged:
execute_toolis already a child ofinvoke_agent, not of the model spans (83d966c).Out of scope, left for discussion on #7625: responses that report no finish reason (as in the issue's fake model) still keep both model spans open until generator teardown. Closing them there would require pulling ahead on the generator.
Testing Plan
Unit Tests:
New test
test_model_spans_end_before_the_tool_call_startsintests/unittests/telemetry/test_functional.py, parametrized over streaming / non-streaming and schema v1 / v2. It asserts that every model span ends before the nextexecute_toolspan starts. It checks span order, not timings, so it isn't timing-sensitive. Without the fix, the two v1 cases fail.tox(pytest tests/unittests) on macOS arm64:The one failure is
tests/unittests/integrations/vmaas/test_agent_engine_environment.py::test_reading_directory_raises(DID NOT RAISE OSError). It fails the same way on unmodifiedmain(8909689) on macOS and is unrelated to this change.Manual End-to-End (E2E) Tests:
The standalone script from #7625 with a fake model that reports
finish_reason=STOP: model delay 0.1 s, tool delay 1.0 s,ADK_TELEMETRY_SCHEMA_VERSION_OPT_IN=1.call_llmcall_llmgenerate_contentgenerate_contentChecklist
Additional context
I know
call_llmis deprecated (v1 is supported until 2027-03-31). If you'd rather not change v1, I'm happy to cut this down to the test plus a docs note on the no-finish-reason case.