Ollama models endlessly loop instead of respond

I have tried multiple models through ollama but all of them end up in a recursive loop where they endlessly call tools to respond and never return back to the conversation. I cannot paste the logs because my terminal simply doesnt respond when this happens. This is what I see in the web UI though:

content:
parts:
0:
functionCall:
id: "call_449f320d-5f80-4058-8732-2a14b39a065f"
args:
query: "Hello there"
name: "greet"
role: "model"
partial: false
invocation_id: "e-8fd53ed3-14e8-4941-b7c3-fc08e85a6998"
author: "basic_agent"
actions:
state_delta:
artifact_delta:
requested_auth_configs:
long_running_tool_ids:
id: "aGYQW3D7"
timestamp: 1744369858.379057

content:
parts:
0:
functionResponse:
id: "call_449f320d-5f80-4058-8732-2a14b39a065f"
name: "greet"
response:
status: "success"
report: "Hello! I am a weather and time agent. I can answer your questions about the time and weather in a city."
role: "user"
invocation_id: "e-8fd53ed3-14e8-4941-b7c3-fc08e85a6998"
author: "basic_agent"
actions:
state_delta:
artifact_delta:
requested_auth_configs:
id: "WIAKDg4e"
timestamp: 1744369862.21992

The models I used are relatively small. Gemma, Llama & Mistral 7b 8b versions. Is this an issue with ollama, the models or with the adk?

@liatoutou

@liatoutou I help you with the log (from my test):

Terminal version

adk run multi_tool_agent                                                                                                                                  14:39:17 
Log setup complete: /var/folders/0t/60hzjwl11r9blfgm7hphh65c0000gn/T/agents_log/agent.20250411_144005.log
To access latest log: tail -F /var/folders/0t/60hzjwl11r9blfgm7hphh65c0000gn/T/agents_log/agent.latest.log
tellRunning agent weather_time_agent, type exit to exit.
user: tell me about new york
14:40:13 - LiteLLM:INFO: utils.py:3076 - 
LiteLLM completion() model= llama3.2:latest; provider = ollama
14:40:14 - LiteLLM:INFO: cost_calculator.py:634 - selected model name for cost calculation: ollama/llama3.2:latest
14:40:14 - LiteLLM:INFO: utils.py:3076 - 
LiteLLM completion() model= llama3.2:latest; provider = ollama
14:40:14 - LiteLLM:INFO: cost_calculator.py:634 - selected model name for cost calculation: ollama/llama3.2:latest
14:40:14 - LiteLLM:INFO: cost_calculator.py:634 - selected model name for cost calculation: ollama/llama3.2:latest
14:40:14 - LiteLLM:INFO: utils.py:3076 - 
LiteLLM completion() model= llama3.2:latest; provider = ollama
14:40:14 - LiteLLM:INFO: cost_calculator.py:634 - selected model name for cost calculation: ollama/llama3.2:latest
14:40:15 - LiteLLM:INFO: cost_calculator.py:634 - selected model name for cost calculation: ollama/llama3.2:latest
14:40:15 - LiteLLM:INFO: utils.py:3076 - 
LiteLLM completion() model= llama3.2:latest; provider = ollama
14:40:15 - LiteLLM:INFO: cost_calculator.py:634 - selected model name for cost calculation: ollama/llama3.2:latest
14:40:15 - LiteLLM:INFO: cost_calculator.py:634 - selected model name for cost calculation: ollama/llama3.2:latest
14:40:15 - LiteLLM:INFO: utils.py:3076 - 
LiteLLM completion() model= llama3.2:latest; provider = ollama
14:40:15 - LiteLLM:INFO: cost_calculator.py:634 - selected model name for cost calculation: ollama/llama3.2:latest
^C
Aborted!

web version

adk web                                                                                                                                                   14:41:16 
INFO:     Started server process [35480]
INFO:     Waiting for application startup.
+-----------------------------------------------------------------------------+
| ADK Web Server started                                                      |
|                                                                             |
| For local testing, access at http://localhost:8000.                         |
+-----------------------------------------------------------------------------+

INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     127.0.0.1:53691 - "GET / HTTP/1.1" 307 Temporary Redirect
INFO:     127.0.0.1:53691 - "GET /dev-ui HTTP/1.1" 200 OK
INFO:     127.0.0.1:53691 - "GET /polyfills-FFHMD2TL.js HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:53691 - "GET /main-SLIAU2JL.js HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:53692 - "GET /styles-4VDSPQ37.css HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:53692 - "GET /assets/config/runtime-config.json HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:53692 - "GET /list-apps?relative_path=./ HTTP/1.1" 200 OK
2025-04-11 14:41:35,980 - INFO - fast_api.py:294 - New session created
INFO:     127.0.0.1:53702 - "POST /apps/multi_tool_agent/users/user/sessions HTTP/1.1" 200 OK
INFO:     127.0.0.1:53702 - "GET /apps/multi_tool_agent/eval_sets HTTP/1.1" 200 OK
INFO:     127.0.0.1:53703 - "GET /apps/multi_tool_agent/users/user/sessions HTTP/1.1" 200 OK
INFO:     127.0.0.1:53703 - "GET /apps/multi_tool_agent/users/user/sessions HTTP/1.1" 200 OK
INFO:     127.0.0.1:53718 - "POST /run_sse HTTP/1.1" 200 OK
2025-04-11 14:41:40,523 - INFO - envs.py:47 - Loaded .env file for multi_tool_agent at /Users/montimage/buildspace/buildwizai/what-the-dog-say/multi_tool_agent/.env
2025-04-11 14:41:40,523 - INFO - envs.py:53 - Reloaded .env file for multi_tool_agent at /Users/montimage/buildspace/buildwizai/what-the-dog-say/multi_tool_agent/.env
2025-04-11 14:41:41,044 - INFO - _client.py:1025 - HTTP Request: GET https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json "HTTP/1.1 200 OK"
2025-04-11 14:41:41,390 - INFO - lite_llm.py:597 - 
LLM Request:
-----------------------------------------------------------
System Instruction:
I can answer your questions about the time and weather in a city.

You are an agent. Your internal name is "weather_time_agent".

 The description about you is "Agent to answer questions about the time and weather in a city."
-----------------------------------------------------------
Contents:
{"parts":[{"text":"new york"}],"role":"user"}
-----------------------------------------------------------
Functions:
get_weather: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
get_current_time: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
-----------------------------------------------------------

14:41:41 - LiteLLM:INFO: utils.py:3076 - 
LiteLLM completion() model= gemma3:12b; provider = ollama
2025-04-11 14:41:41,393 - INFO - utils.py:3076 - 
LiteLLM completion() model= gemma3:12b; provider = ollama
2025-04-11 14:41:46,189 - INFO - _client.py:1740 - HTTP Request: POST http://localhost:11434/api/generate "HTTP/1.1 200 OK"
14:41:46 - LiteLLM:INFO: cost_calculator.py:634 - selected model name for cost calculation: ollama/gemma3:12b
2025-04-11 14:41:46,193 - INFO - cost_calculator.py:634 - selected model name for cost calculation: ollama/gemma3:12b
2025-04-11 14:41:46,257 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-11 14:41:46,262 - INFO - fast_api.py:613 - Generated event in agent run streaming: {"content":{"parts":[{"functionCall":{"id":"call_79154f6b-21fb-4bd6-8a30-9b2f041e1c65","args":{"city":"new york"},"name":"get_weather"}}],"role":"model"},"partial":false,"invocation_id":"e-79cc5e7b-09e1-4c50-acd1-35b2643c6c5a","author":"weather_time_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"long_running_tool_ids":[],"id":"tbW1UWKa","timestamp":1744375301.390833}
2025-04-11 14:41:46,262 - INFO - fast_api.py:613 - Generated event in agent run streaming: {"content":{"parts":[{"functionResponse":{"id":"call_79154f6b-21fb-4bd6-8a30-9b2f041e1c65","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"},"invocation_id":"e-79cc5e7b-09e1-4c50-acd1-35b2643c6c5a","author":"weather_time_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"id":"FcIY48Aq","timestamp":1744375306.262607}
2025-04-11 14:41:46,263 - INFO - lite_llm.py:597 - 
LLM Request:
-----------------------------------------------------------
System Instruction:
I can answer your questions about the time and weather in a city.

You are an agent. Your internal name is "weather_time_agent".

 The description about you is "Agent to answer questions about the time and weather in a city."
-----------------------------------------------------------
Contents:
{"parts":[{"text":"new york"}],"role":"user"}
{"parts":[{"function_call":{"id":"call_79154f6b-21fb-4bd6-8a30-9b2f041e1c65","args":{"city":"new york"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_79154f6b-21fb-4bd6-8a30-9b2f041e1c65","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"}
-----------------------------------------------------------
Functions:
get_weather: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
get_current_time: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
-----------------------------------------------------------

14:41:46 - LiteLLM:INFO: utils.py:3076 - 
LiteLLM completion() model= gemma3:12b; provider = ollama
2025-04-11 14:41:46,264 - INFO - utils.py:3076 - 
LiteLLM completion() model= gemma3:12b; provider = ollama
2025-04-11 14:41:46,334 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
14:41:46 - LiteLLM:INFO: cost_calculator.py:634 - selected model name for cost calculation: ollama/gemma3:12b
2025-04-11 14:41:46,338 - INFO - cost_calculator.py:634 - selected model name for cost calculation: ollama/gemma3:12b
2025-04-11 14:41:46,397 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-11 14:41:46,472 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-11 14:41:47,203 - INFO - _client.py:1740 - HTTP Request: POST http://localhost:11434/api/generate "HTTP/1.1 200 OK"
14:41:47 - LiteLLM:INFO: cost_calculator.py:634 - selected model name for cost calculation: ollama/gemma3:12b
2025-04-11 14:41:47,205 - INFO - cost_calculator.py:634 - selected model name for cost calculation: ollama/gemma3:12b
2025-04-11 14:41:47,265 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-11 14:41:47,271 - INFO - fast_api.py:613 - Generated event in agent run streaming: {"content":{"parts":[{"functionCall":{"id":"call_f39f8005-57b2-464f-83ed-182bf93d5337","args":{"city":"new york"},"name":"get_weather"}}],"role":"model"},"partial":false,"invocation_id":"e-79cc5e7b-09e1-4c50-acd1-35b2643c6c5a","author":"weather_time_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"long_running_tool_ids":[],"id":"aNIS1b2E","timestamp":1744375306.263028}
2025-04-11 14:41:47,271 - INFO - fast_api.py:613 - Generated event in agent run streaming: {"content":{"parts":[{"functionResponse":{"id":"call_f39f8005-57b2-464f-83ed-182bf93d5337","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"},"invocation_id":"e-79cc5e7b-09e1-4c50-acd1-35b2643c6c5a","author":"weather_time_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"id":"0pp38B5G","timestamp":1744375307.271681}
2025-04-11 14:41:47,272 - INFO - lite_llm.py:597 - 
LLM Request:
-----------------------------------------------------------
System Instruction:
I can answer your questions about the time and weather in a city.

You are an agent. Your internal name is "weather_time_agent".

 The description about you is "Agent to answer questions about the time and weather in a city."
-----------------------------------------------------------
Contents:
{"parts":[{"text":"new york"}],"role":"user"}
{"parts":[{"function_call":{"id":"call_79154f6b-21fb-4bd6-8a30-9b2f041e1c65","args":{"city":"new york"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_79154f6b-21fb-4bd6-8a30-9b2f041e1c65","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_f39f8005-57b2-464f-83ed-182bf93d5337","args":{"city":"new york"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_f39f8005-57b2-464f-83ed-182bf93d5337","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"}
-----------------------------------------------------------
Functions:
get_weather: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
get_current_time: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
-----------------------------------------------------------

Could you provide a minimal example?

My hunch is you need a bit more system instruction to instruct what the model should do for agentic applications. Some models have built-in agentic capabilities and some models(esp. smaller ones) doesn't.

@liatoutou

@liatoutou @luongnv89 Based on the logs you have provided, you are likely setting the model with ollama provider. Please try using ollama_chat instead:

LiteLlm(model="ollama_chat/<YOUR_MODEL_NAME>")

The reason: The underlying litellm library seems to be adding the following between system instructions and the function descriptions here when ollama provider is used:

Produce JSON OUTPUT ONLY! Adhere to this format {\"name\": \"function_name\", \"arguments\":{\"argument_name\": \"argument_value\"}} The following functions are available to you:

This should help. Please let us know.

@selcukgun

@selcukgun

Thank you for the proposed solution. It partially worked, because the model isn't endlessly looping with its toolcalls. However, with ollama_chat it doesnt seem to be able to call tools anymore. Now to my question "what is the time?", it responds with a dict but doesnt call any tools:

content:
parts:
0:
text: "{"name": "get_current_time", "arguments": {"city": "current_location"}}"
role: "model"
partial: false
invocation_id: "e-576d5a8e-5e8e-4d2c-aa87-16ed031f1e25"
author: "weather_time_agent"
actions:
state_delta:
artifact_delta:
requested_auth_configs:
id: "rKkC9mOD"

For reference, this is my code now which I got from following the quickstart guide:

import datetime
from zoneinfo import ZoneInfo
from google.adk.agents import Agent
from google.adk.models.lite_llm import LiteLlm

def get_weather(city: str) -> dict:
    """Retrieves the current weather report for a specified city.

    Args:
        city (str): The name of the city for which to retrieve the weather report.

    Returns:
        dict: status and result or error msg.
    """
    if city.lower() == "new york":
        return {
            "status": "success",
            "report": (
                "The weather in New York is sunny with a temperature of 25 degrees"
                " Celsius (41 degrees Fahrenheit)."
            ),
        }
    else:
        return {
            "status": "error",
            "error_message": f"Weather information for '{city}' is not available.",
        }


def get_current_time(city: str) -> dict:
    """Returns the current time in a specified city.

    Args:
        city (str): The name of the city for which to retrieve the current time.

    Returns:
        dict: status and result or error msg.
    """

    if city.lower() == "new york":
        tz_identifier = "America/New_York"
    else:
        return {
            "status": "error",
            "error_message": (
                f"Sorry, I don't have timezone information for {city}."
            ),
        }

    tz = ZoneInfo(tz_identifier)
    now = datetime.datetime.now(tz)
    report = (
        f'The current time in {city} is {now.strftime("%Y-%m-%d %H:%M:%S %Z%z")}'
    )
    return {"status": "success", "report": report}


root_agent = Agent(
    name="weather_time_agent",
     model=LiteLlm(
        model='ollama_chat/gemma3:12b'
    ),
    description=(
        "Agent to answer questions about the time and weather in a city."
    ),
    instruction=(
        "I can answer your questions about the time and weather in a city."
    ),
    tools=[get_weather, get_current_time],
)

Is the gemma model just not capable of performing these toolcalls?

@hangfei

I tried it with a larger model mistral-small3.1, and now indeed it is conversating better, and it also seems to call tools at the appropriate time. However, now I run into the following error:

LLM Request:
-----------------------------------------------------------
System Instruction:
I can answer your questions about the time and weather in a city.

You are an agent. Your internal name is "weather_time_agent".

 The description about you is "Agent to answer questions about the time and weather in a city."
-----------------------------------------------------------
Contents:
{"parts":[{"text":"Hello there"}],"role":"user"}
-----------------------------------------------------------
Functions:
get_weather: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
get_current_time: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
-----------------------------------------------------------

14:06:00 - LiteLLM:INFO: utils.py:3075 - 
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-12 14:06:00,062 - INFO - utils.py:3075 -
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-12 14:06:02,145 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
14:06:08 - LiteLLM:INFO: cost_calculator.py:622 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-12 14:06:08,285 - INFO - cost_calculator.py:622 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-12 14:06:10,336 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-12 14:06:10,337 - INFO - fast_api.py:613 - Generated event in agent run streaming: {"content":{"parts":[{"text":"Hi, I am here to answer your questions about the time and weather in a city. Please let me know if you have any questions."}],"role":"model"},"partial":false,"invocation_id":"e-8aaac3d3-ed48-4479-afd0-03399e7c53df","author":"weather_time_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"id":"hWgPOXpY","timestamp":1744459560.059232}
2025-04-12 14:06:10,365 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
14:06:10 - LiteLLM:INFO: cost_calculator.py:622 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-12 14:06:10,366 - INFO - cost_calculator.py:622 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-12 14:06:10,393 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-12 14:06:10,420 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
INFO:     127.0.0.1:52007 - "GET /apps/nosana_agent/users/user/sessions/65732d11-53a7-456a-ac79-2d950742c5c9 HTTP/1.1" 200 OK
INFO:     127.0.0.1:52007 - "POST /run_sse HTTP/1.1" 200 OK
2025-04-12 14:06:13,606 - INFO - lite_llm.py:597 - 
LLM Request:
-----------------------------------------------------------
System Instruction:
I can answer your questions about the time and weather in a city.

You are an agent. Your internal name is "weather_time_agent".

 The description about you is "Agent to answer questions about the time and weather in a city."
-----------------------------------------------------------
Contents:
{"parts":[{"text":"Hello there"}],"role":"user"}
{"parts":[{"text":"Hi, I am here to answer your questions about the time and weather in a city. Please let me know if you have any questions."}],"role":"model"}
{"parts":[{"text":"Can you tell me the time?"}],"role":"user"}
-----------------------------------------------------------
Functions:
get_weather: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
get_current_time: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
-----------------------------------------------------------

14:06:13 - LiteLLM:INFO: utils.py:3075 -
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-12 14:06:13,607 - INFO - utils.py:3075 -
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-12 14:06:13,635 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
14:06:16 - LiteLLM:INFO: cost_calculator.py:622 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-12 14:06:16,576 - INFO - cost_calculator.py:622 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-12 14:06:16,604 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-12 14:06:16,605 - INFO - fast_api.py:613 - Generated event in agent run streaming: {"content":{"parts":[{"text":"Of course! Please tell me the city name."}],"role":"model"},"partial":false,"invocation_id":"e-c2c2ed5e-8774-4871-8465-cb396d23ecec","author":"weather_time_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"id":"9kVVbakO","timestamp":1744459573.606774}
2025-04-12 14:06:16,634 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
14:06:16 - LiteLLM:INFO: cost_calculator.py:622 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-12 14:06:16,635 - INFO - cost_calculator.py:622 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-12 14:06:16,662 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-12 14:06:16,690 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
INFO:     127.0.0.1:52007 - "GET /apps/nosana_agent/users/user/sessions/65732d11-53a7-456a-ac79-2d950742c5c9 HTTP/1.1" 200 OK
INFO:     127.0.0.1:52007 - "POST /run_sse HTTP/1.1" 200 OK
2025-04-12 14:06:19,666 - INFO - lite_llm.py:597 - 
LLM Request:
-----------------------------------------------------------
System Instruction:
I can answer your questions about the time and weather in a city.

You are an agent. Your internal name is "weather_time_agent".

 The description about you is "Agent to answer questions about the time and weather in a city."
-----------------------------------------------------------
Contents:
{"parts":[{"text":"Hello there"}],"role":"user"}
{"parts":[{"text":"Hi, I am here to answer your questions about the time and weather in a city. Please let me know if you have any questions."}],"role":"model"}
{"parts":[{"text":"Can you tell me the time?"}],"role":"user"}
{"parts":[{"text":"Of course! Please tell me the city name."}],"role":"model"}
{"parts":[{"text":"New york"}],"role":"user"}
-----------------------------------------------------------
Functions:
get_weather: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
get_current_time: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
-----------------------------------------------------------

14:06:19 - LiteLLM:INFO: utils.py:3075 -
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-12 14:06:19,667 - INFO - utils.py:3075 -
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-12 14:06:19,696 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
14:06:28 - LiteLLM:INFO: cost_calculator.py:622 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-12 14:06:28,522 - INFO - cost_calculator.py:622 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-12 14:06:30,589 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-12 14:06:30,590 - INFO - fast_api.py:613 - Generated event in agent run streaming: {"content":{"parts":[{"functionCall":{"id":"e396e9b4-2a85-42c2-abc4-f2b4dce101b4","args":{"city":"new york"},"name":"get_current_time"}},{"functionCall":{"id":"40a5d21c-948a-499e-8118-cf87291a04bb","args":{"city":"new york"},"name":"get_weather"}}],"role":"model"},"partial":false,"invocation_id":"e-6868a996-8073-43aa-adbf-be76248469ed","author":"weather_time_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"long_running_tool_ids":[],"id":"8Ee0X7Y5","timestamp":1744459579.666351}
2025-04-12 14:06:30,593 - INFO - fast_api.py:613 - Generated event in agent run streaming: {"content":{"parts":[{"functionResponse":{"id":"e396e9b4-2a85-42c2-abc4-f2b4dce101b4","name":"get_current_time","response":{"status":"success","report":"The current time in new york is 2025-04-12 08:06:30 EDT-0400"}}},{"functionResponse":{"id":"40a5d21c-948a-499e-8118-cf87291a04bb","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"},"invocation_id":"kgRZ6w2Y","author":"weather_time_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"id":"JCdHE25k","timestamp":1744459590.593309}
2025-04-12 14:06:30,594 - INFO - lite_llm.py:597 -
LLM Request:
-----------------------------------------------------------
System Instruction:
I can answer your questions about the time and weather in a city.

You are an agent. Your internal name is "weather_time_agent".

 The description about you is "Agent to answer questions about the time and weather in a city."
-----------------------------------------------------------
Contents:
{"parts":[{"text":"Hello there"}],"role":"user"}
{"parts":[{"text":"Hi, I am here to answer your questions about the time and weather in a city. Please let me know if you have any questions."}],"role":"model"}
{"parts":[{"text":"Can you tell me the time?"}],"role":"user"}
{"parts":[{"text":"Of course! Please tell me the city name."}],"role":"model"}
{"parts":[{"text":"New york"}],"role":"user"}
{"parts":[{"function_call":{"id":"e396e9b4-2a85-42c2-abc4-f2b4dce101b4","args":{"city":"new york"},"name":"get_current_time"}},{"function_call":{"id":"40a5d21c-948a-499e-8118-cf87291a04bb","args":{"city":"new york"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"e396e9b4-2a85-42c2-abc4-f2b4dce101b4","name":"get_current_time","response":{"status":"success","report":"The current time in new york is 2025-04-12 08:06:30 EDT-0400"}}},{"function_response":{"id":"40a5d21c-948a-499e-8118-cf87291a04bb","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"}
-----------------------------------------------------------
Functions:
get_weather: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
get_current_time: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
-----------------------------------------------------------

14:06:30 - LiteLLM:INFO: utils.py:3075 -
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-12 14:06:30,596 - INFO - utils.py:3075 -
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-12 14:06:30,624 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
14:06:30 - LiteLLM:INFO: cost_calculator.py:622 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-12 14:06:30,625 - INFO - cost_calculator.py:622 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-12 14:06:30,651 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-12 14:06:30,677 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-12 14:06:32,650 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.


Provider List: https://docs.litellm.ai/docs/providers

2025-04-12 14:06:32,680 - ERROR - fast_api.py:616 - Error in event_generator: litellm.APIConnectionError: litellm.types.llms.openai.ChatCompletionAssistantToolCall() argument after ** must be a mapping, not ChatCompletionMessageToolCall
Traceback (most recent call last):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\main.py", line 2887, in completion
    generator = ollama_chat.get_ollama_response(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\llms\ollama_chat.py", line 242, in get_ollama_response
    typed_tool = ChatCompletionAssistantToolCall(**tool)  # type: ignore
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: litellm.types.llms.openai.ChatCompletionAssistantToolCall() argument after ** must be a mapping, not ChatCompletionMessageToolCall    
Traceback (most recent call last):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\main.py", line 2887, in completion
    generator = ollama_chat.get_ollama_response(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\llms\ollama_chat.py", line 242, in get_ollama_response
    typed_tool = ChatCompletionAssistantToolCall(**tool)  # type: ignore
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: litellm.types.llms.openai.ChatCompletionAssistantToolCall() argument after ** must be a mapping, not ChatCompletionMessageToolCall    

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\cli\fast_api.py", line 605, in event_generator     
    async for event in runner.run_async(
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\runners.py", line 197, in run_async
    async for event in invocation_context.agent.run_async(invocation_context):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\agents\base_agent.py", line 141, in run_async      
    async for event in self._run_async_impl(ctx):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\agents\llm_agent.py", line 232, in _run_async_impl 
    async for event in self._llm_flow.run_async(ctx):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\flows\llm_flows\base_llm_flow.py", line 231, in run_async
    async for event in self._run_one_step_async(invocation_context):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\flows\llm_flows\base_llm_flow.py", line 257, in _run_one_step_async
    async for llm_response in self._call_llm_async(
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\flows\llm_flows\base_llm_flow.py", line 470, in _call_llm_async
    async for llm_response in llm.generate_content_async(
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\models\lite_llm.py", line 658, in generate_content_async
    response = await self.llm_client.acompletion(**completion_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\models\lite_llm.py", line 88, in acompletion       
    return await acompletion(
           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\utils.py", line 1452, in wrapper_async
    raise e
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\utils.py", line 1313, in wrapper_async
    result = await original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\main.py", line 496, in acompletion
    raise exception_type(
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\main.py", line 469, in acompletion
    init_response = await loop.run_in_executor(None, func_with_context)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\miniconda3\Lib\concurrent\futures\thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\utils.py", line 982, in wrapper
    result = original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\main.py", line 3166, in completion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\litellm_core_utils\exception_mapping_utils.py", line 2214, in exception_type
    raise e
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\litellm_core_utils\exception_mapping_utils.py", line 2190, in exception_type
    raise APIConnectionError(
litellm.exceptions.APIConnectionError: litellm.APIConnectionError: litellm.types.llms.openai.ChatCompletionAssistantToolCall() argument after ** must be a mapping, not ChatCompletionMessageToolCall
Traceback (most recent call last):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\main.py", line 2887, in completion
    generator = ollama_chat.get_ollama_response(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\llms\ollama_chat.py", line 242, in get_ollama_response
    typed_tool = ChatCompletionAssistantToolCall(**tool)  # type: ignore
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: litellm.types.llms.openai.ChatCompletionAssistantToolCall() argument after ** must be a mapping, not ChatCompletionMessageToolCall

@hangfei I assume this doesn't have to do with the system instruction? Or is there a way to resolve this with better instructions?

Hello!
I have the same loop problem.
I created the agent from the example in the doc and instead of receiving a response, an infinite loop occurs.

Code:

import datetime
from zoneinfo import ZoneInfo
from google.adk.agents import LlmAgent
from google.adk.models.lite_llm import LiteLlm


def get_weather(city: str) -> dict:
    """Retrieves the current weather report for a specified city.

    Args:
        city (str): The name of the city for which to retrieve the weather report.

    Returns:
        dict: status and result or error msg.
    """
    if city.lower() == "new york":
        return {
            "status": "success",
            "report": (
                "The weather in New York is sunny with a temperature of 25 degrees"
                " Celsius (41 degrees Fahrenheit)."
            ),
        }
    else:
        return {
            "status": "error",
            "error_message": f"Weather information for '{city}' is not available.",
        }


def get_current_time(city: str) -> dict:
    """Returns the current time in a specified city.

    Args:
        city (str): The name of the city for which to retrieve the current time.

    Returns:
        dict: status and result or error msg.
    """

    if city.lower() == "new york":
        tz_identifier = "America/New_York"
    else:
        return {
            "status": "error",
            "error_message": (
                f"Sorry, I don't have timezone information for {city}."
            ),
        }

    tz = ZoneInfo(tz_identifier)
    now = datetime.datetime.now(tz)
    report = (
        f'The current time in {city} is {now.strftime("%Y-%m-%d %H:%M:%S %Z%z")}'
    )
    return {"status": "success", "report": report}


root_agent = LlmAgent(
    model=LiteLlm(model="ollama/gemma:2b"),
    name="ollama_gemma_agent",
    description=(
        "Agent to answer questions about the time and weather in a city."
    ),
    instruction=(
        "I can answer your questions about the time and weather in a city."
    ),
    tools=[get_weather, get_current_time],
)

My message was just a "hello". In the following image you can see the size of the scroll bar, its small size shows the number of repeated requests.

This is a part of what appears in the terminal:

18:53:47 - LiteLLM:INFO: utils.py:3085 - 
LiteLLM completion() model= gemma:2b; provider = ollama
2025-04-12 18:53:47,375 - INFO - utils.py:3085 - 
LiteLLM completion() model= gemma:2b; provider = ollama
2025-04-12 18:53:47,410 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
18:53:47 - LiteLLM:INFO: cost_calculator.py:636 - selected model name for cost calculation: ollama/gemma:2b
2025-04-12 18:53:47,411 - INFO - cost_calculator.py:636 - selected model name for cost calculation: ollama/gemma:2b
2025-04-12 18:53:47,446 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-12 18:53:47,481 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-12 18:53:48,405 - INFO - _client.py:1740 - HTTP Request: POST http://localhost:11434/api/generate "HTTP/1.1 200 OK"
18:53:48 - LiteLLM:INFO: cost_calculator.py:636 - selected model name for cost calculation: ollama/gemma:2b
2025-04-12 18:53:48,406 - INFO - cost_calculator.py:636 - selected model name for cost calculation: ollama/gemma:2b
2025-04-12 18:53:48,443 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-12 18:53:48,444 - INFO - fast_api.py:613 - Generated event in agent run streaming: {"content":{"parts":[{"functionCall":{"id":"call_ba8d2b0d-844d-4df5-8582-4e5d972eb493","args":{"city":"New York City"},"name":"get_weather"}}],"role":"model"},"partial":false,"invocation_id":"e-417eceb1-61b8-4667-9237-452f6232b872","author":"ollama_gemma_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"long_running_tool_ids":[],"id":"pchT3lVt","timestamp":1744494827.371338}
2025-04-12 18:53:48,445 - INFO - fast_api.py:613 - Generated event in agent run streaming: {"content":{"parts":[{"functionResponse":{"id":"call_ba8d2b0d-844d-4df5-8582-4e5d972eb493","name":"get_weather","response":{"status":"error","error_message":"Weather information for 'New York City' is not available."}}}],"role":"user"},"invocation_id":"e-417eceb1-61b8-4667-9237-452f6232b872","author":"ollama_gemma_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"id":"4bksYYLq","timestamp":1744494828.445112}
2025-04-12 18:53:48,448 - INFO - lite_llm.py:597 - 
LLM Request:
-----------------------------------------------------------
System Instruction:
I can answer your questions about the time and weather in a city.

You are an agent. Your internal name is "ollama_gemma_agent".

 The description about you is "Agent to answer questions about the time and weather in a city."
-----------------------------------------------------------
Contents:
{"parts":[{"text":"hello"}],"role":"user"}
{"parts":[{"function_call":{"id":"call_e747e3ca-e580-4a98-b667-9607962be1c3","args":{"city":"New York"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_e747e3ca-e580-4a98-b667-9607962be1c3","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_4bf0c7b0-a56e-48e3-9cff-fee895fee262","args":{"city":"New York"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_4bf0c7b0-a56e-48e3-9cff-fee895fee262","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_07acbe56-05ec-4775-8e2f-124237955322","args":{"city":"New York"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_07acbe56-05ec-4775-8e2f-124237955322","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_e8099203-a6c9-49fe-a129-beefdce528d4","args":{"city":"London"},"name":"get_current_time"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_e8099203-a6c9-49fe-a129-beefdce528d4","name":"get_current_time","response":{"status":"error","error_message":"Sorry, I don't have timezone information for London."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_1d3dce7e-bdcb-4e91-aae8-bbc3f36dd42e","args":{"city":"London"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_1d3dce7e-bdcb-4e91-aae8-bbc3f36dd42e","name":"get_weather","response":{"status":"error","error_message":"Weather information for 'London' is not available."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_4d069082-2989-43d9-976b-1e0c19fce55b","args":{"city":"New York"},"name":"get_current_time"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_4d069082-2989-43d9-976b-1e0c19fce55b","name":"get_current_time","response":{"status":"success","report":"The current time in New York is 2025-04-12 17:52:11 EDT-0400"}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_87ca1a49-57b0-4ddd-ad33-6c47f1435870","args":{"city":"New York"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_87ca1a49-57b0-4ddd-ad33-6c47f1435870","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_2551c77c-69bb-4737-a361-e6ef36142bc7","args":{"city":"New York"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_2551c77c-69bb-4737-a361-e6ef36142bc7","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_ed7092d5-0fde-43d6-914c-53e2997699f6","args":{"city":"New York"},"name":"get_current_time"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_ed7092d5-0fde-43d6-914c-53e2997699f6","name":"get_current_time","response":{"status":"success","report":"The current time in New York is 2025-04-12 17:52:14 EDT-0400"}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_5164330b-9725-47ab-9b8e-c19bf8d3a453","args":{"city":"New York"},"name":"get_current_time"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_5164330b-9725-47ab-9b8e-c19bf8d3a453","name":"get_current_time","response":{"status":"success","report":"The current time in New York is 2025-04-12 17:52:15 EDT-0400"}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_1e2490f7-0f08-44d2-a384-8c30e14c4b4d","args":{"city":"New York"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_1e2490f7-0f08-44d2-a384-8c30e14c4b4d","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_ba915d1c-81df-4fd2-b669-5e2a4068f69c","args":{"city":"New York"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_ba915d1c-81df-4fd2-b669-5e2a4068f69c","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_f9dd164c-8667-4435-8fcb-f97373d84d7b","args":{"city":"London"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_f9dd164c-8667-4435-8fcb-f97373d84d7b","name":"get_weather","response":{"status":"error","error_message":"Weather information for 'London' is not available."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_82f344d4-66b3-483a-98a4-3e5d5a4f4262","args":{"city":"London"},"name":"get_current_time"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_82f344d4-66b3-483a-98a4-3e5d5a4f4262","name":"get_current_time","response":{"status":"error","error_message":"Sorry, I don't have timezone information for London."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_b3e4eaa1-de5c-4bdc-9b4b-dca158afac73","args":{"city":"New York"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_b3e4eaa1-de5c-4bdc-9b4b-dca158afac73","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_443eab7f-8dc1-4a63-91fc-e0ae752269a1","args":{"city":"New York"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_443eab7f-8dc1-4a63-91fc-e0ae752269a1","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_0208167b-da83-45f1-a775-48711869b923","args":{"city":"New York"},"name":"get_current_time"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_0208167b-da83-45f1-a775-48711869b923","name":"get_current_time","response":{"status":"success","report":"The current time in New York is 2025-04-12 17:52:21 EDT-0400"}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_1a01592c-7e5f-4cf8-b666-3d0bbd020ba9","args":{"city":"New York City"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_1a01592c-7e5f-4cf8-b666-3d0bbd020ba9","name":"get_weather","response":{"status":"error","error_message":"Weather information for 'New York City' is not available."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_ad6c3d0c-5f6d-45b8-939f-1f5a954120eb","args":{"city":"London"},"name":"get_current_time"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_ad6c3d0c-5f6d-45b8-939f-1f5a954120eb","name":"get_current_time","response":{"status":"error","error_message":"Sorry, I don't have timezone information for London."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_0ad0bd09-a2f0-49bf-a5f6-6827bcba945a","args":{"city":"New York"},"name":"get_weather"}}],"role":"model"}
{"parts":[{"function_response":{"id":"call_0ad0bd09-a2f0-49bf-a5f6-6827bcba945a","name":"get_weather","response":{"status":"success","report":"The weather in New York is sunny with a temperature of 25 degrees Celsius (41 degrees Fahrenheit)."}}}],"role":"user"}
{"parts":[{"function_call":{"id":"call_9bf1fa50-b14b-40e1-9647-b74d252536f8","args":{"city":"New York"},"name":"get_weather"}}],"role":"model"}

Is this a problem that occurs with any Ollama model?

I changed the model configuration line:

model=LiteLlm(model="ollama_chat/gemma:2b"),

Now the loop does not occur, but only returns an internal code in json.

@liatoutou

@liatoutou

I could reproduce the error in your logs on litellm v1.65.4. It works fine on 1.65.5 and the latest 1.65.8. Would you please try the latest version: pip install litellm==1.65.8 (and restart adk web/api_server)

@brunomcuesta

ollama template for gemma:2b does not have tools. you can choose another model on ollama site after enabling tools toggle. here is the link with tools filter on: https://ollama.com/search?c=tools

You can verify tool support by looking into model: ollama show <model_name> should show tools under capabilities.

You can also tweak the tool related instructions:

ollama show --modelfile <model_name> > model_file_to_tweak

ollama create <modified_model_name> -f model_file_to_tweak

@selcukgun

@selcukgun

With the latest version of liteLLM I got another error:

INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
2025-04-13 07:15:32,967 - INFO - fast_api.py:294 - New session created
INFO:     127.0.0.1:57236 - "GET /list-apps?relative_path=./ HTTP/1.1" 200 OK
INFO:     127.0.0.1:57238 - "POST /apps/nosana_agent/users/user/sessions HTTP/1.1" 200 OK
INFO:     127.0.0.1:57238 - "GET /apps/nosana_agent/eval_sets HTTP/1.1" 200 OK
INFO:     127.0.0.1:57236 - "GET /apps/nosana_agent/users/user/sessions HTTP/1.1" 200 OK
INFO:     127.0.0.1:57236 - "GET /apps/nosana_agent/users/user/sessions HTTP/1.1" 200 OK
INFO:     127.0.0.1:57236 - "POST /run_sse HTTP/1.1" 200 OK
2025-04-13 07:15:38,035 - INFO - envs.py:47 - Loaded .env file for nosana_agent at C:\Users\benni\Documents\nosana\demos\agent-sandbox\nosana_agent\.env
2025-04-13 07:15:38,035 - INFO - envs.py:53 - Reloaded .env file for nosana_agent at C:\Users\benni\Documents\nosana\demos\agent-sandbox\nosana_agent\.env
2025-04-13 07:15:59,188 - INFO - _client.py:1025 - HTTP Request: GET https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json "HTTP/1.1 200 OK"
2025-04-13 07:16:00,104 - INFO - lite_llm.py:597 - 
LLM Request:
-----------------------------------------------------------
System Instruction:
I can answer your questions about the time and weather in a city.

You are an agent. Your internal name is "weather_time_agent".

 The description about you is "Agent to answer questions about the time and weather in a city."
-----------------------------------------------------------
Contents:
{"parts":[{"text":"Hello there"}],"role":"user"}
-----------------------------------------------------------
Functions:
get_weather: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
get_current_time: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
-----------------------------------------------------------

07:16:00 - LiteLLM:INFO: utils.py:3085 - 
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-13 07:16:00,107 - INFO - utils.py:3085 -
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-13 07:16:02,173 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
07:16:06 - LiteLLM:INFO: cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-13 07:16:06,912 - INFO - cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-13 07:16:06,940 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-13 07:16:06,942 - INFO - fast_api.py:613 - Generated event in agent run streaming: {"content":{"parts":[{"text":"Hello, I can help you find the weather and the time in a city. Please let me know what you need."}],"role":"model"},"partial":false,"invocation_id":"e-607e7dc9-1615-409e-8474-9898fc69585e","author":"weather_time_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"id":"hXmnAxTw","timestamp":1744521360.104177}
2025-04-13 07:16:06,969 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
07:16:06 - LiteLLM:INFO: cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-13 07:16:06,970 - INFO - cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-13 07:16:06,999 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-13 07:16:07,025 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
INFO:     127.0.0.1:57236 - "GET /apps/nosana_agent/users/user/sessions/9a976a68-c46b-40ab-a108-20a8ca1d05df HTTP/1.1" 200 OK
INFO:     127.0.0.1:57239 - "POST /run_sse HTTP/1.1" 200 OK
2025-04-13 07:16:12,401 - INFO - lite_llm.py:597 - 
LLM Request:
-----------------------------------------------------------
System Instruction:
I can answer your questions about the time and weather in a city.

You are an agent. Your internal name is "weather_time_agent".

 The description about you is "Agent to answer questions about the time and weather in a city."
-----------------------------------------------------------
Contents:
{"parts":[{"text":"Hello there"}],"role":"user"}
{"parts":[{"text":"Hello, I can help you find the weather and the time in a city. Please let me know what you need."}],"role":"model"}
{"parts":[{"text":"Can you tell me the time?"}],"role":"user"}
-----------------------------------------------------------
Functions:
get_weather: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
get_current_time: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
-----------------------------------------------------------

07:16:12 - LiteLLM:INFO: utils.py:3085 -
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-13 07:16:12,401 - INFO - utils.py:3085 -
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-13 07:16:14,449 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
07:16:18 - LiteLLM:INFO: cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-13 07:16:18,444 - INFO - cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-13 07:16:18,471 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-13 07:16:18,472 - INFO - fast_api.py:613 - Generated event in agent run streaming: {"content":{"parts":[{"text":"Yes, please tell me in which city do you want to know the time?"}],"role":"model"},"partial":false,"invocation_id":"e-7ac2ec16-f4b2-4045-9cd9-bbc4ded1cd4e","author":"weather_time_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"id":"TuqSAtc7","timestamp":1744521372.401067}     
2025-04-13 07:16:18,499 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
07:16:18 - LiteLLM:INFO: cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-13 07:16:18,500 - INFO - cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-13 07:16:18,528 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-13 07:16:18,554 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
INFO:     127.0.0.1:57239 - "GET /apps/nosana_agent/users/user/sessions/9a976a68-c46b-40ab-a108-20a8ca1d05df HTTP/1.1" 200 OK
INFO:     127.0.0.1:57284 - "POST /run_sse HTTP/1.1" 200 OK
2025-04-13 07:16:35,864 - INFO - lite_llm.py:597 - 
LLM Request:
-----------------------------------------------------------
System Instruction:
I can answer your questions about the time and weather in a city.

You are an agent. Your internal name is "weather_time_agent".

 The description about you is "Agent to answer questions about the time and weather in a city."
-----------------------------------------------------------
Contents:
{"parts":[{"text":"Hello there"}],"role":"user"}
{"parts":[{"text":"Hello, I can help you find the weather and the time in a city. Please let me know what you need."}],"role":"model"}
{"parts":[{"text":"Can you tell me the time?"}],"role":"user"}
{"parts":[{"text":"Yes, please tell me in which city do you want to know the time?"}],"role":"model"}
{"parts":[{"text":"New york"}],"role":"user"}
-----------------------------------------------------------
Functions:
get_weather: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
get_current_time: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
-----------------------------------------------------------

07:16:35 - LiteLLM:INFO: utils.py:3085 -
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-13 07:16:35,864 - INFO - utils.py:3085 -
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-13 07:16:37,922 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
07:16:42 - LiteLLM:INFO: cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-13 07:16:42,396 - INFO - cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-13 07:16:42,423 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-13 07:16:42,424 - INFO - fast_api.py:613 - Generated event in agent run streaming: {"content":{"parts":[{"functionCall":{"id":"64bc51cb-5390-4bc3-bdbc-b1d0d48741e5","args":{"city":"new york"},"name":"get_current_time"}}],"role":"model"},"partial":false,"invocation_id":"e-958dcf12-7a84-47c5-9354-dab3d20ce79d","author":"weather_time_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"long_running_tool_ids":[],"id":"asMmYWVX","timestamp":1744521395.862999}
2025-04-13 07:16:42,428 - INFO - fast_api.py:613 - Generated event in agent run streaming: {"content":{"parts":[{"functionResponse":{"id":"64bc51cb-5390-4bc3-bdbc-b1d0d48741e5","name":"get_current_time","response":{"status":"success","report":"The current time in new york is 2025-04-13 01:16:42 EDT-0400"}}}],"role":"user"},"invocation_id":"e-958dcf12-7a84-47c5-9354-dab3d20ce79d","author":"weather_time_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"id":"2vg6lrWT","timestamp":1744521402.428904}
2025-04-13 07:16:42,429 - INFO - lite_llm.py:597 -
LLM Request:
-----------------------------------------------------------
System Instruction:
I can answer your questions about the time and weather in a city.

You are an agent. Your internal name is "weather_time_agent".

 The description about you is "Agent to answer questions about the time and weather in a city."
-----------------------------------------------------------
Contents:
{"parts":[{"text":"Hello there"}],"role":"user"}
{"parts":[{"text":"Hello, I can help you find the weather and the time in a city. Please let me know what you need."}],"role":"model"}
{"parts":[{"text":"Can you tell me the time?"}],"role":"user"}
{"parts":[{"text":"Yes, please tell me in which city do you want to know the time?"}],"role":"model"}
{"parts":[{"text":"New york"}],"role":"user"}
{"parts":[{"function_call":{"id":"64bc51cb-5390-4bc3-bdbc-b1d0d48741e5","args":{"city":"new york"},"name":"get_current_time"}}],"role":"model"}  
{"parts":[{"function_response":{"id":"64bc51cb-5390-4bc3-bdbc-b1d0d48741e5","name":"get_current_time","response":{"status":"success","report":"The current time in new york is 2025-04-13 01:16:42 EDT-0400"}}}],"role":"user"}
-----------------------------------------------------------
Functions:
get_weather: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
get_current_time: {'city': {'type': <Type.STRING: 'STRING'>}} -> None
-----------------------------------------------------------

07:16:42 - LiteLLM:INFO: utils.py:3085 -
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-13 07:16:42,430 - INFO - utils.py:3085 -
LiteLLM completion() model= mistral-small3.1; provider = ollama_chat
2025-04-13 07:16:42,458 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
07:16:42 - LiteLLM:INFO: cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-13 07:16:42,459 - INFO - cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/mistral-small3.1
2025-04-13 07:16:42,485 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-13 07:16:42,512 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-13 07:16:44,490 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

2025-04-13 07:16:44,756 - ERROR - fast_api.py:616 - Error in event_generator: litellm.APIConnectionError: Ollama_chatException - {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages.content of type string"}
Traceback (most recent call last):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\main.py", line 477, in acompletion
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\llms\ollama_chat.py", line 607, in ollama_acompletion 
    raise e  # don't use verbose_logger.exception, if exception is raised
    ^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\llms\ollama_chat.py", line 546, in ollama_acompletion 
    raise OllamaError(status_code=resp.status, message=text)
litellm.llms.ollama_chat.OllamaError: {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages.content of type string"}  

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\cli\fast_api.py", line 605, in event_generator     
    async for event in runner.run_async(
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\runners.py", line 197, in run_async
    async for event in invocation_context.agent.run_async(invocation_context):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\agents\base_agent.py", line 141, in run_async      
    async for event in self._run_async_impl(ctx):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\agents\llm_agent.py", line 232, in _run_async_impl 
    async for event in self._llm_flow.run_async(ctx):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\flows\llm_flows\base_llm_flow.py", line 231, in run_async
    async for event in self._run_one_step_async(invocation_context):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\flows\llm_flows\base_llm_flow.py", line 257, in _run_one_step_async
    async for llm_response in self._call_llm_async(
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\flows\llm_flows\base_llm_flow.py", line 470, in _call_llm_async
    async for llm_response in llm.generate_content_async(
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\models\lite_llm.py", line 658, in generate_content_async
    response = await self.llm_client.acompletion(**completion_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\google\adk\models\lite_llm.py", line 88, in acompletion       
    return await acompletion(
           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\utils.py", line 1452, in wrapper_async
    raise e
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\utils.py", line 1313, in wrapper_async
    result = await original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\main.py", line 496, in acompletion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\litellm_core_utils\exception_mapping_utils.py", line 2214, in exception_type
    raise e
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\litellm_core_utils\exception_mapping_utils.py", line 2183, in exception_type
    raise APIConnectionError(
litellm.exceptions.APIConnectionError: litellm.APIConnectionError: Ollama_chatException - {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages.content of type string"}

This is already fixed and merged: #102 but likely it did not make into the wheel yet.

you can use the latest:

git clone https://github.com/google/adk-python.git
cd adk-python
python3.11 -m venv venv
source ./venv/bin/activate
pip install flit wheel
flit install --pth-file --deps all

bring in your agent as usual.

@selcukgun

@selcukgun

Its working, thank you very much!

Also, I am not sure if its up to you, but in this section:

https://google.github.io/adk-docs/agents/models/#ollama-integration

It would be helpful to change model=LiteLlm(model="ollama/gemma:2b") to model=LiteLlm(model="ollama_chat/gemma:2b")

Just tested with llama3.1 and also worked with the manual process!

but then I tested with a more complex agent from the samples(travel_concierge) and got the error again and a new one:

itellm.exceptions.APIConnectionError: litellm.APIConnectionError: Ollama_chatException - {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages.content of type string"}
2025-04-13 12:00:36,458 - ERROR - __init__.py:157 - Failed to detach context
Traceback (most recent call last):
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/opentelemetry/trace/__init__.py", line 587, in use_span
    yield span
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/opentelemetry/sdk/trace/__init__.py", line 1105, in start_as_current_span
    yield span
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/opentelemetry/trace/__init__.py", line 452, in start_as_current_span
    yield span
  File "/Users/joao/projects/adk-python/src/google/adk/flows/llm_flows/base_llm_flow.py", line 487, in _call_llm_async
    yield llm_response
GeneratorExit
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/opentelemetry/context/__init__.py", line 155, in detach
    _RUNTIME_CONTEXT.detach(token)
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/opentelemetry/context/contextvars_context.py", line 53, in detach
    self._current_context.reset(token)
ValueError: <Token var=<ContextVar name='current_context' default={} at 0x1113ec090> at 0x157651940> was created in a different Context

To reproduce basically replace instances of

Agent(
    model="gemini-2.0-flash-001",

to

LlmAgent(
    model=LiteLlm(
        model='ollama_chat/llama3.1'
    ),

and import the modules.

Full log here:

INFO:     Started server process [9819]
INFO:     Waiting for application startup.
+-----------------------------------------------------------------------------+
| ADK Web Server started                                                      |
|                                                                             |
| For local testing, access at http://localhost:8000.                         |
+-----------------------------------------------------------------------------+

INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     127.0.0.1:59240 - "GET / HTTP/1.1" 307 Temporary Redirect
INFO:     127.0.0.1:59240 - "GET /dev-ui HTTP/1.1" 200 OK
INFO:     127.0.0.1:59240 - "GET /polyfills-FFHMD2TL.js HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:59241 - "GET /styles-4VDSPQ37.css HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:59240 - "GET /main-SLIAU2JL.js HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:59240 - "GET /assets/config/runtime-config.json HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:59240 - "GET /list-apps?relative_path=./ HTTP/1.1" 200 OK
2025-04-13 12:00:25,076 - INFO - fast_api.py:297 - New session created
INFO:     127.0.0.1:59240 - "POST /apps/travel_concierge/users/user/sessions HTTP/1.1" 200 OK
INFO:     127.0.0.1:59240 - "GET /apps/travel_concierge/eval_sets HTTP/1.1" 200 OK
INFO:     127.0.0.1:59240 - "GET /apps/travel_concierge/users/user/sessions HTTP/1.1" 200 OK
INFO:     127.0.0.1:59240 - "GET /apps/travel_concierge/users/user/sessions HTTP/1.1" 200 OK
INFO:     127.0.0.1:59242 - "POST /run_sse HTTP/1.1" 200 OK
2025-04-13 12:00:30,719 - INFO - envs.py:57 - No .env file found for travel_concierge
2025-04-13 12:00:31,073 - INFO - _client.py:1025 - HTTP Request: GET https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json "HTTP/1.1 200 OK"

Loading Initial State: {'state': {'user_profile': {'passport_nationality': 'US Citizen', 'seat_preference': 'window', 'food_preference': 'vegan', 'allergies': [], 'likes': [], 'dislikes': [], 'price_sensitivity': [], 'home': {'event_type': 'home', 'address': '6420 Sequence Dr #400, San Diego, CA 92121, United States', 'local_prefer_mode': 'drive'}}, 'itinerary': {}, 'origin': '', 'destination': '', 'start_date': '', 'end_date': '', 'outbound_flight_selection': '', 'outbound_seat_number': '', 'return_flight_selection': '', 'return_seat_number': '', 'hotel_selection': '', 'room_selection': '', 'poi': '', 'itinerary_datetime': '', 'itinerary_start_date': '', 'itinerary_end_date': ''}}

2025-04-13 12:00:31,371 - INFO - fast_api.py:616 - Generated event in agent run streaming: {"invocation_id":"e-0db2d892-6269-4b13-8fd1-100de18f3ccc","author":"root_agent","actions":{"state_delta":{"_time":"2025-04-13 12:00:31.371157","_itin_initialized":true,"user_profile":{"passport_nationality":"US Citizen","seat_preference":"window","food_preference":"vegan","allergies":[],"likes":[],"dislikes":[],"price_sensitivity":[],"home":{"event_type":"home","address":"6420 Sequence Dr #400, San Diego, CA 92121, United States","local_prefer_mode":"drive"}},"itinerary":{},"origin":"","destination":"","start_date":"","end_date":"","outbound_flight_selection":"","outbound_seat_number":"","return_flight_selection":"","return_seat_number":"","hotel_selection":"","room_selection":"","poi":"","itinerary_datetime":"","itinerary_start_date":"","itinerary_end_date":""},"artifact_delta":{},"requested_auth_configs":{}},"id":"KH4W9HTt","timestamp":1744538431.371169}
2025-04-13 12:00:31,371 - INFO - lite_llm.py:599 - 
LLM Request:
-----------------------------------------------------------
System Instruction:

- You are a exclusive travel conceirge agent
- You help users to discover their dream vacation, planning for the vacation, book flights and hotels
- You want to gather a minimal information to help the user
- After every tool call, pretend you're showing the result to the user and keep your response limited to a phrase.
- Please use only the agents and tools to fulfill all user rquest
- If the user asks about general knowledge, vacation inspiration or things to do, transfer to the agent `inspiration_agent`
- If the user asks about finding flight deals, making seat selection, or lodging, transfer to the agent `planning_agent`
- If the user is ready to make the flight booking or process payments, transfer to the agent `booking_agent`
- Please use the context info below for any user preferences
               
Current user:
  <user_profile>
  {'passport_nationality': 'US Citizen', 'seat_preference': 'window', 'food_preference': 'vegan', 'allergies': [], 'likes': [], 'dislikes': [], 'price_sensitivity': [], 'home': {'event_type': 'home', 'address': '6420 Sequence Dr #400, San Diego, CA 92121, United States', 'local_prefer_mode': 'drive'}}
  </user_profile>

Current time: 2025-04-13 12:00:31.371157
      
Trip phases:
If we have a non-empty itinerary, follow the following logic to deteermine a Trip phase:
- First focus on the start_date "" and the end_date "" of the itinerary.
- if "" is before the start date "" of the trip, we are in the "pre_trip" phase. 
- if "" is between the start date "" and end date "" of the trip, we are in the "in_trip" phase. 
- When we are in the "in_trip" phase, the "" dictates if we have "day_of" matters to handle.
- if "" is after the end date of the trip, we are in the "post_trip" phase. 

<itinerary>
{}
</itinerary>

Upon knowing the trip phase, delegate the control of the dialog to the respective agents accordingly: 
pre_trip, in_trip, post_trip.


You are an agent. Your internal name is "root_agent".

 The description about you is "A Travel Conceirge using the services of multiple sub-agents"


You have a list of other agents to transfer to:


Agent name: inspiration_agent
Agent description: A travel inspiration agent who inspire users, and discover their next vacations; Provide information about places, activities, interests,


Agent name: planning_agent
Agent description: Helps users with travel planning, complete a full itinerary for their vacation, finding best deals for flights and hotels.


Agent name: booking_agent
Agent description: Given an itinerary, complete the bookings of items by handling payment choices and processing.


Agent name: pre_trip_agent
Agent description: Given an itinerary, this agent keeps up to date and provides relevant travel information to the user before the trip.


Agent name: in_trip_agent
Agent description: Provide information about what the users need as part of the tour.


Agent name: post_trip_agent
Agent description: A follow up agent to learn from user's experience; In turn improves the user's future trips planning and in-trip experience.


If you are the best to answer the question according to your description, you
can answer it.

If another agent is better for answering the question according to its
description, call `transfer_to_agent` function to transfer the
question to that agent. When transfering, do not generate any text other than
the function call.

-----------------------------------------------------------
Contents:
{"parts":[{"text":"help"}],"role":"user"}
-----------------------------------------------------------
Functions:
transfer_to_agent: {'agent_name': {'type': <Type.STRING: 'STRING'>}} -> None
-----------------------------------------------------------

12:00:31 - LiteLLM:INFO: utils.py:3085 - 
LiteLLM completion() model= llama3.1; provider = ollama_chat
2025-04-13 12:00:31,373 - INFO - utils.py:3085 - 
LiteLLM completion() model= llama3.1; provider = ollama_chat
2025-04-13 12:00:31,403 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
12:00:36 - LiteLLM:INFO: cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/llama3.1
2025-04-13 12:00:36,300 - INFO - cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/llama3.1
2025-04-13 12:00:36,321 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-13 12:00:36,324 - INFO - fast_api.py:616 - Generated event in agent run streaming: {"content":{"parts":[{"functionCall":{"id":"0c4587d5-893c-471e-a3b7-3342a7cbac74","args":{"agent_name":"inspiration_agent"},"name":"transfer_to_agent"}}],"role":"model"},"partial":false,"invocation_id":"e-0db2d892-6269-4b13-8fd1-100de18f3ccc","author":"root_agent","actions":{"state_delta":{},"artifact_delta":{},"requested_auth_configs":{}},"long_running_tool_ids":[],"id":"RsW1lZLV","timestamp":1744538431.371472}
2025-04-13 12:00:36,326 - INFO - fast_api.py:616 - Generated event in agent run streaming: {"content":{"parts":[{"functionResponse":{"id":"0c4587d5-893c-471e-a3b7-3342a7cbac74","name":"transfer_to_agent","response":{}}}],"role":"user"},"invocation_id":"e-0db2d892-6269-4b13-8fd1-100de18f3ccc","author":"root_agent","actions":{"state_delta":{},"artifact_delta":{},"transfer_to_agent":"inspiration_agent","requested_auth_configs":{}},"id":"8etKc3cb","timestamp":1744538436.326258}
2025-04-13 12:00:36,329 - INFO - lite_llm.py:599 - 
LLM Request:
-----------------------------------------------------------
System Instruction:

You are travel inspiration agent who help users find their next big dream vacation destinations.
Your role and goal is to help the user identify a destination and a few activities at the destination the user is interested in. 

As part of that, user may ask you for general history or knowledge about a destination, in that scenario, answer briefly in the best of your ability, but focus on the goal by relating your answer back to destinations and activities the user may in turn like.
- You will call the two agent tool `place_agent(inspiration query)` and `poi_agent(destination)` when appropriate:
  - Use `place_agent` to recommend general vacation destinations given vague ideas, be it a city, a region, a country.
  - Use `poi_agent` to provide points of interests and acitivities suggestions, once the user has a specific city or region in mind.
  - Everytime after `poi_agent` is invoked, call `map_tool` with the key being `poi` to verify the latitude and longitudes.
- Avoid asking too many questions. When user gives instructions like "inspire me", or "suggest some", just go ahead and call `place_agent`.
- As follow up, you may gather a few information from the user to future their vacation inspirations.
- Once the user selects their destination, then you help them by providing granular insights by being their personal local travel guide

- Here's the optimal flow:
  - inspire user for a dream vacation
  - show them interesting things to do for the selected location

- Your role is only to identify possible destinations and acitivites. 
- Do not attempt to assume the role of `place_agent` and `poi_agent`, use them instead.
- Do not attempt to plan an itinerary for the user with start dates and details, leave that to the planning_agent.
- Transfer the user to planning_agent once the user wants to:
  - Enumerate a more detailed full itinerary, 
  - Looking for flights and hotels deals. 

- Please use the context info below for any user preferences:
Current user:
  <user_profile>
  {'passport_nationality': 'US Citizen', 'seat_preference': 'window', 'food_preference': 'vegan', 'allergies': [], 'likes': [], 'dislikes': [], 'price_sensitivity': [], 'home': {'event_type': 'home', 'address': '6420 Sequence Dr #400, San Diego, CA 92121, United States', 'local_prefer_mode': 'drive'}}
  </user_profile>

Current time: 2025-04-13 12:00:31.371157


You are an agent. Your internal name is "inspiration_agent".

 The description about you is "A travel inspiration agent who inspire users, and discover their next vacations; Provide information about places, activities, interests,"


You have a list of other agents to transfer to:


Agent name: root_agent
Agent description: A Travel Conceirge using the services of multiple sub-agents


Agent name: planning_agent
Agent description: Helps users with travel planning, complete a full itinerary for their vacation, finding best deals for flights and hotels.


Agent name: booking_agent
Agent description: Given an itinerary, complete the bookings of items by handling payment choices and processing.


Agent name: pre_trip_agent
Agent description: Given an itinerary, this agent keeps up to date and provides relevant travel information to the user before the trip.


Agent name: in_trip_agent
Agent description: Provide information about what the users need as part of the tour.


Agent name: post_trip_agent
Agent description: A follow up agent to learn from user's experience; In turn improves the user's future trips planning and in-trip experience.


If you are the best to answer the question according to your description, you
can answer it.

If another agent is better for answering the question according to its
description, call `transfer_to_agent` function to transfer the
question to that agent. When transfering, do not generate any text other than
the function call.

Your parent agent is root_agent. If neither the other agents nor
you are best for answering the question according to the descriptions, transfer
to your parent agent. If you don't have parent agent, try answer by yourself.

-----------------------------------------------------------
Contents:
{"parts":[{"text":"help"}],"role":"user"}
{"parts":[{"text":"For context:"},{"text":"[root_agent] called tool `transfer_to_agent` with parameters: {'agent_name': 'inspiration_agent'}"}],"role":"user"}
{"parts":[{"text":"For context:"},{"text":"[root_agent] `transfer_to_agent` tool returned result: {}"}],"role":"user"}
-----------------------------------------------------------
Functions:
transfer_to_agent: {'agent_name': {'type': <Type.STRING: 'STRING'>}} -> None
place_agent: {'request': {'type': <Type.STRING: 'STRING'>}} -> None
poi_agent: {'request': {'type': <Type.STRING: 'STRING'>}} -> None
map_tool: {'key': {'type': <Type.STRING: 'STRING'>}} -> None
-----------------------------------------------------------

12:00:36 - LiteLLM:INFO: utils.py:3085 - 
LiteLLM completion() model= llama3.1; provider = ollama_chat
2025-04-13 12:00:36,337 - INFO - utils.py:3085 - 
LiteLLM completion() model= llama3.1; provider = ollama_chat
2025-04-13 12:00:36,356 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-13 12:00:36,356 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
12:00:36 - LiteLLM:INFO: cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/llama3.1
2025-04-13 12:00:36,360 - INFO - cost_calculator.py:636 - selected model name for cost calculation: ollama_chat/llama3.1
2025-04-13 12:00:36,394 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"
2025-04-13 12:00:36,422 - INFO - _client.py:1025 - HTTP Request: POST http://localhost:11434/api/show "HTTP/1.1 200 OK"

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

2025-04-13 12:00:36,455 - ERROR - fast_api.py:619 - Error in event_generator: litellm.APIConnectionError: Ollama_chatException - {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages.content of type string"}
Traceback (most recent call last):
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/litellm/main.py", line 477, in acompletion
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/litellm/llms/ollama_chat.py", line 607, in ollama_acompletion
    raise e  # don't use verbose_logger.exception, if exception is raised
    ^^^^^^^
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/litellm/llms/ollama_chat.py", line 546, in ollama_acompletion
    raise OllamaError(status_code=resp.status, message=text)
litellm.llms.ollama_chat.OllamaError: {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages.content of type string"}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/joao/projects/adk-python/src/google/adk/cli/fast_api.py", line 608, in event_generator
    async for event in runner.run_async(
  File "/Users/joao/projects/adk-python/src/google/adk/runners.py", line 197, in run_async
    async for event in invocation_context.agent.run_async(invocation_context):
  File "/Users/joao/projects/adk-python/src/google/adk/agents/base_agent.py", line 141, in run_async
    async for event in self._run_async_impl(ctx):
  File "/Users/joao/projects/adk-python/src/google/adk/agents/llm_agent.py", line 232, in _run_async_impl
    async for event in self._llm_flow.run_async(ctx):
  File "/Users/joao/projects/adk-python/src/google/adk/flows/llm_flows/base_llm_flow.py", line 231, in run_async
    async for event in self._run_one_step_async(invocation_context):
  File "/Users/joao/projects/adk-python/src/google/adk/flows/llm_flows/base_llm_flow.py", line 261, in _run_one_step_async
    async for event in self._postprocess_async(
  File "/Users/joao/projects/adk-python/src/google/adk/flows/llm_flows/base_llm_flow.py", line 329, in _postprocess_async
    async for event in self._postprocess_handle_function_calls_async(
  File "/Users/joao/projects/adk-python/src/google/adk/flows/llm_flows/base_llm_flow.py", line 419, in _postprocess_handle_function_calls_async
    async for event in agent_to_run.run_async(invocation_context):
  File "/Users/joao/projects/adk-python/src/google/adk/agents/base_agent.py", line 141, in run_async
    async for event in self._run_async_impl(ctx):
  File "/Users/joao/projects/adk-python/src/google/adk/agents/llm_agent.py", line 232, in _run_async_impl
    async for event in self._llm_flow.run_async(ctx):
  File "/Users/joao/projects/adk-python/src/google/adk/flows/llm_flows/base_llm_flow.py", line 231, in run_async
    async for event in self._run_one_step_async(invocation_context):
  File "/Users/joao/projects/adk-python/src/google/adk/flows/llm_flows/base_llm_flow.py", line 257, in _run_one_step_async
    async for llm_response in self._call_llm_async(
  File "/Users/joao/projects/adk-python/src/google/adk/flows/llm_flows/base_llm_flow.py", line 470, in _call_llm_async
    async for llm_response in llm.generate_content_async(
  File "/Users/joao/projects/adk-python/src/google/adk/models/lite_llm.py", line 660, in generate_content_async
    response = await self.llm_client.acompletion(**completion_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joao/projects/adk-python/src/google/adk/models/lite_llm.py", line 88, in acompletion
    return await acompletion(
           ^^^^^^^^^^^^^^^^^^
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/litellm/utils.py", line 1452, in wrapper_async
    raise e
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/litellm/utils.py", line 1313, in wrapper_async
    result = await original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/litellm/main.py", line 496, in acompletion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2214, in exception_type
    raise e
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2183, in exception_type
    raise APIConnectionError(
litellm.exceptions.APIConnectionError: litellm.APIConnectionError: Ollama_chatException - {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages.content of type string"}
2025-04-13 12:00:36,458 - ERROR - __init__.py:157 - Failed to detach context
Traceback (most recent call last):
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/opentelemetry/trace/__init__.py", line 587, in use_span
    yield span
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/opentelemetry/sdk/trace/__init__.py", line 1105, in start_as_current_span
    yield span
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/opentelemetry/trace/__init__.py", line 452, in start_as_current_span
    yield span
  File "/Users/joao/projects/adk-python/src/google/adk/flows/llm_flows/base_llm_flow.py", line 487, in _call_llm_async
    yield llm_response
GeneratorExit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/opentelemetry/context/__init__.py", line 155, in detach
    _RUNTIME_CONTEXT.detach(token)
  File "/Users/joao/projects/adk-python/venv/lib/python3.11/site-packages/opentelemetry/context/contextvars_context.py", line 53, in detach
    self._current_context.reset(token)
ValueError: <Token var=<ContextVar name='current_context' default={} at 0x1113ec090> at 0x157651940> was created in a different Context

Image:

@selcukgun

@selcukgun

I also have the issue again in a more complex setting. This is my code from the tutorial:

import os
import asyncio
from google.adk.agents import Agent
from google.adk.models.lite_llm import LiteLlm
from google.adk.sessions import InMemorySessionService
from google.adk.runners import Runner
from google.genai import types

def get_weather(city: str) -> dict:
    """Retrieves the current weather report for a specified city.

    Args:
        city (str): The name of the city (e.g., "New York", "London", "Tokyo").

    Returns:
        dict: A dictionary containing the weather information.
              Includes a 'status' key ('success' or 'error').
              If 'success', includes a 'report' key with weather details.
              If 'error', includes an 'error_message' key.
    """
    # Best Practice: Log tool execution for easier debugging
    print(f"--- Tool: get_weather called for city: {city} ---")
    city_normalized = city.lower().replace(" ", "") # Basic input normalization

    # Mock weather data for simplicity
    mock_weather_db = {
        "newyork": {"status": "success", "report": "The weather in New York is sunny with a temperature of 25°C."},
        "london": {"status": "success", "report": "It's cloudy in London with a temperature of 15°C."},
        "tokyo": {"status": "success", "report": "Tokyo is experiencing light rain and a temperature of 18°C."},
    }

    # Best Practice: Handle potential errors gracefully within the tool
    if city_normalized in mock_weather_db:
        return mock_weather_db[city_normalized]
    else:
        return {"status": "error", "error_message": f"Sorry, I don't have weather information for '{city}'."}

def say_hello(name: str = "there") -> str:
    """Provides a simple greeting, optionally addressing the user by name.

    Args:
        name (str, optional): The name of the person to greet. Defaults to "there".

    Returns:
        str: A friendly greeting message.
    """
    print(f"--- Tool: say_hello called with name: {name} ---")
    return f"Hello, {name}!"

def say_goodbye() -> str:
    """Provides a simple farewell message to conclude the conversation."""
    print(f"--- Tool: say_goodbye called ---")
    return "Goodbye! Have a great day."

greeting_agent = Agent(
    model=LiteLlm(
        model='ollama_chat/mistral-small3.1'
    ),
    name="greeting_agent",
    instruction="You are the Greeting Agent. Your ONLY task is to provide a friendly greeting to the user. "
                "Use the 'say_hello' tool to generate the greeting. "
                "If the user provides their name, make sure to pass it to the tool. "
                "Do not engage in any other conversation or tasks.",
    description="Handles simple greetings and hellos using the 'say_hello' tool.",
    tools=[say_hello],
)

farewell_agent = Agent(
    model=LiteLlm(
        model='ollama_chat/mistral-small3.1'
    ),
    name="farewell_agent",
    instruction="You are the Farewell Agent. Your ONLY task is to provide a polite goodbye message. "
                "Use the 'say_goodbye' tool when the user indicates they are leaving or ending the conversation "
                "(e.g., using words like 'bye', 'goodbye', 'thanks bye', 'see you'). "
                "Do not perform any other actions.",
    description="Handles simple farewells and goodbyes using the 'say_goodbye' tool.",
    tools=[say_goodbye],
)

weather_agent_team = Agent(
    name="weather_agent_v2",
    model=LiteLlm(
        model='ollama_chat/mistral-small3.1'
    ),
    description="The main coordinator agent. Handles weather requests and delegates greetings/farewells to specialists.",
    instruction="You are the main Weather Agent coordinating a team. Your primary responsibility is to provide weather information. "
                "Use the 'get_weather' tool ONLY for specific weather requests (e.g., 'weather in London'). "
                "You have specialized sub-agents: "
                "1. 'greeting_agent': Handles simple greetings like 'Hi', 'Hello'. Delegate to it for these. "
                "2. 'farewell_agent': Handles simple farewells like 'Bye', 'See you'. Delegate to it for these. "
                "Analyze the user's query. If it's a greeting, delegate to 'greeting_agent'. If it's a farewell, delegate to 'farewell_agent'. "
                "If it's a weather request, handle it yourself using 'get_weather'. "
                "For anything else, respond appropriately or state you cannot handle it.",
    tools=[get_weather],
    sub_agents=[greeting_agent, farewell_agent]
)

async def run_team_conversation():
    print("\n--- Testing Agent Team Delegation ---")
    # InMemorySessionService is simple, non-persistent storage
    session_service = InMemorySessionService()

    # Define constants for identifying the interaction context
    APP_NAME = "weather_tutorial_agent_team"
    USER_ID = "user_1_agent_team"
    SESSION_ID = "session_001_agent_team"

    # Create the specific session where the conversation will happen
    session = session_service.create_session(
        app_name=APP_NAME,
        user_id=USER_ID,
        session_id=SESSION_ID
    )
    print(f"Session created: App='{APP_NAME}', User='{USER_ID}', Session='{SESSION_ID}'")

    # --- Get the actual root agent object ---
    # Use the determined variable name
    actual_root_agent = weather_agent_team

    # Create a runner specific to this agent team test
    runner_agent_team = Runner(
        agent=actual_root_agent,
        app_name=APP_NAME,
        session_service=session_service
        )
    # Corrected print statement to show the actual root agent's name
    print(f"Runner created for agent '{actual_root_agent.name}'.")

    # Always interact via the root agent's runner, passing the correct IDs
    await call_agent_async(query = "Hello there!",
                            runner=runner_agent_team,
                            user_id=USER_ID,
                            session_id=SESSION_ID)
    await call_agent_async(query = "What is the weather in New York?",
                            runner=runner_agent_team,
                            user_id=USER_ID,
                            session_id=SESSION_ID)
    await call_agent_async(query = "Thanks, bye!",
                            runner=runner_agent_team,
                            user_id=USER_ID,
                            session_id=SESSION_ID)

async def call_agent_async(query: str, runner: Runner, user_id: str, session_id: str):
  """Sends a query to the agent and prints the final response."""
  print(f"\n>>> User Query: {query}")

  # Prepare the user's message in ADK format
  content = types.Content(role='user', parts=[types.Part(text=query)])

  final_response_text = "Agent did not produce a final response." # Default

  # Key Concept: run_async executes the agent logic and yields Events.
  # We iterate through events to find the final answer.
  async for event in runner.run_async(user_id=user_id, session_id=session_id, new_message=content):
      # You can uncomment the line below to see *all* events during execution
      # print(f"  [Event] Author: {event.author}, Type: {type(event).__name__}, Final: {event.is_final_response()}, Content: {event.content}")

      # Key Concept: is_final_response() marks the concluding message for the turn.
      if event.is_final_response():
          if event.content and event.content.parts:
             # Assuming text response in the first part
             final_response_text = event.content.parts[0].text
          elif event.actions and event.actions.escalate: # Handle potential errors/escalations
             final_response_text = f"Agent escalated: {event.error_message or 'No specific message.'}"
          # Add more checks here if needed (e.g., specific error codes)
          break # Stop processing events once the final response is found

  print(f"<<< Agent Response: {final_response_text}")

async def main():
    await run_team_conversation()

if __name__ == "__main__":
    asyncio.run(main())

These are the logs:

--- Testing Agent Team Delegation ---
Session created: App='weather_tutorial_agent_team', User='user_1_agent_team', Session='session_001_agent_team'
Runner created for agent 'weather_agent_v2'.

>>> User Query: Hello there!
Default value is not supported in function declaration schema for Google AI.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

Traceback (most recent call last):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\main.py", line 477, in acompletion
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\llms\ollama_chat.py", line 607, in ollama_acompletion 
    raise e  # don't use verbose_logger.exception, if exception is raised
    ^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\llms\ollama_chat.py", line 546, in ollama_acompletion 
    raise OllamaError(status_code=resp.status, message=text)
litellm.llms.ollama_chat.OllamaError: {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages.content of type string"}  

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\nosana_agent\agent.py", line 175, in <module>
    asyncio.run(main())
  File "C:\Users\benni\miniconda3\Lib\asyncio\runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\benni\miniconda3\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\miniconda3\Lib\asyncio\base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\nosana_agent\agent.py", line 172, in main
    await run_team_conversation()
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\nosana_agent\agent.py", line 131, in run_team_conversation
    await call_agent_async(query = "Hello there!",
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\nosana_agent\agent.py", line 155, in call_agent_async
    async for event in runner.run_async(user_id=user_id, session_id=session_id, new_message=content):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\adk-python\src\google\adk\runners.py", line 197, in run_async
    async for event in invocation_context.agent.run_async(invocation_context):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\adk-python\src\google\adk\agents\base_agent.py", line 141, in run_async
    async for event in self._run_async_impl(ctx):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\adk-python\src\google\adk\agents\llm_agent.py", line 232, in _run_async_impl
    async for event in self._llm_flow.run_async(ctx):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\adk-python\src\google\adk\flows\llm_flows\base_llm_flow.py", line 231, in run_async  
    async for event in self._run_one_step_async(invocation_context):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\adk-python\src\google\adk\flows\llm_flows\base_llm_flow.py", line 261, in _run_one_step_async
    async for event in self._postprocess_async(
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\adk-python\src\google\adk\flows\llm_flows\base_llm_flow.py", line 329, in _postprocess_async
    async for event in self._postprocess_handle_function_calls_async(
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\adk-python\src\google\adk\flows\llm_flows\base_llm_flow.py", line 419, in _postprocess_handle_function_calls_async
    async for event in agent_to_run.run_async(invocation_context):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\adk-python\src\google\adk\agents\base_agent.py", line 141, in run_async
    async for event in self._run_async_impl(ctx):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\adk-python\src\google\adk\agents\llm_agent.py", line 232, in _run_async_impl
    async for event in self._llm_flow.run_async(ctx):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\adk-python\src\google\adk\flows\llm_flows\base_llm_flow.py", line 231, in run_async  
    async for event in self._run_one_step_async(invocation_context):
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\adk-python\src\google\adk\flows\llm_flows\base_llm_flow.py", line 257, in _run_one_step_async
    async for llm_response in self._call_llm_async(
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\adk-python\src\google\adk\flows\llm_flows\base_llm_flow.py", line 470, in _call_llm_async
    async for llm_response in llm.generate_content_async(
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\adk-python\src\google\adk\models\lite_llm.py", line 660, in generate_content_async   
    response = await self.llm_client.acompletion(**completion_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\adk-python\src\google\adk\models\lite_llm.py", line 88, in acompletion
    return await acompletion(
           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\utils.py", line 1452, in wrapper_async
    raise e
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\utils.py", line 1313, in wrapper_async
    result = await original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\main.py", line 496, in acompletion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\litellm_core_utils\exception_mapping_utils.py", line 2214, in exception_type
    raise e
  File "C:\Users\benni\Documents\nosana\demos\agent-sandbox\venv\Lib\site-packages\litellm\litellm_core_utils\exception_mapping_utils.py", line 2183, in exception_type
    raise APIConnectionError(
litellm.exceptions.APIConnectionError: litellm.APIConnectionError: Ollama_chatException - {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages.content of type string"}```

For now I fixed it by replacing _get_content() with this:

def _get_content(
    parts: Iterable[types.Part],
) -> Optional[str]:
    """Converts a list of parts to a single string content, suitable for models
    expecting string content like Ollama via LiteLLM. Concatenates text parts.

    Args:
        parts: The parts to convert.

    Returns:
        A single string concatenating text parts, or None if no text parts exist.
    """
    all_text = []
    has_text = False
    for part in parts:
        if part.text:
            all_text.append(part.text)
            has_text = True

    if not has_text:
        return None

    # Join all extracted text parts into a single string.
    # Using newline as a separator, adjust if space or other separator is preferred.
    return "\n".join(all_text)

But I dont believe this should be the real solution

Hello!
I made the suggested changes and it didn't work.

This is the model configuration with tools support:

model=LiteLlm(model="ollama_chat/llama3.1:8b"),

This is the error output:

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

2025-04-14 21:47:14,676 - ERROR - fast_api.py:616 - Error in event_generator: litellm.APIConnectionError: Ollama_chatException - {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages.content of type string"}
Traceback (most recent call last):
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/litellm/main.py", line 477, in acompletion
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/litellm/llms/ollama_chat.py", line 607, in ollama_acompletion
    raise e  # don't use verbose_logger.exception, if exception is raised
    ^^^^^^^
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/litellm/llms/ollama_chat.py", line 546, in ollama_acompletion
    raise OllamaError(status_code=resp.status, message=text)
litellm.llms.ollama_chat.OllamaError: {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages.content of type string"}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/google/adk/cli/fast_api.py", line 605, in event_generator
    async for event in runner.run_async(
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/google/adk/runners.py", line 197, in run_async
    async for event in invocation_context.agent.run_async(invocation_context):
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/google/adk/agents/base_agent.py", line 141, in run_async
    async for event in self._run_async_impl(ctx):
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/google/adk/agents/llm_agent.py", line 232, in _run_async_impl
    async for event in self._llm_flow.run_async(ctx):
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 231, in run_async
    async for event in self._run_one_step_async(invocation_context):
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 257, in _run_one_step_async
    async for llm_response in self._call_llm_async(
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 470, in _call_llm_async
    async for llm_response in llm.generate_content_async(
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/google/adk/models/lite_llm.py", line 658, in generate_content_async
    response = await self.llm_client.acompletion(**completion_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/google/adk/models/lite_llm.py", line 88, in acompletion
    return await acompletion(
           ^^^^^^^^^^^^^^^^^^
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/litellm/utils.py", line 1452, in wrapper_async
    raise e
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/litellm/utils.py", line 1313, in wrapper_async
    result = await original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/litellm/main.py", line 496, in acompletion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2214, in exception_type
    raise e
  File "/home/bruno/Development/AI/google-agent/.env/lib/python3.11/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2183, in exception_type
    raise APIConnectionError(
litellm.exceptions.APIConnectionError: litellm.APIConnectionError: Ollama_chatException - {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages.content of type string"}

This is fixed in the latest wheel. Please let us know if you still see this issue.

@hangfei

@hangfei Hello!
The latest update fixed the issue.
Thanks!

@hangfei Hello! The latest update fixed the issue. Thanks!

How did you applied the fix? git update?

@hangfei Hello! The latest update fixed the issue. Thanks!

How did you applied the fix? git update?

Hello!
I removed my venv environment and installed the google-adk package again to get the update.

python3 -m venv .env
source .env/bin/activate
python -m pip install --upgrade pip
pip install google-adk

If you need litellm too:

pip install litellm

Or you can do the update directly inside your .env:

pip install --upgrade google-adk

I am seeing the same error on 0.5.0

I'm also having the same issue. The agent is endlessly calling the tool.

Hi guys,
I struggled with that too, and found a solution.
Use the openai provider instead of ollama. Still use ollama, but tell LiteLlm that the provider is openai.

Here is an agent i wrote that was looping, but now works fine :

os.environ["OPENAI_API_BASE"] = os.getenv("OPENAI_BASE_URL")
llm_agent = LlmAgent(
        name="my_agent",
        model=LiteLlm(model="openai/"+OLLAMA_TOOL_CHAT_MODEL),
        tools=[
            extract_issue_keys
        ],
        description="Agent specialized in....",
        output_key="issue_keys",
        instruction=system_prompt
    )

And in my .env file :

# Ollama url
OPENAI_BASE_URL=http://192.168.1.200:11434/v1

What is the conclusion for this error?
I am getting the same looping error even after installing again from the wheel

This is already fixed and merged: #102 but likely it did not make into the wheel yet.

you can use the latest:

git clone https://github.com/google/adk-python.git
cd adk-python
python3.11 -m venv venv
source ./venv/bin/activate
pip install flit wheel
flit install --pth-file --deps all
bring in your agent as usual.

Hello all,
Just forget about the ollama and ollama_chat providers and use the openai/ provider. the ollamas providers are so buggy and you cant do streaming and tool calling at the same time.
Any other tool calling bugs like empty responses, those were bugs in Ollama that are fixed in the 0.9.4. Just update your Ollama server.

Navigation Menu

Repository navigation

Ollama models endlessly loop instead of respond #81

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Ollama models endlessly loop instead of respond #81

Description

Activity

luongnv89 commented on Apr 11, 2025

hangfei commented on Apr 12, 2025

selcukgun commented on Apr 12, 2025

liatoutou commented on Apr 12, 2025

liatoutou commented on Apr 12, 2025

brunomcuesta commented on Apr 12, 2025

brunomcuesta commented on Apr 12, 2025

selcukgun commented on Apr 13, 2025

liatoutou commented on Apr 13, 2025

selcukgun commented on Apr 13, 2025

liatoutou commented on Apr 13, 2025

INFINIVagas commented on Apr 13, 2025

INFINIVagas commented on Apr 13, 2025

liatoutou commented on Apr 13, 2025

liatoutou commented on Apr 13, 2025

brunomcuesta commented on Apr 15, 2025

hangfei commented on Apr 24, 2025

brunomcuesta commented on Apr 24, 2025

burak-aksoy commented on Apr 26, 2025

brunomcuesta commented on Apr 28, 2025

ankitdave06 commented on May 18, 2025

idling-mind commented on May 20, 2025

fesnault commented on Jun 4, 2025

krkarthik2006 commented on Jun 25, 2025

GreazySpoon commented on Jul 2, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions