[BUG]: Before load_artifacts, Incorrect file_uri handling for OpenAI and Claude/Anthropic providers in LiteLLM

Describe the bug

This issue occurs specifically when using the SaveFilesAsArtifactsPlugin. Before the agent calls load_artifacts to load artifacts into the session, the artifact file URIs are included in the message content as file_data.file_uri. When these messages are sent to OpenAI or Claude models, the incorrect format causes the errors described below.

The _get_content function in google.adk.models.lite_llm incorrectly handles file_data.file_uri (GCS URIs) for OpenAI and Claude/Anthropic providers. The current implementation sends GCS URIs directly as file_id, which causes two distinct errors:

OpenAI/Azure: GCS URIs cannot be used as file_id. Files must be uploaded to OpenAI's API first to obtain a valid file_id.
Claude/Anthropic via Vertex AI: The API rejects source.type: 'file' format. It expects source.type: 'url' or source.type: 'base64' instead.

The current code treats all providers the same way, sending GCS URIs directly without provider-specific conversion.

To Reproduce

Example 1: Claude/Anthropic Request (Failing)

This is the actual curl request sent by LiteLLM that causes the error:

curl -X POST \
https://us-east5-aiplatform.googleapis.com/v1/projects/hubx-sdxl-api/locations/us-east5/publishers/anthropic/models/claude-sonnet-4@20250514:streamRawPredict \
-H 'Authorization: Be****gF' -H 'anthropic-version: 2023-06-01' -H 'x-api-key: ya****gF' -H 'accept: application/json' -H 'content-type: application/json' \
-d '{
  "messages": [{
    "role": "user", 
    "content": [
      {"type": "text", "text": "bu pdf ne anlatıyor ? akademik olarak açıklayabilir misin ?"}, 
      {"type": "text", "text": "[Uploaded Artifact: \"artifact_e-759bdd7d-7722-4500-8a4e-ec9fc5884cf0_1\"]"}, 
      {
        "type": "document", 
        "source": {
          "type": "file",  # ❌ This is the problem - should be "url" or "base64"
          "file_id": "gs://agent-artifact-bucket/nova_bot/inputtestuser/inputtestsession-81/artifact_e-759bdd7d-7722-4500-8a4e-ec9fc5884cf0_1/0"
        }
      }, 
      {"type": "text", "text": "For context:"}, 
      {"type": "text", "text": "[NovaAgent] called tool `load_artifacts` with parameters: {\"artifact_names\": [\"artifact_e-759bdd7d-7722-4500-8a4e-ec9fc5884cf0_1\"]}"}, 
      {"type": "text", "text": "For context:"}, 
      {"type": "text", "text": "[NovaAgent] `load_artifacts` tool returned result: {\"artifact_names\": [\"artifact_e-759bdd7d-7722-4500-8a4e-ec9fc5884cf0_1\"], \"status\": \"artifact contents temporarily inserted and removed. to access these artifacts, call load_artifacts tool again.\"}"}, 
      {"type": "text", "text": "For context:"}, 
      {"type": "text", "text": "[NovaAgent] called tool `transfer_to_agent` with parameters: {\"agent_name\": \"AcademicContentAgent\"}"}, 
      {"type": "text", "text": "For context:"}, 
      {"type": "text", "text": "[NovaAgent] `transfer_to_agent` tool returned result: {\"result\": null}"}
    ]
  }], 
  "temperature": 1, 
  "max_tokens": 8192, 
  "tools": [{
    "name": "load_artifacts", 
    "input_schema": {
      "type": "object", 
      "properties": {
        "artifact_names": {
          "items": {"type": "string"}, 
          "type": "array"
        }
      }
    }, 
    "description": "Loads artifacts into the session for this request.\n\nNOTE: Call when you need access to artifacts (for example, uploads saved by the\nweb UI)."
  }], 
  "thinking": {"type": "disabled"}, 
  "anthropic_version": "vertex-2023-10-16", 
  "system": [{
    "type": "text", 
    "text": "You are an academic research assistant..."
  }]
}'

Error Response:

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "messages.0.content.2.document.source: Input tag 'file' found using 'type' does not match any of the expected tags: 'base64', 'content', 'text', 'url'"
  },
  "request_id": "req_vrtx_011CWbF6Vs7aniezoeS7xNm7"
}

Example 2: OpenAI Request (Failing)

This is the actual curl request sent by LiteLLM that causes the error:

curl -X POST \
https://api.openai.com/v1/chat/completions \
-H 'Authorization: Bearer sk-...' \
-H 'Content-Type: application/json' \
-d '{
  "model": "gpt-4.1",
  "messages": [{
    "role": "system",
    "content": "You are an academic research assistant..."
  }, {
    "role": "user",
    "content": [
      {"type": "text", "text": "bu pdf ne anlatıyor ? akademik olarak açıklayabilir misin ?"},
      {"type": "text", "text": "[Uploaded Artifact: \"artifact_e-441b5ea7-4e34-4bda-9cba-9b2b1bce3134_1\"]"},
      {
        "type": "file",
        "file": {
          "file_id": "gs://agent-artifact-bucket/nova_bot/inputtestuser/inputtestsession-88/artifact_e-441b5ea7-4e34-4bda-9cba-9b2b1bce3134_1/0"  # ❌ GCS URI used as file_id - doesn't work
        }
      },
      {"type": "text", "text": "For context:"},
      {"type": "text", "text": "[NovaAgent] called tool `load_artifacts` with parameters: {\"artifact_names\": [\"artifact_e-441b5ea7-4e34-4bda-9cba-9b2b1bce3134_1\"]}"},
      {"type": "text", "text": "For context:"},
      {"type": "text", "text": "[NovaAgent] `load_artifacts` tool returned result: {\"artifact_names\": [\"artifact_e-441b5ea7-4e34-4bda-9cba-9b2b1bce3134_1\"], \"status\": \"artifact contents temporarily inserted and removed. to access these artifacts, call load_artifacts tool again.\"}"},
      {"type": "text", "text": "For context:"},
      {"type": "text", "text": "[NovaAgent] called tool `transfer_to_agent` with parameters: {\"agent_name\": \"AcademicContentAgent\"}"},
      {"type": "text", "text": "For context:"},
      {"type": "text", "text": "[NovaAgent] `transfer_to_agent` tool returned result: {\"result\": null}"}
    ]
  }],
  "temperature": 1,
  "max_completion_tokens": 8192,
  "tools": [{
    "type": "function",
    "function": {
      "name": "load_artifacts",
      "description": "Loads artifacts into the session for this request.\n\nNOTE: Call when you need access to artifacts (for example, uploads saved by the\nweb UI).",
      "parameters": {
        "type": "object",
        "properties": {
          "artifact_names": {
            "items": {"type": "string"},
            "type": "array"
          }
        }
      }
    }
  }],
  "stream": true,
  "stream_options": {"include_usage": true}
}'

Error Response:

{
  "error": {
    "message": "Invalid file data: 'file_id'. Expected a file with an application/pdf MIME type, but got unsupported MIME type 'None'.",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

Expected behavior

The _get_content function should handle file_data.file_uri differently based on the provider:

For OpenAI/Azure:
- Download the file from GCS
- Upload it to OpenAI/Azure API using litellm.acreate_file()
- Use the returned file_id in the content object
- Format: {"type": "file", "file": {"file_id": "<uploaded_file_id>"}}
For Claude/Anthropic via Vertex AI:
- Convert GCS URI to a signed URL, OR
- Download file and convert to base64
- Use document format with source.type: 'url' or source.type: 'base64'
- Format: {"type": "document", "source": {"type": "url", "url": "<signed_url>"}}
For Gemini/Google providers:
- Current behavior is correct (GCS URIs work natively)
- No changes needed

Screenshots

Error 1: OpenAI

BadRequestError: litellm.BadRequestError: OpenAIException - Files 
[gs://agent-artifact-bucket/nova_bot/inputtestuser/inputtestsession-88/artifact_e-441b5ea7-4e34-4bda-9cba-9b2b1bce3134_1/0] were not found

Error 2: Claude/Anthropic

BadRequestError: litellm.BadRequestError: Vertex_aiException BadRequestError - 
b'{"type":"error","error":{"type":"invalid_request_error",
"message":"messages.0.content.2.document.source: Input tag \'file\' 
found using \'type\' does not match any of the expected tags: 
\'base64\', \'content\', \'text\', \'url\'"}'

Desktop (please complete the following information):

OS: [macOS, Linux]
Python version(python -V): Python 3.10
ADK version(pip show google-adk): google-adk from git+https://github.com/google/adk-python.git@26e77e16947aed1abcfdd7f526532a708f1f073b (GcsArtifactService not work correctly in LiteLLM models #3787 --> This problem solved with this version)

Model Information:

Are you using LiteLLM: Yes
Which model is being used(e.g. gemini-2.5-pro) openai/gpt-4.1, vertex_ai/claude-sonnet-4@20250514

Notification: The author has updated the issue description. Maintainers, please review.

fix: Handle file URI conversion for LiteLLM based on provider and model

Hi! Is this feature being adopted by any version? I have this issue too and I'm using 2.9.0 ADK version

Navigation Menu

Repository navigation

[BUG]: Before load_artifacts, Incorrect file_uri handling for OpenAI and Claude/Anthropic providers in LiteLLM #4038

Example 1: Claude/Anthropic Request (Failing)

Example 2: OpenAI Request (Failing)

Error 1: OpenAI

Error 2: Claude/Anthropic

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[BUG]: Before load_artifacts, Incorrect file_uri handling for OpenAI and Claude/Anthropic providers in LiteLLM #4038

Description

Example 1: Claude/Anthropic Request (Failing)

Example 2: OpenAI Request (Failing)

Error 1: OpenAI

Error 2: Claude/Anthropic

Activity

adk-bot commented on Jan 6, 2026

ecanlar commented on Sep 16, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions