Skip to content

Internal Server Error #296

Open
Open
Internal Server Error#296

Description

@shobeg
shobeg
opened on May 22, 2025

After successfully starting the docker for the basic rag and opening the webpage, the page shows an Internal Server error. And, the web console shows a 500 error trying to get /content/converse/?__theme=light.

Activity

  1. mrsaso commented on May 27, 2025

    @mrsaso
    mrsaso
    on May 27, 2025

    Hi, what is the output when you check the chain-server logs? What is the specified error is it giving for the 500 Internal Server?
    In terminal, run $ docker logs chain-server

  2. yxchia98 commented on Jul 8, 2025

    @yxchia98
    yxchia98
    on Jul 8, 2025

    Facing the same issue as well, seems like something to do with the rag-playground container on the frontend.
    Chain-server is running fine, here's my logs after running docker logs chain-server:

    /usr/local/lib/python3.10/dist-packages/pymilvus/client/__init__.py:6: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  from pkg_resources import DistributionNotFound, get_distribution
INFO:     Started server process [1]
INFO:     Waiting for application startup.
INFO:faiss.loader:Loading faiss with AVX2 support.
INFO:faiss.loader:Successfully loaded faiss with AVX2 support.
INFO:RAG.src.chain_server.utils:Using nvidia-ai-endpoints as model engine and nvidia/nv-embedqa-e5-v5 and model for embeddings
INFO:RAG.src.chain_server.utils:Using embedding model nvidia/nv-embedqa-e5-v5 hosted at api catalog
INFO:RAG.src.chain_server.utils:Using milvus collection: multimodal_rag
INFO:RAG.src.chain_server.utils:Using milvus collection: multimodal_rag
INFO:RAG.src.chain_server.utils:Vector store created and saved.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8081 (Press CTRL+C to quit)

    And here's my logs for rag-playground after running docker logs rag-playground:

    [1] Jul 08 03:41:34 [   INFO] - httpx - HTTP Request: GET https://api.gradio.app/gradio-messaging/en "HTTP/1.1 200 OK"
[1] Jul 08 03:41:36 [   INFO] - frontend - Loading application configuration.
[1] Jul 08 03:41:36 [   INFO] - frontend - Configuration: 
model-name: meta/llama3-8b-instruct
server-port: '8081'
server-url: http://chain-server

[1] Jul 08 03:41:36 [   INFO] - frontend - Instantiating the API Server.
[1] Jul 08 03:41:36 [   INFO] - httpx - HTTP Request: GET https://checkip.amazonaws.com/ "HTTP/1.1 200 "
[1] Jul 08 03:41:36 [  ERROR] - frontend.chat_client - Failed to connect /documents endpoint of chain-server. Error details: HTTPConnectionPool(host='chain-server', port=8081): Max retries exceeded with url: /documents (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7b3156737ca0>: Failed to establish a new connection: [Errno 111] Connection refused')).
[1] Jul 08 03:41:36 [WARNING] - matplotlib - mkdir -p failed for path /.config/matplotlib: [Errno 13] Permission denied: '/.config'
[1] Jul 08 03:41:36 [WARNING] - matplotlib - Matplotlib created a temporary cache directory at /tmp/matplotlib-me1b8lqh because there was an issue with the default path (/.config/matplotlib); it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
[1] Jul 08 03:41:36 [   INFO] - httpx - HTTP Request: GET https://api.gradio.app/pkg-version "HTTP/1.1 200 OK"
/usr/local/lib/python3.10/dist-packages/gradio/analytics.py:106: UserWarning: IMPORTANT: You are using gradio version 4.43.0, however version 4.44.1 is available, please upgrade. 
--------
  warnings.warn(
[1] Jul 08 03:41:36 [   INFO] - httpx - HTTP Request: GET https://api.gradio.app/pkg-version "HTTP/1.1 200 OK"
/usr/local/lib/python3.10/dist-packages/gradio/analytics.py:106: UserWarning: IMPORTANT: You are using gradio version 4.43.0, however version 4.44.1 is available, please upgrade. 
--------
  warnings.warn(
[1] Jul 08 03:41:36 [   INFO] - matplotlib.font_manager - generated new fontManager
[1] Jul 08 03:41:37 [   INFO] - frontend - Starting the API Server.
[1] Jul 08 03:41:37 [   INFO] - uvicorn.error - Started server process [1]
[1] Jul 08 03:41:37 [   INFO] - uvicorn.error - Waiting for application startup.
[1] Jul 08 03:41:37 [   INFO] - uvicorn.error - Application startup complete.
[1] Jul 08 03:41:37 [   INFO] - uvicorn.error - Uvicorn running on http://0.0.0.0:8090 (Press CTRL+C to quit)
[1] Jul 08 03:42:05 [   INFO] - uvicorn.access - 132.237.184.249:33675 - "GET /kb HTTP/1.1" 200
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:33675 - "GET /_next/static/css/7636246223312442.css HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:32867 - "GET /_next/static/chunks/webpack-5146130448d8adf7.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:33675 - "GET /_next/static/chunks/main-92011a1a7f336a6f.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:32867 - "GET /_next/static/WuNGAl0x4o1D5HqLxhHMt/_buildManifest.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:59656 - "GET /_next/static/chunks/pages/kb-cf0d102293dc0a74.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:2741 - "GET /_next/static/chunks/pages/_app-f55c3b932a623280.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:7582 - "GET /_next/static/chunks/78-a36dca5d49fafb86.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:21050 - "GET /_next/static/chunks/framework-7a7e500878b44665.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:33675 - "GET /_next/static/WuNGAl0x4o1D5HqLxhHMt/_ssgManifest.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:59656 - "GET /_next/static/chunks/pages/converse-61880f01babd873a.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:32867 - "GET /content/kb/?__theme=light HTTP/1.1" 500
[1] Jul 08 03:42:06 [  ERROR] - uvicorn.error - Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/uvicorn/protocols/http/h11_impl.py", line 412, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.10/dist-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 460, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/dist-packages/gradio/route_utils.py", line 760, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 73, in app
    response = await f(request)
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 301, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 214, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/usr/local/lib/python3.10/dist-packages/starlette/concurrency.py", line 39, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 967, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 427, in main
    gradio_api_info = api_info(False)
  File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 456, in api_info
    app.api_info = app.get_blocks().get_api_info()
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 2782, in get_api_info
    python_type = client_utils.json_schema_to_python_type(info)
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 893, in json_schema_to_python_type
    type_ = _json_schema_to_python_type(schema, schema.get("$defs"))
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 938, in _json_schema_to_python_type
    elements = _json_schema_to_python_type(items, defs)
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 908, in _json_schema_to_python_type
    return _json_schema_to_python_type(defs[schema["$ref"].split("/")[-1]], defs)
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 947, in _json_schema_to_python_type
    des = [
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 948, in <listcomp>
    f"{n}: {_json_schema_to_python_type(v, defs)}{get_desc(v)}"
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 955, in _json_schema_to_python_type
    f"str, {_json_schema_to_python_type(schema['additionalProperties'], defs)}"
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 901, in _json_schema_to_python_type
    type_ = get_type(schema)
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 863, in get_type
    if "const" in schema:
TypeError: argument of type 'bool' is not iterable
[1] Jul 08 03:42:07 [   INFO] - uvicorn.access - 132.237.184.249:59656 - "GET /favicon.ico HTTP/1.1" 304

    Any fixes to this?

  3. mrsaso commented on Jul 22, 2025

    @mrsaso
    mrsaso
    on Jul 22, 2025

    @yxchia98 What architecture are you running on? x86 or aarch64/Arm?

    I had run this on an aarch64 system, and unfortunately the rag-playground container only supports x86. I made modifications to remove rag-playground deployment and simply used the FastAPI to interface with the RAG. Not sure if may be the issue you are having.

  4. tom5610 commented on Jul 28, 2025

    @tom5610
    tom5610
    on Jul 28, 2025

    hey @mrsaso & @yxchia98, I encountered exact the same error and it was due to some conflicts between pydantic and gradio. I resolved the issue with pinning pydantic v2.10.6 in requirements.txt under "rag-playground' module. (refer github issue in gradio - gradio-app/gradio#10662)

    ...
pydantic==2.10.6
...

    In addition, I couldn't invoke 'meta/llama3-70b-instruct' and encountered an error -

    ... Function id 'a88f115a-4a47-4381-ad62-ca25dc33dc1b' version 'null': Specified function in account ...

    To resolve the issue with using different LLM (e.g. deepseek-r1). @mrsaso, any hints on the error? Thanks.

  5. buildvoc-agent
    added a commit that references this issue on Mar 21, 2026

    refactor: removes wrapper for ChatCompletionContent and adds document…

    151a2a1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions