Skip to content

When making client calls from a sync env (with asyncio.run()) the loop gets closed after the first request, causing subsequent ones to fail #366

Closed

Description

Whenever I make requests via the GraphServiceClient from a sync environment using asyncio.run() or asgiref.sync.async_to_sync() the first call succeeds, however, all subsequent calls fail with an error:

Traceback (most recent call last):
  File "/app/some_path/some_file.py", line 15, in get_drives
    drives = asyncio.run(client.drives.get())
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.11/site-packages/msgraph/generated/drives/drive_request_builder.py", line 66, in get
    return await self.request_adapter.send_async(request_info, DriveItemCollectionResponse, error_mapping)
  File "/usr/local/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py", line 166, in send_async
    response = await self.get_http_response_message(request_info, parent_span)
  File "/usr/local/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py", line 508, in get_http_response_message
    resp = await self._http_client.send(request)
  File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1617, in send
    response = await self._send_handling_auth(
  File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1645, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1682, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1719, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/usr/local/lib/python3.11/site-packages/msgraph_core/middleware/async_graph_transport.py", line 21, in handle_async_request
    response = await self.pipeline.send(request)
  File "/usr/local/lib/python3.11/site-packages/kiota_http/middleware/middleware.py", line 38, in send
    return await self._first_middleware.send(request, self._transport)
  File "/usr/local/lib/python3.11/site-packages/kiota_http/middleware/redirect_handler.py", line 77, in send
    response = await super().send(request, transport)
  File "/usr/local/lib/python3.11/site-packages/kiota_http/middleware/middleware.py", line 65, in send
    return await self.next.send(request, transport)
  File "/usr/local/lib/python3.11/site-packages/kiota_http/middleware/retry_handler.py", line 83, in send
    response = await super().send(request, transport)
  File "/usr/local/lib/python3.11/site-packages/kiota_http/middleware/middleware.py", line 65, in send
    return await self.next.send(request, transport)
  File "/usr/local/lib/python3.11/site-packages/kiota_http/middleware/parameters_name_decoding_handler.py", line 55, in send
    response = await super().send(request, transport)
  File "/usr/local/lib/python3.11/site-packages/kiota_http/middleware/middleware.py", line 65, in send
    return await self.next.send(request, transport)
  File "/usr/local/lib/python3.11/site-packages/kiota_http/middleware/url_replace_handler.py", line 44, in send
    response = await super().send(request, transport)
  File "/usr/local/lib/python3.11/site-packages/kiota_http/middleware/middleware.py", line 65, in send
    return await self.next.send(request, transport)
  File "/usr/local/lib/python3.11/site-packages/kiota_http/middleware/user_agent_handler.py", line 29, in send
    return await super().send(request, transport)
  File "/usr/local/lib/python3.11/site-packages/kiota_http/middleware/middleware.py", line 65, in send
    return await self.next.send(request, transport)
  File "/usr/local/lib/python3.11/site-packages/msgraph_core/middleware/telemetry.py", line 48, in send
    response = await super().send(request, transport)
  File "/usr/local/lib/python3.11/site-packages/kiota_http/middleware/middleware.py", line 62, in send
    response = await transport.handle_async_request(request)
  File "/usr/local/lib/python3.11/site-packages/msgraph_core/middleware/async_graph_transport.py", line 24, in handle_async_request
    response = await self.transport.handle_async_request(request)
  File "/usr/local/lib/python3.11/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/usr/local/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 226, in handle_async_request
    await self._close_expired_connections()
  File "/usr/local/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 194, in _close_expired_connections
    await connection.aclose()
  File "/usr/local/lib/python3.11/site-packages/httpcore/_async/connection.py", line 166, in aclose
    await self._connection.aclose()
  File "/usr/local/lib/python3.11/site-packages/httpcore/_async/http11.py", line 241, in aclose
    await self._network_stream.aclose()
  File "/usr/local/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 54, in aclose
    await self._stream.aclose()
  File "/usr/local/lib/python3.11/site-packages/anyio/streams/tls.py", line 195, in aclose
    await self.transport_stream.aclose()
  File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 1161, in aclose
    self._transport.close()
  File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 860, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
    self._check_closed()
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

The simplest example that reproduces this is:

import asyncio
from azure.identity import ClientSecretCredential
from msgraph import GraphServiceClient

credential = ClientSecretCredential(
    'TENANT_ID',
    'CLIENT_ID',
    'CLIENT_SECRET')
scopes = ["https://graph.microsoft.com/.default"]
client = GraphServiceClient(credential, scopes)

drives_1 = asyncio.run(client.drives.get())
drives_2 = asyncio.run(client.drives.get())

I use the GraphServiceClient inside a task run in a background worker (Django Q) alongside my Django app. So the first time the background task runs it succeeds, while the second time it fails.

As a workaround this does the trick:

def get_loop():
    try:
        loop = asyncio.get_event_loop()
    except RuntimeError:
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
    return loop

# inside the background task
drives = get_loop().run_until_complete(client.drives.get())

Activity

  1. stvpwrs commented on Nov 14, 2023

    @stvpwrs

    Hey Andriy Berezhanskyy (@aberezh) I have encountered this issue too. For me, the issue was because I was running on Windows and I needed to add the following to my script:

    asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
    

    That fixed the issue for me.

    Good luck!

  2. preslavnolimit commented on Nov 14, 2023

    @preslavnolimit

    I am having the same issue on Mac with Apple Silicon M2.

    I manage to duck tape it similar to the example, but the exact example gives that "Future is not in same loop" on the second call. So I basically did that

    try: 
        result = asyncio.run(graphclient.users.post(user_info))
        return result
    except RunTimeError as e
        if str(e) == "Event Loop is closed"
            loop = asyncio.get_new_loop() 
            asyncio.set_loop(loop)
            result = loop.run_until_complete(graphclient.users.post(user_info))
            asyncio.set_loop(None)
            loop.close()
            return result
    

    This works for handling the "Event loop is closed problem", but sometimes I get http error from httpx timeout not sure if I am making it or the graph api is responsible.

    The strange thing about my solution derived from the author of the issue #366 (comment)

    is that asyncio.run actually does the same thing, not any different and it should work only with asyncio.run if look into the source code of the function "run".

  3. steph409 commented on Nov 15, 2023

    @steph409

    I have the same problem running on Linux.

  4. pclausen1 commented on Nov 28, 2023

    @pclausen1

    I had the same problem running in python 3.10 in Linux container. Was able to use similar workaround as others

    client = GraphServiceClient(credentials=credentials, scopes=scopes)
    loop = asyncio.get_event_loop()
    user1 = loop.run_until_complete(client.users.by_user_id(id1).get())
    user2 = loop.run_until_complete(client.users.by_user_id(id2).get())

    I see though that get_event_loop() is deprecated in python 3.12, so perhaps need to replace with something similar to Andriy Berezhanskyy (@aberezh) solution, but with get_running_loop() instead

    def get_loop():
        try:
            loop = asyncio.get_running_loop()
        except RuntimeError:
            loop = asyncio.new_event_loop()
            asyncio.set_event_loop(loop)
        return loop
    
    client = GraphServiceClient(credentials=credentials, scopes=scopes)
    loop = get_loop()
    user1 = loop.run_until_complete(client.users.by_user_id(id1).get())
    user2 = loop.run_until_complete(client.users.by_user_id(id2).get())```
  5. advsysal commented on Dec 13, 2023

    @advsysal

    I have found a straightforward solution that seems to work across multiple hosting platforms (windows, databricks notebooks). It's similar to some of the above solutions, but a 1 liner...
    instead of:
    asyncio.run(my_graph_function())
    use:
    asyncio.get_event_loop().run_until_complete(my_graph_function())

  6. erpola commented on Dec 26, 2023

    @erpola
    def get_loop():
        try: 
            loop = asyncio.get_event_loop()
        except RuntimeError:
            loop = asyncio.new_event_loop()
            asyncio.set_event_loop(loop)
        return loop
    

    calling the asyncio.run as get_loop().run_until_complete() works greats for this issue. thank you Andriy Berezhanskyy (@aberezh)

  7. Nintorac commented on Dec 31, 2023

    @Nintorac

    I get a similar issue in a DuckDB udf, my error looks like this

    duckdb.duckdb.InvalidInputException: Invalid Input Error: Python exception occurred while executing the UDF: RuntimeError: <asyncio.locks.Event object at 0x7f58eef84160 [unset]> is bound to a different event loop
    
    At:
      /opt/conda/envs/proj/lib/python3.10/asyncio/mixins.py(30): _get_loop
      /opt/conda/envs/proj/lib/python3.10/asyncio/locks.py(211): wait
    

    and @cireps workaround works for my situation too

  8. sstoops commented on Feb 1, 2024

    @sstoops

    My team ran into this exact issue, though under slightly different circumstances. We need to use the Graph API within the context of an Async Django request. We found the exact same behavior: the first request would succeed and subsequent requests would throw the RuntimeError: Event loop is closed error.

    While a few folks here have proposed different workaround to deal with the event loop itself, I wanted figure out why the SDK was hanging on to references to event loops.

    Background

    Using Async Django as the example, since it is where I ran into this issue, if you're not running a pure async stack (eg. one of your middlewares is non-async), then Django will run each request in a new thread with a new event loop. The Django debug server will also exhibit the same thread-per-request behavior even if you are top-to-bottom async.

    This is similar to the behavior described in this issue: calling asyncio.run multiple times from a script. I believe this is also the same issue reported in #82.

    Cause

    So something, somewhere in the SDK is grabbing a reference to an event loop, and attempting to use that reference during a subsequent request.

    After much sleuthing, I found the pattern leading to this behavior:

    • Looking at the GraphServiceClient source, I found in its __init__ method that it constructs a GraphRequestAdapter
    • Digging into the GraphRequestAdapter, I found the following used as an argument to the __init__: client = GraphClientFactory.create_with_default_middleware(options=options). That line means that it will generate a default client factory at the time this module is imported, not when it is executed.
    • Let's keep digging.
    • Drilling into the GraphClientFactory.create_with_default_middleware method, we see the same pattern in the arg default: client: httpx.AsyncClient = KiotaClientFactory.get_default_client(). Again, that line is executed when the module is first imported, not when the function is called. That means all instances of GraphClientFactory will share the same client object that was created under whichever thread/event loop the module was imported in.

    To summarize all that, every GraphServiceClient you create is going to end up sharing the same underlying httpx.AsyncClient. That httpx.AsyncClient instance is constructed when you import the module, not when you actually run any code. This client is what (eventually) holds on to the reference to the event loop.

    Our workaround

    I managed to come up with a workaround that seems to be working so far. It has been tested both in a non-pure-async Django stack (meaning there's a sync middleware causing Django shuttle requests out to threads) and in the Django debug server.

    from typing import Optional
    
    from azure.identity.aio import ClientSecretCredential
    from kiota_authentication_azure.azure_identity_authentication_provider import (
        AzureIdentityAuthenticationProvider,
    )
    from kiota_http.kiota_client_factory import KiotaClientFactory
    from msgraph.graph_request_adapter import GraphRequestAdapter, options
    from msgraph.graph_service_client import GraphServiceClient
    from msgraph_core import GraphClientFactory
    
    
    def get_request_adapter(
        credentials: ClientSecretCredential, scopes: Optional[list[str]] = None
    ) -> GraphRequestAdapter:
        if scopes:
            auth_provider = AzureIdentityAuthenticationProvider(credentials=credentials, scopes=scopes)
        else:
            auth_provider = AzureIdentityAuthenticationProvider(credentials=credentials)
    
        return GraphRequestAdapter(
            auth_provider=auth_provider,
            client=GraphClientFactory.create_with_default_middleware(
                options=options, client=KiotaClientFactory.get_default_client()
            ),
        )
    
    def create_client() -> GraphServiceClient:
        return GraphServiceClient(
            request_adapter=get_request_adapter(
                credentials=..., scopes=['https://graph.microsoft.com/.default']
            )
        )

    What does this do?

    GraphServiceClient can accept either credentials and scope, resulting in it constructing an auth scope and request adapter for you, or it can just accept an existing request adapter.

    The get_request_adapter function essentially mimics the existing SDK code, but instead of constructing any clients at the global scope (during import), it constructs a new request adapter each time you construct a client.

    With this code, you could create a new client within the scope of your async code.

    For example:

    async def foo():
        # Client created within each run and with it's own event loop. Good.
        client = create_client()
        client.users.list()
    
    asyncio.run(foo())
    asyncio.run(foo())

    Note that the two calls to asyncio.run(...), and therefore two event loops, now succeeds.

    Addendum

    There are many other instances in the SDK where clients are constructed within the scope of a method's default args, meaning, at the global scope. My workaround was the bare minimum I needed to get around the closed event loop issue for my own use case. If you run into this error in other parts of the SDK, I suggest looking for this problematic pattern.

  9. shemogumbe commented on Apr 23, 2024

    @shemogumbe
    Contributor

    Thanks for documenting this,
    While this works and solves for your use case, having multiple asyncio runs is considered an antipattern.

    Typically, we should have one asyncio.run(), as it sets up the event loop and starts the asynchronous execution.
    Besides being considered an antipattern, it can cause unexpected behaviour, note that an event loop manages asynchronous tasks within a single execution context.

    To execute within a single loop, use asyncio.gather()

    With your workaround, I would consider closing the issue and moving it to a discussion for reference by anyone who might face a similar issue

  10. shemogumbe commented on Apr 23, 2024

    @shemogumbe
    Contributor

    Moved to #683

  11. sstoops commented on Apr 23, 2024

    @sstoops

    Shem Ogumbe (@shemogumbe) The multiple calls to asyncio.run() was to show that the code can be called multiple times without encountering the event loop issue. The issue my team encountered was using a framework where we don't have control over the event loop. Multiple asyncio.run() calls simply emulates this behavior. I will update my comment to clarify.

  12. IamMrandrew commented on May 13, 2024

    @IamMrandrew

    My team ran into this exact issue, though under slightly different circumstances. We need to use the Graph API within the context of an Async Django request. We found the exact same behavior: the first request would succeed and subsequent requests would throw the RuntimeError: Event loop is closed error.

    While a few folks here have proposed different workaround to deal with the event loop itself, I wanted figure out why the SDK was hanging on to references to event loops.

    Background

    Using Async Django as the example, since it is where I ran into this issue, if you're not running a pure async stack (eg. one of your middlewares is non-async), then Django will run each request in a new thread with a new event loop. The Django debug server will also exhibit the same thread-per-request behavior even if you are top-to-bottom async.

    This is similar to the behavior described in this issue: calling asyncio.run multiple times from a script. I believe this is also the same issue reported in #82.

    Cause

    So something, somewhere in the SDK is grabbing a reference to an event loop, and attempting to use that reference during a subsequent request.

    After much sleuthing, I found the pattern leading to this behavior:

    • Looking at the GraphServiceClient source, I found in its __init__ method that it constructs a GraphRequestAdapter
    • Digging into the GraphRequestAdapter, I found the following used as an argument to the __init__: client = GraphClientFactory.create_with_default_middleware(options=options). That line means that it will generate a default client factory at the time this module is imported, not when it is executed.
    • Let's keep digging.
    • Drilling into the GraphClientFactory.create_with_default_middleware method, we see the same pattern in the arg default: client: httpx.AsyncClient = KiotaClientFactory.get_default_client(). Again, that line is executed when the module is first imported, not when the function is called. That means all instances of GraphClientFactory will share the same client object that was created under whichever thread/event loop the module was imported in.

    To summarize all that, every GraphServiceClient you create is going to end up sharing the same underlying httpx.AsyncClient. That httpx.AsyncClient instance is constructed when you import the module, not when you actually run any code. This client is what (eventually) holds on to the reference to the event loop.

    Our workaround

    I managed to come up with a workaround that seems to be working so far. It has been tested both in a non-pure-async Django stack (meaning there's a sync middleware causing Django shuttle requests out to threads) and in the Django debug server.

    from typing import Optional
    
    from azure.identity.aio import ClientSecretCredential
    from kiota_authentication_azure.azure_identity_authentication_provider import (
        AzureIdentityAuthenticationProvider,
    )
    from kiota_http.kiota_client_factory import KiotaClientFactory
    from msgraph.graph_request_adapter import GraphRequestAdapter, options
    from msgraph.graph_service_client import GraphServiceClient
    from msgraph_core import GraphClientFactory
    
    
    def get_request_adapter(
        credentials: ClientSecretCredential, scopes: Optional[list[str]] = None
    ) -> GraphRequestAdapter:
        if scopes:
            auth_provider = AzureIdentityAuthenticationProvider(credentials=credentials, scopes=scopes)
        else:
            auth_provider = AzureIdentityAuthenticationProvider(credentials=credentials)
    
        return GraphRequestAdapter(
            auth_provider=auth_provider,
            client=GraphClientFactory.create_with_default_middleware(
                options=options, client=KiotaClientFactory.get_default_client()
            ),
        )
    
    def create_client() -> GraphServiceClient:
        return GraphServiceClient(
            request_adapter=get_request_adapter(
                credentials=..., scopes=['https://graph.microsoft.com/.default']
            )
        )

    What does this do?

    GraphServiceClient can accept either credentials and scope, resulting in it constructing an auth scope and request adapter for you, or it can just accept an existing request adapter.

    The get_request_adapter function essentially mimics the existing SDK code, but instead of constructing any clients at the global scope (during import), it constructs a new request adapter each time you construct a client.

    With this code, you could create a new client within the scope of your async code.

    For example:

    async def foo():
        # Client created within each run and with it's own event loop. Good.
        client = create_client()
        client.users.list()
    
    asyncio.run(foo())
    asyncio.run(foo())

    Note that the two calls to asyncio.run(...), and therefore two event loops, now succeeds.

    Addendum

    There are many other instances in the SDK where clients are constructed within the scope of a method's default args, meaning, at the global scope. My workaround was the bare minimum I needed to get around the closed event loop issue for my own use case. If you run into this error in other parts of the SDK, I suggest looking for this problematic pattern.

    I used this workaround for my Flask application with async extra, together with creating a new client every time when accepting a request.

    It solves the issue where I was getting the RuntimeError: Event loop is closed error on my second request after the application starts.

    Not sure if it affect performance or as anti-pattern. But I couldn't find relevant examples on how should I implement it with this SDK. Let me know if there's update on this.

  13. vschmidt commented on Aug 28, 2025

    @vschmidt

    My team ran into this exact issue, though under slightly different circumstances. We need to use the Graph API within the context of an Async Django request. We found the exact same behavior: the first request would succeed and subsequent requests would throw the RuntimeError: Event loop is closed error.

    While a few folks here have proposed different workaround to deal with the event loop itself, I wanted figure out why the SDK was hanging on to references to event loops.

    Background

    Using Async Django as the example, since it is where I ran into this issue, if you're not running a pure async stack (eg. one of your middlewares is non-async), then Django will run each request in a new thread with a new event loop. The Django debug server will also exhibit the same thread-per-request behavior even if you are top-to-bottom async.

    This is similar to the behavior described in this issue: calling asyncio.run multiple times from a script. I believe this is also the same issue reported in #82.

    Cause

    So something, somewhere in the SDK is grabbing a reference to an event loop, and attempting to use that reference during a subsequent request.

    After much sleuthing, I found the pattern leading to this behavior:

    • Looking at the GraphServiceClient source, I found in its __init__ method that it constructs a GraphRequestAdapter

    • Digging into the GraphRequestAdapter, I found the following used as an argument to the __init__: client = GraphClientFactory.create_with_default_middleware(options=options). That line means that it will generate a default client factory at the time this module is imported, not when it is executed.

    • Let's keep digging.

    • Drilling into the GraphClientFactory.create_with_default_middleware method, we see the same pattern in the arg default: client: httpx.AsyncClient = KiotaClientFactory.get_default_client(). Again, that line is executed when the module is first imported, not when the function is called. That means all instances of GraphClientFactory will share the same client object that was created under whichever thread/event loop the module was imported in.

    To summarize all that, every GraphServiceClient you create is going to end up sharing the same underlying httpx.AsyncClient. That httpx.AsyncClient instance is constructed when you import the module, not when you actually run any code. This client is what (eventually) holds on to the reference to the event loop.

    Our workaround

    I managed to come up with a workaround that seems to be working so far. It has been tested both in a non-pure-async Django stack (meaning there's a sync middleware causing Django shuttle requests out to threads) and in the Django debug server.

    from typing import Optional
    
    
    
    from azure.identity.aio import ClientSecretCredential
    
    from kiota_authentication_azure.azure_identity_authentication_provider import (
    
        AzureIdentityAuthenticationProvider,
    
    )
    
    from kiota_http.kiota_client_factory import KiotaClientFactory
    
    from msgraph.graph_request_adapter import GraphRequestAdapter, options
    
    from msgraph.graph_service_client import GraphServiceClient
    
    from msgraph_core import GraphClientFactory
    
    
    
    
    
    def get_request_adapter(
    
        credentials: ClientSecretCredential, scopes: Optional[list[str]] = None
    
    ) -> GraphRequestAdapter:
    
        if scopes:
    
            auth_provider = AzureIdentityAuthenticationProvider(credentials=credentials, scopes=scopes)
    
        else:
    
            auth_provider = AzureIdentityAuthenticationProvider(credentials=credentials)
    
    
    
        return GraphRequestAdapter(
    
            auth_provider=auth_provider,
    
            client=GraphClientFactory.create_with_default_middleware(
    
                options=options, client=KiotaClientFactory.get_default_client()
    
            ),
    
        )
    
    
    
    def create_client() -> GraphServiceClient:
    
        return GraphServiceClient(
    
            request_adapter=get_request_adapter(
    
                credentials=..., scopes=['https://graph.microsoft.com/.default']
    
            )
    
        )

    What does this do?

    GraphServiceClient can accept either credentials and scope, resulting in it constructing an auth scope and request adapter for you, or it can just accept an existing request adapter.

    The get_request_adapter function essentially mimics the existing SDK code, but instead of constructing any clients at the global scope (during import), it constructs a new request adapter each time you construct a client.

    With this code, you could create a new client within the scope of your async code.

    For example:

    async def foo():
    
        # Client created within each run and with it's own event loop. Good.
    
        client = create_client()
    
        client.users.list()
    
    
    
    asyncio.run(foo())
    
    asyncio.run(foo())

    Note that the two calls to asyncio.run(...), and therefore two event loops, now succeeds.

    Addendum

    There are many other instances in the SDK where clients are constructed within the scope of a method's default args, meaning, at the global scope. My workaround was the bare minimum I needed to get around the closed event loop issue for my own use case. If you run into this error in other parts of the SDK, I suggest looking for this problematic pattern.

    Thanks for this, this was the unique way to deal with loop breaks.
    I try several ways to deal with my coroutines, and this solve the problem.

  14. Bain-OS commented on Aug 28, 2025

    @Bain-OS
    No description provided.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions