llama-stack-mirror/llama_stack/providers/utils
Eric Huang 67cdeb1bef fix shutdown
# What does this PR do?
Gets rid of this error message below (disclaimer: not sure why, but it does).

ERROR    2025-10-06 12:04:22,837 asyncio:118 uncategorized: Task exception was never retrieved
         future: <Task finished name='Task-36' coro=<AsyncClient.aclose() done, defined at
         /Users/erichuang/projects/llama-stack-git2/.venv/lib/python3.12/site-packages/httpx/_client.py:1978> exception=RuntimeError('unable to perform operation on <TCPTransport
         closed=True reading=False 0x122dc7ad0>; the handler is closed')>
         ╭─────────────────────────────────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────────────────────────────────╮
         │ /Users/erichuang/projects/llama-stack-git2/.venv/lib/python3.12/site-packages/httpx/_client.py:1985 in aclose                                                           │
         │                                                                                                                                                                         │
         │   1982 │   │   if self._state != ClientState.CLOSED:                                                                                                                    │
         │   1983 │   │   │   self._state = ClientState.CLOSED                                                                                                                     │
         │   1984 │   │   │                                                                                                                                                        │
         │ ❱ 1985 │   │   │   await self._transport.aclose()                                                                                                                       │
         │   1986 │   │   │   for proxy in self._mounts.values():                                                                                                                  │
         │   1987 │   │   │   │   if proxy is not None:                                                                                                                            │
         │   1988 │   │   │   │   │   await proxy.aclose()                                                                                                                         │
         │                                                                                                                                                                         │
         │ /Users/erichuang/projects/llama-stack-git2/.venv/lib/python3.12/site-packages/httpx/_transports/default.py:406 in aclose                                                │
         │                                                                                                                                                                         │
         │   403 │   │   )                                                                                                                                                         │
         │   404 │                                                                                                                                                                 │
         │   405 │   async def aclose(self) -> None:                                                                                                                               │
         │ ❱ 406 │   │   await self._pool.aclose()                                                                                                                                 │
         │   407                                                                                                                                                                   │
         │                                                                                                                                                                         │
         │ /Users/erichuang/projects/llama-stack-git2/.venv/lib/python3.12/site-packages/httpcore/_async/connection_pool.py:353 in aclose                                          │
         │                                                                                                                                                                         │
         │   350 │   │   with self._optional_thread_lock:                                                                                                                          │
         │   351 │   │   │   closing_connections = list(self._connections)                                                                                                         │
         │   352 │   │   │   self._connections = []                                                                                                                                │
         │ ❱ 353 │   │   await self._close_connections(closing_connections)                                                                                                        │
         │   354 │                                                                                                                                                                 │
         │   355 │   async def __aenter__(self) -> AsyncConnectionPool:                                                                                                            │
         │   356 │   │   return self                                                                                                                                               │
         │                                                                                                                                                                         │
         │ /Users/erichuang/projects/llama-stack-git2/.venv/lib/python3.12/site-packages/httpcore/_async/connection_pool.py:345 in _close_connections                              │
         │                                                                                                                                                                         │
         │   342 │   │   # Close connections which have been removed from the pool.                                                                                                │
         │   343 │   │   with AsyncShieldCancellation():                                                                                                                           │
         │   344 │   │   │   for connection in closing:                                                                                                                            │
         │ ❱ 345 │   │   │   │   await connection.aclose()                                                                                                                         │
         │   346 │                                                                                                                                                                 │
         │   347 │   async def aclose(self) -> None:                                                                                                                               │
         │   348 │   │   # Explicitly close the connection pool.                                                                                                                   │
         │                                                                                                                                                                         │
         │ /Users/erichuang/projects/llama-stack-git2/.venv/lib/python3.12/site-packages/httpcore/_async/connection.py:173 in aclose                                               │
         │                                                                                                                                                                         │
         │   170 │   async def aclose(self) -> None:                                                                                                                               │
         │   171 │   │   if self._connection is not None:                                                                                                                          │
         │   172 │   │   │   async with Trace("close", logger, None, {}):                                                                                                          │
         │ ❱ 173 │   │   │   │   await self._connection.aclose()                                                                                                                   │
         │   174 │                                                                                                                                                                 │
         │   175 │   def is_available(self) -> bool:                                                                                                                               │
         │   176 │   │   if self._connection is None:                                                                                                                              │
         │                                                                                                                                                                         │
         │ /Users/erichuang/projects/llama-stack-git2/.venv/lib/python3.12/site-packages/httpcore/_async/http11.py:258 in aclose                                                   │
         │                                                                                                                                                                         │
         │   255 │   │   # Note that this method unilaterally closes the connection, and does                                                                                      │
         │   256 │   │   # not have any kind of locking in place around it.                                                                                                        │
         │   257 │   │   self._state = HTTPConnectionState.CLOSED                                                                                                                  │
         │ ❱ 258 │   │   await self._network_stream.aclose()                                                                                                                       │
         │   259 │                                                                                                                                                                 │
         │   260 │   # The AsyncConnectionInterface methods provide information about the state of                                                                                 │
         │   261 │   # the connection, allowing for a connection pooling implementation to                                                                                         │
         │                                                                                                                                                                         │
         │ /Users/erichuang/projects/llama-stack-git2/.venv/lib/python3.12/site-packages/httpcore/_backends/anyio.py:53 in aclose                                                  │
         │                                                                                                                                                                         │
         │    50 │   │   │   │   await self._stream.send(item=buffer)                                                                                                              │
         │    51 │                                                                                                                                                                 │
         │    52 │   async def aclose(self) -> None:                                                                                                                               │
         │ ❱  53 │   │   await self._stream.aclose()                                                                                                                               │
         │    54 │                                                                                                                                                                 │
         │    55 │   async def start_tls(                                                                                                                                          │
         │    56 │   │   self,                                                                                                                                                     │
         │                                                                                                                                                                         │
         │ /Users/erichuang/projects/llama-stack-git2/.venv/lib/python3.12/site-packages/anyio/streams/tls.py:216 in aclose                                                        │
         │                                                                                                                                                                         │
         │   213 │   │   │   │   await aclose_forcefully(self.transport_stream)                                                                                                    │
         │   214 │   │   │   │   raise                                                                                                                                             │
         │   215 │   │                                                                                                                                                             │
         │ ❱ 216 │   │   await self.transport_stream.aclose()                                                                                                                      │
         │   217 │                                                                                                                                                                 │
         │   218 │   async def receive(self, max_bytes: int = 65536) -> bytes:                                                                                                     │
         │   219 │   │   data = await self._call_sslobject_method(self._ssl_object.read, max_bytes)                                                                                │
         │                                                                                                                                                                         │
         │ /Users/erichuang/projects/llama-stack-git2/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py:1310 in aclose                                                │
         │                                                                                                                                                                         │
         │   1307 │   │   if not self._transport.is_closing():                                                                                                                     │
         │   1308 │   │   │   self._closed = True                                                                                                                                  │
         │   1309 │   │   │   try:                                                                                                                                                 │
         │ ❱ 1310 │   │   │   │   self._transport.write_eof()                                                                                                                      │
         │   1311 │   │   │   except OSError:                                                                                                                                      │
         │   1312 │   │   │   │   pass                                                                                                                                             │
         │   1313                                                                                                                                                                  │
         │                                                                                                                                                                         │
         │ in uvloop.loop.UVStream.write_eof:703                                                                                                                                   │
         │                                                                                                                                                                         │
         │ in uvloop.loop.UVHandle._ensure_alive:159                                                                                                                               │
         ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
         RuntimeError: unable to perform operation on <TCPTransport closed=True reading=False 0x122dc7ad0>; the handler is closed

## Test Plan
Run
uv run --with llama-stack llama stack build --distro=starter --image-type=venv --run

No more error
2025-10-06 13:50:26 -07:00
..
bedrock chore: use remoteinferenceproviderconfig for remote inference providers (#3668) 2025-10-03 08:48:42 -07:00
common chore(rename): move llama_stack.distribution to llama_stack.core (#2975) 2025-07-30 23:30:53 -07:00
datasetio chore(misc): make tests and starter faster (#3042) 2025-08-05 14:55:05 -07:00
files fix(expires_after): make sure multipart/form-data is properly parsed (#3612) 2025-09-30 16:14:03 -04:00
inference fix shutdown 2025-10-06 13:50:26 -07:00
kvstore chore: prune mypy exclude list (#3561) 2025-09-26 11:44:43 -04:00
memory feat: implement keyword and hybrid search for Weaviate provider (#3264) 2025-10-03 10:22:30 +02:00
responses fix: responses <> chat completion input conversion (#3645) 2025-10-02 16:01:08 -07:00
scoring chore: enable pyupgrade fixes (#1806) 2025-05-01 14:23:50 -07:00
sqlstore feat: Add OpenAI Conversations API (#3429) 2025-10-03 08:47:18 -07:00
telemetry feat: use FIPS validated CSPRNG for telemetry (#3554) 2025-09-26 11:17:25 +02:00
tools feat(tools)!: substantial clean up of "Tool" related datatypes (#3627) 2025-10-02 15:12:03 -07:00
vector_io feat: migrate to FIPS-validated cryptographic algorithms (#3423) 2025-09-12 11:18:19 +02:00
__init__.py API Updates (#73) 2024-09-17 19:51:35 -07:00
pagination.py chore(refact): move paginate_records fn outside of datasetio (#2137) 2025-05-12 10:56:14 -07:00
scheduler.py refactor(logging): rename llama_stack logger categories (#3065) 2025-08-21 17:31:04 -07:00