phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-28 05:11:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix: cancel scheduler tasks on shutdown

Browse source 
Otherwise the currently running tasks will never exit (before they
actually complete), which means the process can't be properly shut down
(only with SIGKILL).

Ideally, we let tasks know that they are about to shutdown and give them
some time to do so; but in the lack of the mechanism, it's better to
cancel than linger forever.

Signed-off-by: Ihar Hrachyshka <ihar.hrachyshka@gmail.com>
This commit is contained in:
Ihar Hrachyshka 2025-05-09 13:34:04 -04:00
parent 473a07f624
commit 1ceebdc813
2 changed files with 20 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

8
llama_stack/providers/utils/scheduler.py
View file

@ -157,10 +157,14 @@ class _NaiveSchedulerBackend(_SchedulerBackend):
asyncio.set_event_loop(self._loop)
self._loop.run_forever()
# When stopping the loop, give tasks a chance to finish
# TODO: When stopping the loop, give tasks a chance to finish
# TODO: should we explicitly inform jobs of pending stoppage?
# cancel all tasks
for task in asyncio.all_tasks(self._loop):
self._loop.run_until_complete(task)
if not task.done():
task.cancel()
self._loop.close()
async def shutdown(self) -> None: