phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-26 11:14:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

(proxy perf improvement) - use uvloop for higher RPS (10%-20% higher RPS) (#7662)

Browse source 
* uvicorn use uvloop

* fix uvloop==0.21.0

* add uvloop to pyproject

* test_completion_response_ratelimit_headers
This commit is contained in:
Ishaan Jaff 2025-01-09 18:11:20 -08:00 • committed by GitHub
parent a85de46ef7
commit 2507c275f6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 261 additions and 197 deletions
Download patch file Download diff file Expand all files Collapse all files

5
litellm/proxy/proxy_cli.py
View file

@ -671,7 +671,10 @@ def run_server( # noqa: PLR0915
)
uvicorn_args["ssl_keyfile"] = ssl_keyfile_path
uvicorn_args["ssl_certfile"] = ssl_certfile_path
uvicorn.run(**uvicorn_args)
uvicorn.run(
**uvicorn_args,
loop="uvloop",
)
elif run_gunicorn is True:
# Gunicorn Application Class
class StandaloneApplication(gunicorn.app.base.BaseApplication):