phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-25 10:44:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix(route_llm_request.py): move to using common router, even for clie… (#8966)

Browse source 
* fix(route_llm_request.py): move to using common router, even for client-side credentials

ensures fallbacks / cooldown logic still works

* test(test_route_llm_request.py): add unit test for route request

* feat(router.py): generate unique model id when clientside credential passed in

Prevents cooldowns for api key 1 from impacting api key 2

* test(test_router.py): update testing to ensure original litellm params not mutated

* fix(router.py): upsert clientside call into llm router model list

enables cooldown logic to work accurately

* fix: fix linting error

* test(test_router_utils.py): add direct test for new util on router
This commit is contained in:
Krish Dholakia 2025-03-03 22:57:08 -08:00 • committed by GitHub
parent 94563ab1e7
commit 2fc6262675
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 273 additions and 36 deletions
Download patch file Download diff file Expand all files Collapse all files

2
litellm/proxy/route_llm_request.py
View file

@ -53,7 +53,7 @@ async def route_request(
"""
router_model_names = llm_router.model_names if llm_router is not None else []
if "api_key" in data or "api_base" in data:
return getattr(litellm, f"{route_type}")(**data)
return getattr(llm_router, f"{route_type}")(**data)
elif "user_config" in data:
router_config = data.pop("user_config")