phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-25 18:54:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix(lowest_tpm_rpm_routing.py): broaden scope of get deployment logic

Browse source
This commit is contained in:
Krrish Dholakia 2023-12-30 13:27:50 +05:30
parent a6719caebd
commit b66cf0aa43
3 changed files with 90 additions and 22 deletions
Download patch file Download diff file Expand all files Collapse all files

2
litellm/router.py
View file

@ -1622,7 +1622,7 @@ class Router:
and self.lowesttpm_logger is not None
):
min_deployment = self.lowesttpm_logger.get_available_deployments(
model_group=model
model_group=model, healthy_deployments=healthy_deployments
)
if min_deployment is None:
min_deployment = random.choice(healthy_deployments)