[Fix] Router/ Proxy - Tag Based routing, raise correct error when no deployments found and tag filtering is on (#5745)

* fix tag routing - raise correct error when no model with tag based routing

* fix error string from tag based routing

* test router tag based routing

* raise 401 error when no tags avialable for deploymen

* linting fix
This commit is contained in:
Ishaan Jaff 2024-09-17 20:24:28 -07:00 committed by GitHub
parent 911230c434
commit 1bb1f70a47
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 79 additions and 2 deletions

View file

@ -5180,6 +5180,7 @@ class Router:
# check if user wants to do tag based routing
healthy_deployments = await get_deployments_for_tag( # type: ignore
llm_router_instance=self,
model=model,
request_kwargs=request_kwargs,
healthy_deployments=healthy_deployments,
)