phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-26 03:04:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Litellm dev 12 11 2024 v2 (#7215)

Browse source 
* feat(bedrock/): add bedrock converse top k param

Closes https://github.com/BerriAI/litellm/issues/7087

* Fix bedrock empty content error (#7177)

* add resolver

* handle empty content on bedrock with default content

* use existing default message, tests

* Update tests/llm_translation/test_bedrock_completion.py

* fix tests

* Revert "add resolver"

This reverts commit c717e376ee.

* fallback to empty

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* fix(factory.py): handle empty content blocks in messages

Fixes https://github.com/BerriAI/litellm/issues/7169

* feat(router.py): add stripped model check to model fallback search

if model_name="openai/gpt-3.5-turbo" and fallback=[{"gpt-3.5-turbo"..}] the fallback should just work as expected

* fix: fix linting error

* fix(factory.py): fix linting error

* fix(factory.py): in base case still support skip empty text blocks

---------

Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
This commit is contained in:
Krish Dholakia 2024-12-13 12:49:57 -08:00 • committed by GitHub
parent a42f008cd0
commit 550677e63d
9 changed files with 569 additions and 60 deletions
Download patch file Download diff file Expand all files Collapse all files

20
litellm/router.py
View file

@ -79,6 +79,7 @@ from litellm.router_utils.cooldown_handlers import (
_set_cooldown_deployments,
)
from litellm.router_utils.fallback_event_handlers import (
get_fallback_model_group,
log_failure_fallback_event,
log_success_fallback_event,
run_async_fallback,
@ -2753,19 +2754,14 @@ class Router:
)
e.message += "\n{}".format(error_message)
if fallbacks is not None:
if fallbacks is not None and model_group is not None:
verbose_router_logger.debug(f"inside model fallbacks: {fallbacks}")
generic_fallback_idx: Optional[int] = None
## check for specific model group-specific fallbacks
for idx, item in enumerate(fallbacks):
if isinstance(item, dict):
if list(item.keys())[0] == model_group:
fallback_model_group = item[model_group]
break
elif list(item.keys())[0] == "*":
generic_fallback_idx = idx
elif isinstance(item, str):
fallback_model_group = [fallbacks.pop(idx)]
fallback_model_group, generic_fallback_idx = (
get_fallback_model_group(
fallbacks=fallbacks,
model_group=cast(str, model_group),
)
)
## if none, check for generic fallback
if (
fallback_model_group is None