phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-27 11:43:54 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

LiteLLM Minor Fixes and Improvements (09/10/2024) (#5618)

Browse source 
* fix(cost_calculator.py): move to debug for noisy warning message on cost calculation error

Fixes https://github.com/BerriAI/litellm/issues/5610

* fix(databricks/cost_calculator.py): Handles model name issues for databricks models

* fix(main.py): fix stream chunk builder for multiple tool calls

Fixes https://github.com/BerriAI/litellm/issues/5591

* fix: correctly set user_alias when passed in

Fixes https://github.com/BerriAI/litellm/issues/5612

* fix(types/utils.py): allow passing role for message object

https://github.com/BerriAI/litellm/issues/5621

* fix(litellm_logging.py): Fix langfuse logging across multiple projects

Fixes issue where langfuse logger was re-using the old logging object

* feat(proxy/_types.py): support adding key-based tags for tag-based routing

Enable tag based routing at key-level

* fix(proxy/_types.py): fix inheritance

* test(test_key_generate_prisma.py): fix test

* test: fix test

* fix(litellm_logging.py): return used callback object
This commit is contained in:
Krish Dholakia 2024-09-11 11:30:29 -07:00 • committed by GitHub
parent d6e0d5d234
commit 7f47c48b35
15 changed files with 673 additions and 96 deletions
Download patch file Download diff file Expand all files Collapse all files

25
litellm/llms/databricks/cost_calculator.py
View file

@ -25,7 +25,30 @@ def cost_per_token(model: str, usage: Usage) -> Tuple[float, float]:
"dbrx-instruct"
):
base_model = "databricks-dbrx-instruct"
elif model.startswith("databricks/meta-llama-3.1-70b-instruct") or model.startswith(
"meta-llama-3.1-70b-instruct"
):
base_model = "databricks-meta-llama-3-1-70b-instruct"
elif model.startswith(
"databricks/meta-llama-3.1-405b-instruct"
) or model.startswith("meta-llama-3.1-405b-instruct"):
base_model = "databricks-meta-llama-3-1-405b-instruct"
elif model.startswith("databricks/mixtral-8x7b-instruct-v0.1") or model.startswith(
"mixtral-8x7b-instruct-v0.1"
):
base_model = "databricks-mixtral-8x7b-instruct"
elif model.startswith("databricks/mixtral-8x7b-instruct-v0.1") or model.startswith(
"mixtral-8x7b-instruct-v0.1"
):
base_model = "databricks-mixtral-8x7b-instruct"
elif model.startswith("databricks/bge-large-en") or model.startswith(
"bge-large-en"
):
base_model = "databricks-bge-large-en"
elif model.startswith("databricks/gte-large-en") or model.startswith(
"gte-large-en"
):
base_model = "databricks-gte-large-en"
## GET MODEL INFO
model_info = get_model_info(model=base_model, custom_llm_provider="databricks")