phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-26 03:04:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Cost tracking improvements (#5828)

Browse source 
* feat(litellm_logging.py): update standard logging payload to include debug information for cost failures

Also includes fixes for cohere rerank cost tracking + databricks llama2 model cost tracking

 Easier to repro cost failures and improve reliability in prod

* fix(proxy_server.py): emit cost failure debug info for slack alerting

Improves debug information for cost tracking failures, on slack alerting
This commit is contained in:
Krish Dholakia 2024-09-21 21:47:50 -07:00 • committed by GitHub
parent 8039b95aaf
commit 2488e4b45f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 117 additions and 45 deletions
Download patch file Download diff file Expand all files Collapse all files

4
litellm/llms/databricks/cost_calculator.py
View file

@ -49,6 +49,10 @@ def cost_per_token(model: str, usage: Usage) -> Tuple[float, float]:
"gte-large-en"
):
base_model = "databricks-gte-large-en"
elif model.startswith("databricks/llama-2-70b-chat") or model.startswith(
"llama-2-70b-chat"
):
base_model = "databricks-llama-2-70b-chat"
## GET MODEL INFO
model_info = get_model_info(model=base_model, custom_llm_provider="databricks")