phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-25 10:44:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

(feat) openai prompt caching (non streaming) - add prompt_tokens_details in usage response (#6039)

Browse source 
* add prompt_tokens_details in usage response

* use _prompt_tokens_details as a param in Usage

* fix linting errors

* fix type error

* fix ci/cd deps

* bump deps for openai

* bump deps openai

* fix llm translation testing

* fix llm translation embedding
This commit is contained in:
Ishaan Jaff 2024-10-03 11:01:10 -07:00 • committed by GitHub
parent 9fccb4a0da
commit 4e88fd65e1
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 1515 additions and 1428 deletions
Download patch file Download diff file Expand all files Collapse all files

2
tests/llm_translation/test_databricks.py
View file

@ -46,6 +46,7 @@ def mock_chat_response() -> Dict[str, Any]:
"completion_tokens": 38,
"completion_tokens_details": None,
"total_tokens": 268,
"prompt_tokens_details": None,
},
"system_fingerprint": None,
}
@ -201,6 +202,7 @@ def mock_embedding_response() -> Dict[str, Any]:
"total_tokens": 8,
"completion_tokens": 0,
"completion_tokens_details": None,
"prompt_tokens_details": None,
},
}