phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-25 18:54:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix(utils.py): return openai streaming prompt caching tokens (#6051)

Browse source 
* fix(utils.py): return openai streaming prompt caching tokens

Closes https://github.com/BerriAI/litellm/issues/6038

* fix(main.py): fix error in finish_reason updates
This commit is contained in:
Krish Dholakia 2024-10-03 22:20:13 -04:00 • committed by GitHub
parent e97eaea0a3
commit 0774b5eb19
5 changed files with 91 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

4
litellm/utils.py
View file

@ -7813,9 +7813,7 @@ class CustomStreamWrapper:
)
elif isinstance(response_obj["usage"], BaseModel):
model_response.usage = litellm.Usage(
prompt_tokens=response_obj["usage"].prompt_tokens,
completion_tokens=response_obj["usage"].completion_tokens,
total_tokens=response_obj["usage"].total_tokens,
**response_obj["usage"].model_dump()
)
model_response.model = self.model