phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

fix(litellm_logging.py): fix async caching for sync streaming calls (don't do it)

Browse source 
Checks if call is async before running async caching for streaming call

Fixes https://github.com/BerriAI/litellm/issues/4511#issuecomment-2233211808
This commit is contained in:
Krrish Dholakia 2024-07-17 11:15:30 -07:00
parent e7f8ee2aba
commit d2e64f21f3
2 changed files with 22 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

9
litellm/litellm_core_utils/litellm_logging.py
View file

@ -1337,7 +1337,14 @@ class Logging:
if kwargs.get("no-log", False) == True:
print_verbose("no-log request, skipping logging")
continue
if callback == "cache" and litellm.cache is not None:
if (
callback == "cache"
and litellm.cache is not None
and self.model_call_details.get("litellm_params", {}).get(
"acompletion", False
)
is True
):
# set_cache once complete streaming response is built
print_verbose("async success_callback: reaches cache for logging!")
kwargs = self.model_call_details