phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-26 03:04:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix(utils.py): use preset cache key for async calls as well

Browse source
This commit is contained in:
Krrish Dholakia 2024-01-11 19:30:28 +05:30
parent f8d1720690
commit a1af7688ce
1 changed files with 4 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

4
litellm/utils.py
View file

@ -2241,6 +2241,10 @@ def client(original_function):
)
cached_result = await asyncio.gather(*tasks)
else:
preset_cache_key = litellm.cache.get_cache_key(*args, **kwargs)
kwargs[
"preset_cache_key"
] = preset_cache_key # for streaming calls, we need to pass the preset_cache_key
cached_result = litellm.cache.get_cache(*args, **kwargs)
if cached_result is not None and not isinstance(
cached_result, list