phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

Merge pull request #5402 from ajeetdsouza/patch-1

Browse source 
docs: add time.sleep() between streaming calls
This commit is contained in:
Ishaan Jaff 2024-08-28 07:38:15 -07:00 • committed by GitHub
commit 3df1186d72
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 4 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

5
docs/my-website/docs/caching/caching_api.md
View file

@ -51,8 +51,10 @@ LiteLLM can cache your streamed responses for you
### Usage ### Usage
```python ```python
import litellm import litellm
import time
from litellm import completion from litellm import completion
from litellm.caching import Cache from litellm.caching import Cache
litellm.cache = Cache(type="hosted") litellm.cache = Cache(type="hosted")
# Make completion calls # Make completion calls
@ -64,6 +66,7 @@ response1 = completion(
for chunk in response1: for chunk in response1:
print(chunk) print(chunk)
time.sleep(1) # cache is updated asynchronously
response2 = completion( response2 = completion(
model="gpt-3.5-turbo", model="gpt-3.5-turbo",
@ -72,4 +75,4 @@ response2 = completion(
caching=True) caching=True)
for chunk in response2: for chunk in response2:
print(chunk) print(chunk)
``` ```