phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

Update README.md

Browse source
This commit is contained in:
Krish Dholakia 2023-10-10 07:15:10 -07:00 • committed by GitHub
parent ed832a8111
commit 7a0dc6487b
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 0 additions and 29 deletions
Download patch file Download diff file Expand all files Collapse all files

29
README.md
View file

@ -80,35 +80,6 @@ for chunk in result:
print(chunk['choices'][0]['delta']) print(chunk['choices'][0]['delta'])
``` ```
## Caching ([Docs](https://docs.litellm.ai/docs/caching/))
LiteLLM supports caching `completion()` and `embedding()` calls for all LLMs. [Hosted Cache LiteLLM API](https://docs.litellm.ai/docs/caching/caching_api)
```python
import litellm
from litellm.caching import Cache
import os
litellm.cache = Cache()
os.environ['OPENAI_API_KEY'] = ""
# add to cache
response1 = litellm.completion(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "why is LiteLLM amazing?"}],
caching=True
)
# returns cached response
response2 = litellm.completion(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "why is LiteLLM amazing?"}],
caching=True
)
print(f"response1: {response1}")
print(f"response2: {response2}")
```
## OpenAI Proxy Server ([Docs](https://docs.litellm.ai/docs/proxy_server)) ## OpenAI Proxy Server ([Docs](https://docs.litellm.ai/docs/proxy_server))
Spin up a local server to translate openai api calls to any non-openai model (e.g. Huggingface, TogetherAI, Ollama, etc.) Spin up a local server to translate openai api calls to any non-openai model (e.g. Huggingface, TogetherAI, Ollama, etc.)