From 7a0dc6487b2b2c135bf2e88577e9c15b72da4120 Mon Sep 17 00:00:00 2001 From: Krish Dholakia Date: Tue, 10 Oct 2023 07:15:10 -0700 Subject: [PATCH] Update README.md --- README.md | 29 ----------------------------- 1 file changed, 29 deletions(-) diff --git a/README.md b/README.md index 7fef0426d..a0f82bb9a 100644 --- a/README.md +++ b/README.md @@ -80,35 +80,6 @@ for chunk in result: print(chunk['choices'][0]['delta']) ``` - -## Caching ([Docs](https://docs.litellm.ai/docs/caching/)) - -LiteLLM supports caching `completion()` and `embedding()` calls for all LLMs. [Hosted Cache LiteLLM API](https://docs.litellm.ai/docs/caching/caching_api) -```python -import litellm -from litellm.caching import Cache -import os - -litellm.cache = Cache() -os.environ['OPENAI_API_KEY'] = "" -# add to cache -response1 = litellm.completion( - model="gpt-3.5-turbo", - messages=[{"role": "user", "content": "why is LiteLLM amazing?"}], - caching=True -) -# returns cached response -response2 = litellm.completion( - model="gpt-3.5-turbo", - messages=[{"role": "user", "content": "why is LiteLLM amazing?"}], - caching=True -) - -print(f"response1: {response1}") -print(f"response2: {response2}") - -``` - ## OpenAI Proxy Server ([Docs](https://docs.litellm.ai/docs/proxy_server)) Spin up a local server to translate openai api calls to any non-openai model (e.g. Huggingface, TogetherAI, Ollama, etc.)