From b70640e969d34e978033d807b362c532b509fe55 Mon Sep 17 00:00:00 2001 From: Ishaan Jaff Date: Mon, 2 Oct 2023 12:25:01 -0700 Subject: [PATCH] Update README.md --- README.md | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index d1f480079..2aed37263 100644 --- a/README.md +++ b/README.md @@ -74,18 +74,31 @@ for chunk in result: ## Caching ([Docs](https://docs.litellm.ai/docs/caching/)) -LiteLLM supports caching `completion()` and `embedding()` calls for all LLMs + +LiteLLM supports caching `completion()` and `embedding()` calls for all LLMs. [Hosted Cache LiteLLM API](https://docs.litellm.ai/docs/caching/caching_api) ```python import litellm from litellm.caching import Cache -litellm.cache = Cache() # init cache to use api.litellm.ai +import os -# stores this response in cache +litellm.cache = Cache() +os.environ['OPENAI_API_KEY'] = "" +# add to cache response1 = litellm.completion( - model="gpt-3.5-turbo", - messages=[{"role": "user", "content": "Tell me a joke."}] + model="gpt-3.5-turbo", + messages=[{"role": "user", "content": "why is LiteLLM amazing?"}], caching=True ) +# returns cached response +response2 = litellm.completion( + model="gpt-3.5-turbo", + messages=[{"role": "user", "content": "why is LiteLLM amazing?"}], + caching=True +) + +print(f"response1: {response1}") +print(f"response2: {response2}") + ``` ## OpenAI Proxy Server ([Docs](https://docs.litellm.ai/docs/proxy_server))