diff --git a/docs/my-website/docs/caching/redis_cache.md b/docs/my-website/docs/caching/redis_cache.md index f0fcc6952..521c4d00f 100644 --- a/docs/my-website/docs/caching/redis_cache.md +++ b/docs/my-website/docs/caching/redis_cache.md @@ -1,11 +1,14 @@ # Redis Cache + +[**See Code**](https://github.com/BerriAI/litellm/blob/4d7ff1b33b9991dcf38d821266290631d9bcd2dd/litellm/caching.py#L71) + ### Pre-requisites Install redis ``` pip install redis ``` For the hosted version you can setup your own Redis DB here: https://app.redislabs.com/ -### Usage +### Quick Start ```python import litellm from litellm import completion @@ -55,6 +58,11 @@ litellm.cache = cache # set litellm.cache to your cache ### Detecting Cached Responses For resposes that were returned as cache hit, the response includes a param `cache` = True +:::info + +Only valid for OpenAI <= 0.28.1 [Let us know if you still need this](https://github.com/BerriAI/litellm/issues/new?assignees=&labels=bug&projects=&template=bug_report.yml&title=%5BBug%5D%3A+) +::: + Example response with cache hit ```python { diff --git a/docs/my-website/docs/proxy/caching.md b/docs/my-website/docs/proxy/caching.md index d052102db..56e4b4c1c 100644 --- a/docs/my-website/docs/proxy/caching.md +++ b/docs/my-website/docs/proxy/caching.md @@ -12,17 +12,27 @@ model_list: litellm_settings: set_verbose: True cache: # init cache - type: redis # tell litellm to use redis caching + type: redis # tell litellm to use redis caching (Also: `pip install redis`) ``` #### Step 2: Add Redis Credentials to .env -LiteLLM requires the following REDIS credentials in your env to enable caching +Set either `REDIS_URL` or the `REDIS_HOST` in your os environment, to enable caching. ```shell + REDIS_URL = "" # REDIS_URL='redis://username:password@hostname:port/database' + ## OR ## REDIS_HOST = "" # REDIS_HOST='redis-18841.c274.us-east-1-3.ec2.cloud.redislabs.com' REDIS_PORT = "" # REDIS_PORT='18841' REDIS_PASSWORD = "" # REDIS_PASSWORD='liteLlmIsAmazing' ``` + +**Additional kwargs** +You can pass in any additional redis.Redis arg, by storing the variable + value in your os environment, like this: +```shell +REDIS_ = "" +``` + +[**See how it's read from the environment**](https://github.com/BerriAI/litellm/blob/4d7ff1b33b9991dcf38d821266290631d9bcd2dd/litellm/_redis.py#L40) #### Step 3: Run proxy with config ```shell $ litellm --config /path/to/config.yaml diff --git a/docs/my-website/docs/routing.md b/docs/my-website/docs/routing.md index c0ca93b25..3f55ae28e 100644 --- a/docs/my-website/docs/routing.md +++ b/docs/my-website/docs/routing.md @@ -356,6 +356,16 @@ router = Router(model_list=model_list, print(response) ``` + +**Pass in Redis URL, additional kwargs** +```python +router = Router(model_list: Optional[list] = None, + ## CACHING ## + redis_url=os.getenv("REDIS_URL")", + cache_kwargs= {}, # additional kwargs to pass to RedisCache (see caching.py) + cache_responses=True) +``` + #### Default litellm.completion/embedding params You can also set default params for litellm completion/embedding calls. Here's how to do that: diff --git a/litellm/tests/test_caching.py b/litellm/tests/test_caching.py index 713f97b3e..a0980e9de 100644 --- a/litellm/tests/test_caching.py +++ b/litellm/tests/test_caching.py @@ -36,7 +36,7 @@ def test_caching_v2(): # test in memory cache print(f"error occurred: {traceback.format_exc()}") pytest.fail(f"Error occurred: {e}") -# test_caching_v2() +test_caching_v2()