docs: adds redis url to router + proxy docs

This commit is contained in:
Krrish Dholakia 2023-12-05 15:08:00 -08:00
parent 4d7ff1b33b
commit 68ca2a28d4
4 changed files with 32 additions and 4 deletions

View file

@ -1,11 +1,14 @@
# Redis Cache # Redis Cache
[**See Code**](https://github.com/BerriAI/litellm/blob/4d7ff1b33b9991dcf38d821266290631d9bcd2dd/litellm/caching.py#L71)
### Pre-requisites ### Pre-requisites
Install redis Install redis
``` ```
pip install redis pip install redis
``` ```
For the hosted version you can setup your own Redis DB here: https://app.redislabs.com/ For the hosted version you can setup your own Redis DB here: https://app.redislabs.com/
### Usage ### Quick Start
```python ```python
import litellm import litellm
from litellm import completion from litellm import completion
@ -55,6 +58,11 @@ litellm.cache = cache # set litellm.cache to your cache
### Detecting Cached Responses ### Detecting Cached Responses
For resposes that were returned as cache hit, the response includes a param `cache` = True For resposes that were returned as cache hit, the response includes a param `cache` = True
:::info
Only valid for OpenAI <= 0.28.1 [Let us know if you still need this](https://github.com/BerriAI/litellm/issues/new?assignees=&labels=bug&projects=&template=bug_report.yml&title=%5BBug%5D%3A+)
:::
Example response with cache hit Example response with cache hit
```python ```python
{ {

View file

@ -12,17 +12,27 @@ model_list:
litellm_settings: litellm_settings:
set_verbose: True set_verbose: True
cache: # init cache cache: # init cache
type: redis # tell litellm to use redis caching type: redis # tell litellm to use redis caching (Also: `pip install redis`)
``` ```
#### Step 2: Add Redis Credentials to .env #### Step 2: Add Redis Credentials to .env
LiteLLM requires the following REDIS credentials in your env to enable caching Set either `REDIS_URL` or the `REDIS_HOST` in your os environment, to enable caching.
```shell ```shell
REDIS_URL = "" # REDIS_URL='redis://username:password@hostname:port/database'
## OR ##
REDIS_HOST = "" # REDIS_HOST='redis-18841.c274.us-east-1-3.ec2.cloud.redislabs.com' REDIS_HOST = "" # REDIS_HOST='redis-18841.c274.us-east-1-3.ec2.cloud.redislabs.com'
REDIS_PORT = "" # REDIS_PORT='18841' REDIS_PORT = "" # REDIS_PORT='18841'
REDIS_PASSWORD = "" # REDIS_PASSWORD='liteLlmIsAmazing' REDIS_PASSWORD = "" # REDIS_PASSWORD='liteLlmIsAmazing'
``` ```
**Additional kwargs**
You can pass in any additional redis.Redis arg, by storing the variable + value in your os environment, like this:
```shell
REDIS_<redis-kwarg-name> = ""
```
[**See how it's read from the environment**](https://github.com/BerriAI/litellm/blob/4d7ff1b33b9991dcf38d821266290631d9bcd2dd/litellm/_redis.py#L40)
#### Step 3: Run proxy with config #### Step 3: Run proxy with config
```shell ```shell
$ litellm --config /path/to/config.yaml $ litellm --config /path/to/config.yaml

View file

@ -356,6 +356,16 @@ router = Router(model_list=model_list,
print(response) print(response)
``` ```
**Pass in Redis URL, additional kwargs**
```python
router = Router(model_list: Optional[list] = None,
## CACHING ##
redis_url=os.getenv("REDIS_URL")",
cache_kwargs= {}, # additional kwargs to pass to RedisCache (see caching.py)
cache_responses=True)
```
#### Default litellm.completion/embedding params #### Default litellm.completion/embedding params
You can also set default params for litellm completion/embedding calls. Here's how to do that: You can also set default params for litellm completion/embedding calls. Here's how to do that:

View file

@ -36,7 +36,7 @@ def test_caching_v2(): # test in memory cache
print(f"error occurred: {traceback.format_exc()}") print(f"error occurred: {traceback.format_exc()}")
pytest.fail(f"Error occurred: {e}") pytest.fail(f"Error occurred: {e}")
# test_caching_v2() test_caching_v2()