phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-27 11:43:54 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge pull request #5579 from BerriAI/litellm_set_redis_cluster_env

Browse source 
[Feat] Allow setting up Redis Cluster using .env vars
This commit is contained in:
Ishaan Jaff 2024-09-07 11:31:38 -07:00 • committed by GitHub
commit 59d1ccd34b
4 changed files with 139 additions and 15 deletions
Download patch file Download diff file Expand all files Collapse all files

42
docs/my-website/docs/proxy/caching.md
View file

@ -54,6 +54,10 @@ litellm_caching:<hash>
#### Redis Cluster
<Tabs>
<TabItem value="redis-cluster-config" label="Set on config.yaml">
```yaml
model_list:
- model_name: "*"
@ -68,6 +72,44 @@ litellm_settings:
redis_startup_nodes: [{"host": "127.0.0.1", "port": "7001"}]
```
</TabItem>
<TabItem value="redis-env" label="Set on .env">
You can configure redis cluster in your .env by setting `REDIS_CLUSTER_NODES` in your .env
**Example `REDIS_CLUSTER_NODES`** value
```
REDIS_CLUSTER_NODES = "[{"host": "127.0.0.1", "port": "7001"}, {"host": "127.0.0.1", "port": "7003"}, {"host": "127.0.0.1", "port": "7004"}, {"host": "127.0.0.1", "port": "7005"}, {"host": "127.0.0.1", "port": "7006"}, {"host": "127.0.0.1", "port": "7007"}]"
```
:::note
Example python script for setting redis cluster nodes in .env:
```python
# List of startup nodes
startup_nodes = [
{"host": "127.0.0.1", "port": "7001"},
{"host": "127.0.0.1", "port": "7003"},
{"host": "127.0.0.1", "port": "7004"},
{"host": "127.0.0.1", "port": "7005"},
{"host": "127.0.0.1", "port": "7006"},
{"host": "127.0.0.1", "port": "7007"},
]
# set startup nodes in environment variables
os.environ["REDIS_CLUSTER_NODES"] = json.dumps(startup_nodes)
print("REDIS_CLUSTER_NODES", os.environ["REDIS_CLUSTER_NODES"])
```
:::
</TabItem>
</Tabs>
#### TTL
```yaml