phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

docs(load_balancing.md): have proxy routing docs point to sdk routing docs

Browse source
This commit is contained in:
Krrish Dholakia 2024-02-23 08:34:08 -08:00
parent e049858266
commit d1a40ecacf
1 changed files with 8 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

8
docs/my-website/docs/proxy/load_balancing.md
View file

@ -2,6 +2,14 @@
Load balance multiple instances of the same model
The proxy will handle routing requests (using LiteLLM's Router). **Set `rpm` in the config if you want maximize throughput**
:::info
For more details on routing strategies / params, see [Routing](../routing.md)
:::
## Quick Start - Load Balancing
### Step 1 - Set deployments on config