phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

doc provider budget routing

Browse source
This commit is contained in:
Ishaan Jaff 2024-11-19 17:29:52 -08:00
parent 98a7a37c86
commit d5937c820f
1 changed files with 1 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

13
docs/my-website/docs/proxy/provider_budget_routing.md
View file

@ -4,9 +4,6 @@ import TabItem from '@theme/TabItem';
# Provider Budget Routing # Provider Budget Routing
Use this to set budgets for LLM Providers - example $100/day for OpenAI, $100/day for Azure. Use this to set budgets for LLM Providers - example $100/day for OpenAI, $100/day for Azure.
<Tabs>
<TabItem value="proxy-config" label="LiteLLM Proxy Config.yaml">
```yaml ```yaml
model_list: model_list:
- model_name: gpt-3.5-turbo - model_name: gpt-3.5-turbo
@ -21,11 +18,10 @@ model_list:
api_base: os.environ/AZURE_API_BASE api_base: os.environ/AZURE_API_BASE
router_settings: router_settings:
routing_strategy: provider-budget-routing
redis_host: <your-redis-host> redis_host: <your-redis-host>
redis_password: <your-redis-password> redis_password: <your-redis-password>
redis_port: <your-redis-port> redis_port: <your-redis-port>
routing_strategy_args: provider_budget_config:
openai: openai:
budget_limit: 0.000000000001 # float of $ value budget for time period budget_limit: 0.000000000001 # float of $ value budget for time period
time_period: 1d # can be 1d, 2d, 30d time_period: 1d # can be 1d, 2d, 30d
@ -46,13 +42,6 @@ general_settings:
master_key: sk-1234 master_key: sk-1234
``` ```
</TabItem>
<TabItem value="python-sdk" label="Python SDK">
</TabItem>
</Tabs>
#### How provider-budget-routing works #### How provider-budget-routing works