forked from phoenix/litellm-mirror
update provider budget routing
This commit is contained in:
parent
cf76f308de
commit
ac4ecce2bc
1 changed files with 19 additions and 5 deletions
|
@ -22,14 +22,14 @@ router_settings:
|
|||
provider_budget_config:
|
||||
openai:
|
||||
budget_limit: 0.000000000001 # float of $ value budget for time period
|
||||
time_period: 1d # can be 1d, 2d, 30d
|
||||
time_period: 1d # can be 1d, 2d, 30d, 1mo, 2mo
|
||||
azure:
|
||||
budget_limit: 100
|
||||
time_period: 1d
|
||||
anthropic:
|
||||
budget_limit: 100
|
||||
time_period: 10d
|
||||
vertexai:
|
||||
vertex_ai:
|
||||
budget_limit: 100
|
||||
time_period: 12d
|
||||
gemini:
|
||||
|
@ -112,8 +112,11 @@ Expected response on failure
|
|||
- If all providers exceed budget, raises an error
|
||||
|
||||
3. **Supported Time Periods**:
|
||||
- Format: "Xd" where X is number of days
|
||||
- Examples: "1d" (1 day), "30d" (30 days)
|
||||
- Seconds: "Xs" (e.g., "30s")
|
||||
- Minutes: "Xm" (e.g., "10m")
|
||||
- Hours: "Xh" (e.g., "24h")
|
||||
- Days: "Xd" (e.g., "1d", "30d")
|
||||
- Months: "Xmo" (e.g., "1mo", "2mo")
|
||||
|
||||
4. **Requirements**:
|
||||
- Redis required for tracking spend across instances
|
||||
|
@ -136,7 +139,12 @@ The `provider_budget_config` is a dictionary where:
|
|||
- **Key**: Provider name (string) - Must be a valid [LiteLLM provider name](https://docs.litellm.ai/docs/providers)
|
||||
- **Value**: Budget configuration object with the following parameters:
|
||||
- `budget_limit`: Float value representing the budget in USD
|
||||
- `time_period`: String in the format "Xd" where X is the number of days (e.g., "1d", "30d")
|
||||
- `time_period`: Duration string in one of the following formats:
|
||||
- Seconds: `"Xs"` (e.g., "30s")
|
||||
- Minutes: `"Xm"` (e.g., "10m")
|
||||
- Hours: `"Xh"` (e.g., "24h")
|
||||
- Days: `"Xd"` (e.g., "1d", "30d")
|
||||
- Months: `"Xmo"` (e.g., "1mo", "2mo")
|
||||
|
||||
Example structure:
|
||||
```yaml
|
||||
|
@ -147,4 +155,10 @@ provider_budget_config:
|
|||
azure:
|
||||
budget_limit: 500.0 # $500 USD
|
||||
time_period: "30d" # 30 day period
|
||||
anthropic:
|
||||
budget_limit: 200.0 # $200 USD
|
||||
time_period: "1mo" # 1 month period
|
||||
gemini:
|
||||
budget_limit: 50.0 # $50 USD
|
||||
time_period: "24h" # 24 hour period
|
||||
```
|
Loading…
Add table
Add a link
Reference in a new issue