phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

feat(proxy_server.py): retry if virtual key is rate limited

Browse source 
currently for chat completions
This commit is contained in:
Krrish Dholakia 2024-03-05 19:00:03 -08:00
parent f95458dad8
commit ad55f4dbb5
4 changed files with 57 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
proxy_server_config.yaml
View file

@ -38,6 +38,8 @@ litellm_settings:
drop_params: True
max_budget: 100
budget_duration: 30d
num_retries: 5
request_timeout: 600
general_settings:
master_key: sk-1234 # [OPTIONAL] Only use this if you to require all calls to contain this key (Authorization: Bearer sk-1234)
proxy_budget_rescheduler_min_time: 10