phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-25 10:44:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

(Feat) soft budget alerts on keys (#7623)

Browse source 
* class WebhookEvent(CallInfo):
Add

* handle soft budget alerts

* handle soft budget

* fix budget alerts

* fix CallInfo

* fix _get_user_info_str

* test_soft_budget_alerts

* test_soft_budget_alert
This commit is contained in:
Ishaan Jaff 2025-01-07 21:36:34 -08:00 • committed by GitHub
parent 4e69711411
commit 081826a5d6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 201 additions and 33 deletions
Download patch file Download diff file Expand all files Collapse all files

24
litellm/proxy/auth/user_api_key_auth.py
View file

@ -1014,6 +1014,30 @@ async def user_api_key_auth( # noqa: PLR0915
current_cost=valid_token.spend,
max_budget=valid_token.max_budget,
)
if valid_token.soft_budget and valid_token.spend >= valid_token.soft_budget:
verbose_proxy_logger.debug(
"Crossed Soft Budget for token %s, spend %s, soft_budget %s",
valid_token.token,
valid_token.spend,
valid_token.soft_budget,
)
call_info = CallInfo(
token=valid_token.token,
spend=valid_token.spend,
max_budget=valid_token.max_budget,
soft_budget=valid_token.soft_budget,
user_id=valid_token.user_id,
team_id=valid_token.team_id,
team_alias=valid_token.team_alias,
user_email=None,
key_alias=valid_token.key_alias,
)
asyncio.create_task(
proxy_logging_obj.budget_alerts(
type="soft_budget",
user_info=call_info,
)
)
# Check 5. Token Model Spend is under Model budget
max_budget_per_model = valid_token.model_max_budget