phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

add doc on spend report frequency

Browse source
This commit is contained in:
Ishaan Jaff 2024-09-07 11:54:33 -07:00
parent 805e4c5754
commit ecb774c3e8
2 changed files with 14 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

21
docs/my-website/docs/proxy/alerting.md
View file

@ -45,6 +45,7 @@ export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/<>/<>/<>"
general_settings: general_settings:
alerting: ["slack"] alerting: ["slack"]
alerting_threshold: 300 # sends alerts if requests hang for 5min+ and responses take 5min+ alerting_threshold: 300 # sends alerts if requests hang for 5min+ and responses take 5min+
spend_report_frequency: "1d" # [Optional] set as 1d, 2d, 30d .... Specifiy how often you want a Spend Report to be sent
``` ```
Start proxy Start proxy
@ -61,7 +62,9 @@ curl -X GET 'http://0.0.0.0:4000/health/services?service=slack' \
-H 'Authorization: Bearer sk-1234' -H 'Authorization: Bearer sk-1234'
``` ```
## Advanced - Redacting Messages from Alerts ## Advanced
### Redacting Messages from Alerts
By default alerts show the `messages/input` passed to the LLM. If you want to redact this from slack alerting set the following setting on your config By default alerts show the `messages/input` passed to the LLM. If you want to redact this from slack alerting set the following setting on your config
@ -76,7 +79,7 @@ litellm_settings:
``` ```
## Advanced - Add Metadata to alerts ### Add Metadata to alerts
Add alerting metadata to proxy calls for debugging. Add alerting metadata to proxy calls for debugging.
@ -105,7 +108,7 @@ response = client.chat.completions.create(
<Image img={require('../../img/alerting_metadata.png')}/> <Image img={require('../../img/alerting_metadata.png')}/>
## Advanced - Opting into specific alert types ### Opting into specific alert types
Set `alert_types` if you want to Opt into only specific alert types Set `alert_types` if you want to Opt into only specific alert types
@ -134,7 +137,7 @@ AlertType = Literal[
``` ```
## Advanced - set specific slack channels per alert type ### Set specific slack channels per alert type
Use this if you want to set specific channels per alert type Use this if you want to set specific channels per alert type
@ -190,7 +193,7 @@ curl -i http://localhost:4000/v1/chat/completions \
``` ```
## Advanced - provide multiple slack channels for a given alert type ### Provide multiple slack channels for a given alert type
Just add it like this - `alert_type: [<hook_url_channel_1>, <hook_url_channel_2>]`. Just add it like this - `alert_type: [<hook_url_channel_1>, <hook_url_channel_2>]`.
@ -220,7 +223,7 @@ curl -X GET 'http://0.0.0.0:4000/health/services?service=slack' \
In case of error, check server logs for the error message! In case of error, check server logs for the error message!
## Advanced - Using MS Teams Webhooks ### Using MS Teams Webhooks
MS Teams provides a slack compatible webhook url that you can use for alerting MS Teams provides a slack compatible webhook url that you can use for alerting
@ -262,7 +265,7 @@ curl --location 'http://0.0.0.0:4000/health/services?service=slack' \
<Image img={require('../../img/ms_teams_alerting.png')}/> <Image img={require('../../img/ms_teams_alerting.png')}/>
## Advanced - Using Discord Webhooks ### Using Discord Webhooks
Discord provides a slack compatible webhook url that you can use for alerting Discord provides a slack compatible webhook url that you can use for alerting
@ -294,7 +297,7 @@ environment_variables:
``` ```
## Advanced - [BETA] Webhooks for Budget Alerts ## [BETA] Webhooks for Budget Alerts
**Note**: This is a beta feature, so the spec might change. **Note**: This is a beta feature, so the spec might change.
@ -374,7 +377,7 @@ curl -X GET --location 'http://0.0.0.0:4000/health/services?service=webhook' \
- `event_message` *str*: A human-readable description of the event. - `event_message` *str*: A human-readable description of the event.
## Advanced - Region-outage alerting (✨ Enterprise feature) ## Region-outage alerting (✨ Enterprise feature)
:::info :::info
[Get a free 2-week license](https://forms.gle/P518LXsAZ7PhXpDn8) [Get a free 2-week license](https://forms.gle/P518LXsAZ7PhXpDn8)

2
litellm/proxy/proxy_config.yaml
View file

@ -17,4 +17,6 @@ guardrails:
general_settings: general_settings:
master_key: sk-1234 master_key: sk-1234
alerting: ["slack"]
spend_report_frequency: "1d"