forked from phoenix/litellm-mirror
docs(alerting.md): add alerting metadata to docs
This commit is contained in:
parent
9e8eacbc77
commit
4eca63ede6
2 changed files with 40 additions and 21 deletions
|
@ -32,41 +32,33 @@ Get a slack webhook url from https://api.slack.com/messaging/webhooks
|
|||
|
||||
You can also use Discord Webhooks, see [here](#using-discord-webhooks)
|
||||
|
||||
### Step 2: Update config.yaml
|
||||
|
||||
- Set `SLACK_WEBHOOK_URL` in your proxy env to enable Slack alerts.
|
||||
- Just for testing purposes, let's save a bad key to our proxy.
|
||||
Set `SLACK_WEBHOOK_URL` in your proxy env to enable Slack alerts.
|
||||
|
||||
```bash
|
||||
export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/<>/<>/<>"
|
||||
```
|
||||
|
||||
### Step 2: Setup Proxy
|
||||
|
||||
```yaml
|
||||
model_list:
|
||||
model_name: "azure-model"
|
||||
litellm_params:
|
||||
model: "azure/gpt-35-turbo"
|
||||
api_key: "my-bad-key" # 👈 bad key
|
||||
|
||||
general_settings:
|
||||
alerting: ["slack"]
|
||||
alerting_threshold: 300 # sends alerts if requests hang for 5min+ and responses take 5min+
|
||||
|
||||
environment_variables:
|
||||
SLACK_WEBHOOK_URL: "https://hooks.slack.com/services/<>/<>/<>"
|
||||
SLACK_DAILY_REPORT_FREQUENCY: "86400" # 24 hours; Optional: defaults to 12 hours
|
||||
```
|
||||
|
||||
|
||||
### Step 3: Start proxy
|
||||
|
||||
Start proxy
|
||||
```bash
|
||||
$ litellm --config /path/to/config.yaml
|
||||
```
|
||||
|
||||
## Testing Alerting is Setup Correctly
|
||||
|
||||
Make a GET request to `/health/services`, expect to see a test slack alert in your provided webhook slack channel
|
||||
### Step 3: Test it!
|
||||
|
||||
```shell
|
||||
curl -X GET 'http://localhost:4000/health/services?service=slack' \
|
||||
-H 'Authorization: Bearer sk-1234'
|
||||
|
||||
```bash
|
||||
curl -X GET 'http://0.0.0.0:4000/health/services?service=slack' \
|
||||
-H 'Authorization: Bearer sk-1234'
|
||||
```
|
||||
|
||||
## Advanced - Redacting Messages from Alerts
|
||||
|
@ -84,7 +76,34 @@ litellm_settings:
|
|||
```
|
||||
|
||||
|
||||
## Advanced - Add Metadata to alerts
|
||||
|
||||
Add alerting metadata to proxy calls for debugging.
|
||||
|
||||
```python
|
||||
import openai
|
||||
client = openai.OpenAI(
|
||||
api_key="anything",
|
||||
base_url="http://0.0.0.0:4000"
|
||||
)
|
||||
|
||||
# request sent to model set on litellm proxy, `litellm --model`
|
||||
response = client.chat.completions.create(
|
||||
model="gpt-3.5-turbo",
|
||||
messages = [],
|
||||
extra_body={
|
||||
"metadata": {
|
||||
"alerting_metadata": {
|
||||
"hello": "world"
|
||||
}
|
||||
}
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
**Expected Response**
|
||||
|
||||
<Image img={require('../../img/alerting_metadata.png')}/>
|
||||
|
||||
## Advanced - Opting into specific alert types
|
||||
|
||||
|
|
BIN
docs/my-website/img/alerting_metadata.png
Normal file
BIN
docs/my-website/img/alerting_metadata.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 207 KiB |
Loading…
Add table
Add a link
Reference in a new issue