forked from phoenix/litellm-mirror
docs(alerting.md): add alerting metadata to docs
This commit is contained in:
parent
9e8eacbc77
commit
4eca63ede6
2 changed files with 40 additions and 21 deletions
|
@ -32,41 +32,33 @@ Get a slack webhook url from https://api.slack.com/messaging/webhooks
|
||||||
|
|
||||||
You can also use Discord Webhooks, see [here](#using-discord-webhooks)
|
You can also use Discord Webhooks, see [here](#using-discord-webhooks)
|
||||||
|
|
||||||
### Step 2: Update config.yaml
|
|
||||||
|
|
||||||
- Set `SLACK_WEBHOOK_URL` in your proxy env to enable Slack alerts.
|
Set `SLACK_WEBHOOK_URL` in your proxy env to enable Slack alerts.
|
||||||
- Just for testing purposes, let's save a bad key to our proxy.
|
|
||||||
|
```bash
|
||||||
|
export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/<>/<>/<>"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 2: Setup Proxy
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
model_list:
|
|
||||||
model_name: "azure-model"
|
|
||||||
litellm_params:
|
|
||||||
model: "azure/gpt-35-turbo"
|
|
||||||
api_key: "my-bad-key" # 👈 bad key
|
|
||||||
|
|
||||||
general_settings:
|
general_settings:
|
||||||
alerting: ["slack"]
|
alerting: ["slack"]
|
||||||
alerting_threshold: 300 # sends alerts if requests hang for 5min+ and responses take 5min+
|
alerting_threshold: 300 # sends alerts if requests hang for 5min+ and responses take 5min+
|
||||||
|
|
||||||
environment_variables:
|
|
||||||
SLACK_WEBHOOK_URL: "https://hooks.slack.com/services/<>/<>/<>"
|
|
||||||
SLACK_DAILY_REPORT_FREQUENCY: "86400" # 24 hours; Optional: defaults to 12 hours
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Start proxy
|
||||||
### Step 3: Start proxy
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
$ litellm --config /path/to/config.yaml
|
$ litellm --config /path/to/config.yaml
|
||||||
```
|
```
|
||||||
|
|
||||||
## Testing Alerting is Setup Correctly
|
|
||||||
|
|
||||||
Make a GET request to `/health/services`, expect to see a test slack alert in your provided webhook slack channel
|
### Step 3: Test it!
|
||||||
|
|
||||||
```shell
|
|
||||||
curl -X GET 'http://localhost:4000/health/services?service=slack' \
|
```bash
|
||||||
-H 'Authorization: Bearer sk-1234'
|
curl -X GET 'http://0.0.0.0:4000/health/services?service=slack' \
|
||||||
|
-H 'Authorization: Bearer sk-1234'
|
||||||
```
|
```
|
||||||
|
|
||||||
## Advanced - Redacting Messages from Alerts
|
## Advanced - Redacting Messages from Alerts
|
||||||
|
@ -84,7 +76,34 @@ litellm_settings:
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## Advanced - Add Metadata to alerts
|
||||||
|
|
||||||
|
Add alerting metadata to proxy calls for debugging.
|
||||||
|
|
||||||
|
```python
|
||||||
|
import openai
|
||||||
|
client = openai.OpenAI(
|
||||||
|
api_key="anything",
|
||||||
|
base_url="http://0.0.0.0:4000"
|
||||||
|
)
|
||||||
|
|
||||||
|
# request sent to model set on litellm proxy, `litellm --model`
|
||||||
|
response = client.chat.completions.create(
|
||||||
|
model="gpt-3.5-turbo",
|
||||||
|
messages = [],
|
||||||
|
extra_body={
|
||||||
|
"metadata": {
|
||||||
|
"alerting_metadata": {
|
||||||
|
"hello": "world"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Expected Response**
|
||||||
|
|
||||||
|
<Image img={require('../../img/alerting_metadata.png')}/>
|
||||||
|
|
||||||
## Advanced - Opting into specific alert types
|
## Advanced - Opting into specific alert types
|
||||||
|
|
||||||
|
|
BIN
docs/my-website/img/alerting_metadata.png
Normal file
BIN
docs/my-website/img/alerting_metadata.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 207 KiB |
Loading…
Add table
Add a link
Reference in a new issue