(docs) litellm + datadog

This commit is contained in:
Ishaan Jaff 2024-03-18 17:06:00 -07:00
parent 0dab5841cd
commit 038c9d5781
2 changed files with 28 additions and 61 deletions

View file

@ -3,13 +3,13 @@ import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# 🔎 Logging - Custom Callbacks, Langfuse, ClickHouse, s3 Bucket, Sentry, OpenTelemetry, Athina
# 🔎 Logging - Custom Callbacks, DataDog, Langfuse, s3 Bucket, Sentry, OpenTelemetry, Athina
Log Proxy Input, Output, Exceptions using Custom Callbacks, Langfuse, OpenTelemetry, LangFuse, DynamoDB, s3 Bucket
- [Async Custom Callbacks](#custom-callback-class-async)
- [Async Custom Callback APIs](#custom-callback-apis-async)
- [Logging to ClickHouse](#logging-proxy-inputoutput---clickhouse)
- [Logging to DataDog](#logging-proxy-inputoutput---datadog)
- [Logging to Langfuse](#logging-proxy-inputoutput---langfuse)
- [Logging to s3 Buckets](#logging-proxy-inputoutput---s3-buckets)
- [Logging to DynamoDB](#logging-proxy-inputoutput---dynamodb)
@ -539,32 +539,8 @@ print(response)
</Tabs>
## Logging Proxy Input/Output - Clickhouse
We will use the `--config` to set `litellm.success_callback = ["clickhouse"]` this will log all successfull LLM calls to ClickHouse DB
### [Optional] - Docker Compose - LiteLLM Proxy + Self Hosted Clickhouse DB
Use this docker compose yaml to start LiteLLM Proxy + Clickhouse DB
```yaml
version: "3.9"
services:
litellm:
image: ghcr.io/berriai/litellm:main-latest
volumes:
- ./proxy_server_config.yaml:/app/proxy_server_config.yaml # mount your litellm config.yaml
ports:
- "4000:4000"
environment:
- AZURE_API_KEY=sk-123
clickhouse:
image: clickhouse/clickhouse-server
environment:
- CLICKHOUSE_DB=litellm-test
- CLICKHOUSE_USER=admin
- CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1
- CLICKHOUSE_PASSWORD=admin
ports:
- "8123:8123"
```
## Logging Proxy Input/Output - DataDog
We will use the `--config` to set `litellm.success_callback = ["datadog"]` this will log all successfull LLM calls to DataDog
**Step 1**: Create a `config.yaml` file and set `litellm_settings`: `success_callback`
```yaml
@ -573,43 +549,16 @@ model_list:
litellm_params:
model: gpt-3.5-turbo
litellm_settings:
success_callback: ["clickhouse"]
success_callback: ["datadog"]
```
**Step 2**: Set Required env variables for clickhouse
<Tabs>
<TabItem value="self" label="Self Hosted Clickhouse">
Env Variables for self hosted click house
```shell
CLICKHOUSE_HOST = "localhost"
CLICKHOUSE_PORT = "8123"
CLICKHOUSE_USERNAME = "admin"
CLICKHOUSE_PASSWORD = "admin"
```
</TabItem>
<TabItem value="cloud" label="Clickhouse.cloud">
Env Variables for cloud click house
**Step 2**: Set Required env variables for datadog
```shell
CLICKHOUSE_HOST = "hjs1z7j37j.us-east1.gcp.clickhouse.cloud"
CLICKHOUSE_PORT = "8443"
CLICKHOUSE_USERNAME = "default"
CLICKHOUSE_PASSWORD = "M~PimRs~c3Z6b"
DD_API_KEY="5f2d0f310***********" # your datadog API Key
DD_SITE="us5.datadoghq.com" # your datadog base url
```
</TabItem>
</Tabs>
**Step 3**: Start the proxy, make a test request
Start proxy
@ -618,9 +567,27 @@ litellm --config config.yaml --debug
```
Test Request
```shell
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
],
"metadata": {
"your-custom-metadata": "custom-field",
}
}'
```
litellm --test
```
Expected output on Datadog
<Image img={require('../../img/dd_small1.png')} />
## Logging Proxy Input/Output - s3 Buckets

Binary file not shown.

After

Width:  |  Height:  |  Size: 230 KiB