(docs) litellm + datadog

This commit is contained in:
Ishaan Jaff 2024-03-18 17:06:00 -07:00
parent 0dab5841cd
commit 038c9d5781
2 changed files with 28 additions and 61 deletions

View file

@ -3,13 +3,13 @@ import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem'; import TabItem from '@theme/TabItem';
# 🔎 Logging - Custom Callbacks, Langfuse, ClickHouse, s3 Bucket, Sentry, OpenTelemetry, Athina # 🔎 Logging - Custom Callbacks, DataDog, Langfuse, s3 Bucket, Sentry, OpenTelemetry, Athina
Log Proxy Input, Output, Exceptions using Custom Callbacks, Langfuse, OpenTelemetry, LangFuse, DynamoDB, s3 Bucket Log Proxy Input, Output, Exceptions using Custom Callbacks, Langfuse, OpenTelemetry, LangFuse, DynamoDB, s3 Bucket
- [Async Custom Callbacks](#custom-callback-class-async) - [Async Custom Callbacks](#custom-callback-class-async)
- [Async Custom Callback APIs](#custom-callback-apis-async) - [Async Custom Callback APIs](#custom-callback-apis-async)
- [Logging to ClickHouse](#logging-proxy-inputoutput---clickhouse) - [Logging to DataDog](#logging-proxy-inputoutput---datadog)
- [Logging to Langfuse](#logging-proxy-inputoutput---langfuse) - [Logging to Langfuse](#logging-proxy-inputoutput---langfuse)
- [Logging to s3 Buckets](#logging-proxy-inputoutput---s3-buckets) - [Logging to s3 Buckets](#logging-proxy-inputoutput---s3-buckets)
- [Logging to DynamoDB](#logging-proxy-inputoutput---dynamodb) - [Logging to DynamoDB](#logging-proxy-inputoutput---dynamodb)
@ -539,32 +539,8 @@ print(response)
</Tabs> </Tabs>
## Logging Proxy Input/Output - Clickhouse ## Logging Proxy Input/Output - DataDog
We will use the `--config` to set `litellm.success_callback = ["clickhouse"]` this will log all successfull LLM calls to ClickHouse DB We will use the `--config` to set `litellm.success_callback = ["datadog"]` this will log all successfull LLM calls to DataDog
### [Optional] - Docker Compose - LiteLLM Proxy + Self Hosted Clickhouse DB
Use this docker compose yaml to start LiteLLM Proxy + Clickhouse DB
```yaml
version: "3.9"
services:
litellm:
image: ghcr.io/berriai/litellm:main-latest
volumes:
- ./proxy_server_config.yaml:/app/proxy_server_config.yaml # mount your litellm config.yaml
ports:
- "4000:4000"
environment:
- AZURE_API_KEY=sk-123
clickhouse:
image: clickhouse/clickhouse-server
environment:
- CLICKHOUSE_DB=litellm-test
- CLICKHOUSE_USER=admin
- CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1
- CLICKHOUSE_PASSWORD=admin
ports:
- "8123:8123"
```
**Step 1**: Create a `config.yaml` file and set `litellm_settings`: `success_callback` **Step 1**: Create a `config.yaml` file and set `litellm_settings`: `success_callback`
```yaml ```yaml
@ -573,43 +549,16 @@ model_list:
litellm_params: litellm_params:
model: gpt-3.5-turbo model: gpt-3.5-turbo
litellm_settings: litellm_settings:
success_callback: ["clickhouse"] success_callback: ["datadog"]
``` ```
**Step 2**: Set Required env variables for clickhouse **Step 2**: Set Required env variables for datadog
<Tabs>
<TabItem value="self" label="Self Hosted Clickhouse">
Env Variables for self hosted click house
```shell
CLICKHOUSE_HOST = "localhost"
CLICKHOUSE_PORT = "8123"
CLICKHOUSE_USERNAME = "admin"
CLICKHOUSE_PASSWORD = "admin"
```
</TabItem>
<TabItem value="cloud" label="Clickhouse.cloud">
Env Variables for cloud click house
```shell ```shell
CLICKHOUSE_HOST = "hjs1z7j37j.us-east1.gcp.clickhouse.cloud" DD_API_KEY="5f2d0f310***********" # your datadog API Key
CLICKHOUSE_PORT = "8443" DD_SITE="us5.datadoghq.com" # your datadog base url
CLICKHOUSE_USERNAME = "default"
CLICKHOUSE_PASSWORD = "M~PimRs~c3Z6b"
``` ```
</TabItem>
</Tabs>
**Step 3**: Start the proxy, make a test request **Step 3**: Start the proxy, make a test request
Start proxy Start proxy
@ -618,9 +567,27 @@ litellm --config config.yaml --debug
``` ```
Test Request Test Request
```shell
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
],
"metadata": {
"your-custom-metadata": "custom-field",
}
}'
``` ```
litellm --test
``` Expected output on Datadog
<Image img={require('../../img/dd_small1.png')} />
## Logging Proxy Input/Output - s3 Buckets ## Logging Proxy Input/Output - s3 Buckets

Binary file not shown.

After

Width:  |  Height:  |  Size: 230 KiB