phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-27 11:43:54 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

(Feat) DataDog Logger - Add HOSTNAME and POD_NAME to DataDog logs (#7189)

Browse source 
* add unit test for test_datadog_static_methods

* docs dd vars

* test_datadog_payload_environment_variables

* test_datadog_static_methods

* docs env vars

* fix table
This commit is contained in:
Ishaan Jaff 2024-12-12 12:06:26 -08:00 • committed by GitHub
parent e14ce4a0d3
commit 02fc8d8738
5 changed files with 161 additions and 81 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/my-website/docs/proxy/config_settings.md
View file

@ -382,6 +382,7 @@ router_settings:
| GOOGLE_KMS_RESOURCE_NAME | Name of the resource in Google KMS
| HF_API_BASE | Base URL for Hugging Face API
| HELICONE_API_KEY | API key for Helicone service
| HOSTNAME | Hostname for the server, this will be [emitted to `datadog` logs](https://docs.litellm.ai/docs/proxy/logging#datadog)
| HUGGINGFACE_API_BASE | Base URL for Hugging Face API
| IAM_TOKEN_DB_AUTH | IAM token for database authentication
| JSON_LOGS | Enable JSON formatted logging
@ -442,6 +443,7 @@ router_settings:
| OTEL_HEADERS | Headers for OpenTelemetry requests
| OTEL_SERVICE_NAME | Service name identifier for OpenTelemetry
| OTEL_TRACER_NAME | Tracer name for OpenTelemetry tracing
| POD_NAME | Pod name for the server, this will be [emitted to `datadog` logs](https://docs.litellm.ai/docs/proxy/logging#datadog) as `POD_NAME`
| PREDIBASE_API_BASE | Base URL for Predibase API
| PRESIDIO_ANALYZER_API_BASE | Base URL for Presidio Analyzer service
| PRESIDIO_ANONYMIZER_API_BASE | Base URL for Presidio Anonymizer service

168
docs/my-website/docs/proxy/logging.md
View file

@ -997,6 +997,97 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
Your logs should be available on the specified s3 Bucket
## DataDog
LiteLLM Supports logging to the following Datdog Integrations:
- `datadog` [Datadog Logs](https://docs.datadoghq.com/logs/)
- `datadog_llm_observability` [Datadog LLM Observability](https://www.datadoghq.com/product/llm-observability/)
<Tabs>
<TabItem value="datadog" label="Datadog Logs">
We will use the `--config` to set `litellm.callbacks = ["datadog"]` this will log all successfull LLM calls to DataDog
**Step 1**: Create a `config.yaml` file and set `litellm_settings`: `success_callback`
```yaml
model_list:
- model_name: gpt-3.5-turbo
litellm_params:
model: gpt-3.5-turbo
litellm_settings:
callbacks: ["datadog"] # logs llm success + failure logs on datadog
service_callback: ["datadog"] # logs redis, postgres failures on datadog
```
</TabItem>
<TabItem value="datadog_llm_observability" label="Datadog LLM Observability">
```yaml
model_list:
- model_name: gpt-3.5-turbo
litellm_params:
model: gpt-3.5-turbo
litellm_settings:
callbacks: ["datadog_llm_observability"] # logs llm success logs on datadog
```
</TabItem>
</Tabs>
**Step 2**: Set Required env variables for datadog
```shell
DD_API_KEY="5f2d0f310***********" # your datadog API Key
DD_SITE="us5.datadoghq.com" # your datadog base url
DD_SOURCE="litellm_dev" # [OPTIONAL] your datadog source. use to differentiate dev vs. prod deployments
```
**Step 3**: Start the proxy, make a test request
Start proxy
```shell
litellm --config config.yaml --debug
```
Test Request
```shell
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
],
"metadata": {
"your-custom-metadata": "custom-field",
}
}'
```
Expected output on Datadog
<Image img={require('../../img/dd_small1.png')} />
### Set DD variables (`DD_SERVICE` etc)
LiteLLM supports customizing the following Datadog environment variables
| Environment Variable | Description | Default Value | Required |
|---------------------|-------------|---------------|----------|
| `DD_API_KEY` | Your Datadog API key for authentication | None | ✅ Yes |
| `DD_SITE` | Your Datadog site (e.g., "us5.datadoghq.com") | None | ✅ Yes |
| `DD_ENV` | Environment tag for your logs (e.g., "production", "staging") | "unknown" | ❌ No |
| `DD_SERVICE` | Service name for your logs | "litellm-server" | ❌ No |
| `DD_SOURCE` | Source name for your logs | "litellm" | ❌ No |
| `DD_VERSION` | Version tag for your logs | "unknown" | ❌ No |
| `HOSTNAME` | Hostname tag for your logs | "" | ❌ No |
| `POD_NAME` | Pod name tag (useful for Kubernetes deployments) | "unknown" | ❌ No |
## Custom Callback Class [Async]
@ -1638,83 +1729,6 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
<Image img={require('../../img/openmeter_img_2.png')} />
## DataDog
LiteLLM Supports logging to the following Datdog Integrations:
- `datadog` [Datadog Logs](https://docs.datadoghq.com/logs/)
- `datadog_llm_observability` [Datadog LLM Observability](https://www.datadoghq.com/product/llm-observability/)
<Tabs>
<TabItem value="datadog" label="Datadog Logs">
We will use the `--config` to set `litellm.success_callback = ["datadog"]` this will log all successfull LLM calls to DataDog
**Step 1**: Create a `config.yaml` file and set `litellm_settings`: `success_callback`
```yaml
model_list:
- model_name: gpt-3.5-turbo
litellm_params:
model: gpt-3.5-turbo
litellm_settings:
success_callback: ["datadog"] # logs llm success logs on datadog
service_callback: ["datadog"] # logs redis, postgres failures on datadog
```
</TabItem>
<TabItem value="datadog_llm_observability" label="Datadog LLM Observability">
```yaml
model_list:
- model_name: gpt-3.5-turbo
litellm_params:
model: gpt-3.5-turbo
litellm_settings:
callbacks: ["datadog_llm_observability"] # logs llm success logs on datadog
```
</TabItem>
</Tabs>
**Step 2**: Set Required env variables for datadog
```shell
DD_API_KEY="5f2d0f310***********" # your datadog API Key
DD_SITE="us5.datadoghq.com" # your datadog base url
DD_SOURCE="litellm_dev" # [OPTIONAL] your datadog source. use to differentiate dev vs. prod deployments
```
**Step 3**: Start the proxy, make a test request
Start proxy
```shell
litellm --config config.yaml --debug
```
Test Request
```shell
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
],
"metadata": {
"your-custom-metadata": "custom-field",
}
}'
```
Expected output on Datadog
<Image img={require('../../img/dd_small1.png')} />
## DynamoDB
We will use the `--config` to set

8
litellm/integrations/datadog/datadog.py
View file

@ -451,7 +451,7 @@ class DataDogLogger(CustomBatchLogger):
@staticmethod
def _get_datadog_tags():
return f"env:{os.getenv('DD_ENV', 'unknown')},service:{os.getenv('DD_SERVICE', 'litellm')},version:{os.getenv('DD_VERSION', 'unknown')}"
return f"env:{os.getenv('DD_ENV', 'unknown')},service:{os.getenv('DD_SERVICE', 'litellm')},version:{os.getenv('DD_VERSION', 'unknown')},HOSTNAME:{DataDogLogger._get_datadog_hostname()},POD_NAME:{os.getenv('POD_NAME', 'unknown')}"
@staticmethod
def _get_datadog_source():
@ -463,8 +463,12 @@ class DataDogLogger(CustomBatchLogger):
@staticmethod
def _get_datadog_hostname():
return ""
return os.getenv("HOSTNAME", "")
@staticmethod
def _get_datadog_env():
return os.getenv("DD_ENV", "unknown")
@staticmethod
def _get_datadog_pod_name():
return os.getenv("POD_NAME", "unknown")

5
litellm/proxy/proxy_config.yaml
View file

@ -2,5 +2,10 @@ model_list:
- model_name: gpt-4o
litellm_params:
model: openai/gpt-4o
api_base: https://exampleopenaiendpoint-production.up.railway.app/
- model_name: anthropic/*
litellm_params:
model: anthropic/fake
api_base: https://exampleanthropicendpoint-production.up.railway.app/
litellm_settings:
callbacks: ["datadog"]

59
tests/logging_callback_tests/test_datadog.py
View file

@ -382,9 +382,10 @@ async def test_datadog_payload_environment_variables():
assert (
dd_payload["service"] == "test-service"
), "Incorrect service in payload"
assert (
dd_payload["ddtags"]
== "env:test-env,service:test-service,version:1.0.0"
"env:test-env,service:test-service,version:1.0.0,HOSTNAME:"
in dd_payload["ddtags"]
), "Incorrect tags in payload"
except Exception as e:
@ -443,3 +444,57 @@ async def test_datadog_payload_content_truncation():
assert (
len(str(message_dict["response"])) < 10_100
), "response not truncated correctly"
def test_datadog_static_methods():
"""Test the static helper methods in DataDogLogger class"""
# Test with default environment variables
assert DataDogLogger._get_datadog_source() == "litellm"
assert DataDogLogger._get_datadog_service() == "litellm-server"
assert DataDogLogger._get_datadog_hostname() is not None
assert DataDogLogger._get_datadog_env() == "unknown"
assert DataDogLogger._get_datadog_pod_name() == "unknown"
# Test tags format with default values
assert (
"env:unknown,service:litellm,version:unknown,HOSTNAME:"
in DataDogLogger._get_datadog_tags()
)
# Test with custom environment variables
test_env = {
"DD_SOURCE": "custom-source",
"DD_SERVICE": "custom-service",
"HOSTNAME": "test-host",
"DD_ENV": "production",
"DD_VERSION": "1.0.0",
"POD_NAME": "pod-123",
}
with patch.dict(os.environ, test_env):
assert DataDogLogger._get_datadog_source() == "custom-source"
print(
"DataDogLogger._get_datadog_source()", DataDogLogger._get_datadog_source()
)
assert DataDogLogger._get_datadog_service() == "custom-service"
print(
"DataDogLogger._get_datadog_service()", DataDogLogger._get_datadog_service()
)
assert DataDogLogger._get_datadog_hostname() == "test-host"
print(
"DataDogLogger._get_datadog_hostname()",
DataDogLogger._get_datadog_hostname(),
)
assert DataDogLogger._get_datadog_env() == "production"
print("DataDogLogger._get_datadog_env()", DataDogLogger._get_datadog_env())
assert DataDogLogger._get_datadog_pod_name() == "pod-123"
print(
"DataDogLogger._get_datadog_pod_name()",
DataDogLogger._get_datadog_pod_name(),
)
# Test tags format with custom values
expected_custom_tags = "env:production,service:custom-service,version:1.0.0,HOSTNAME:test-host,POD_NAME:pod-123"
print("DataDogLogger._get_datadog_tags()", DataDogLogger._get_datadog_tags())
assert DataDogLogger._get_datadog_tags() == expected_custom_tags