forked from phoenix/litellm-mirror
add clickhouse docs
This commit is contained in:
parent
18c27303aa
commit
9bb55a0671
3 changed files with 95 additions and 4 deletions
|
@ -8,8 +8,13 @@ services:
|
|||
- "4000:4000"
|
||||
environment:
|
||||
- AZURE_API_KEY=sk-123
|
||||
|
||||
clickhouse:
|
||||
image: clickhouse/clickhouse-server
|
||||
|
||||
|
||||
|
||||
environment:
|
||||
- CLICKHOUSE_DB=litellm-test
|
||||
- CLICKHOUSE_USER=admin
|
||||
- CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1
|
||||
- CLICKHOUSE_PASSWORD=admin
|
||||
ports:
|
||||
- "8123:8123"
|
||||
|
|
|
@ -3,12 +3,13 @@ import Tabs from '@theme/Tabs';
|
|||
import TabItem from '@theme/TabItem';
|
||||
|
||||
|
||||
# 🔎 Logging - Custom Callbacks, Langfuse, s3 Bucket, Sentry, OpenTelemetry, Athina
|
||||
# 🔎 Logging - Custom Callbacks, Langfuse, ClickHouse, s3 Bucket, Sentry, OpenTelemetry, Athina
|
||||
|
||||
Log Proxy Input, Output, Exceptions using Custom Callbacks, Langfuse, OpenTelemetry, LangFuse, DynamoDB, s3 Bucket
|
||||
|
||||
- [Async Custom Callbacks](#custom-callback-class-async)
|
||||
- [Async Custom Callback APIs](#custom-callback-apis-async)
|
||||
- [Logging to ClickHouse](#logging-proxy-inputoutput---clickHouse)
|
||||
- [Logging to Langfuse](#logging-proxy-inputoutput---langfuse)
|
||||
- [Logging to s3 Buckets](#logging-proxy-inputoutput---s3-buckets)
|
||||
- [Logging to DynamoDB](#logging-proxy-inputoutput---dynamodb)
|
||||
|
@ -538,6 +539,90 @@ print(response)
|
|||
</Tabs>
|
||||
|
||||
|
||||
## Logging Proxy Input/Output - Clickhouse
|
||||
We will use the `--config` to set `litellm.success_callback = ["clickhouse"]` this will log all successfull LLM calls to ClickHouse DB
|
||||
|
||||
### [Optional] - Docker Compose - LiteLLM Proxy + Self Hosted Clickhouse DB
|
||||
Use this docker compose yaml to start LiteLLM Proxy + Clickhouse DB
|
||||
```yaml
|
||||
version: "3.9"
|
||||
services:
|
||||
litellm:
|
||||
image: ghcr.io/berriai/litellm:main-latest
|
||||
volumes:
|
||||
- ./proxy_server_config.yaml:/app/proxy_server_config.yaml # mount your litellm config.yaml
|
||||
ports:
|
||||
- "4000:4000"
|
||||
environment:
|
||||
- AZURE_API_KEY=sk-123
|
||||
clickhouse:
|
||||
image: clickhouse/clickhouse-server
|
||||
environment:
|
||||
- CLICKHOUSE_DB=litellm-test
|
||||
- CLICKHOUSE_USER=admin
|
||||
- CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1
|
||||
- CLICKHOUSE_PASSWORD=admin
|
||||
ports:
|
||||
- "8123:8123"
|
||||
```
|
||||
|
||||
**Step 1**: Create a `config.yaml` file and set `litellm_settings`: `success_callback`
|
||||
```yaml
|
||||
model_list:
|
||||
- model_name: gpt-3.5-turbo
|
||||
litellm_params:
|
||||
model: gpt-3.5-turbo
|
||||
litellm_settings:
|
||||
success_callback: ["clickhouse"]
|
||||
```
|
||||
|
||||
**Step 2**: Set Required env variables for clickhouse
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="self" label="Self Hosted Clickhouse">
|
||||
|
||||
Env Variables for self hosted click house
|
||||
```shell
|
||||
CLICKHOUSE_HOST = "localhost"
|
||||
CLICKHOUSE_PORT = "8123"
|
||||
CLICKHOUSE_USERNAME = "admin"
|
||||
CLICKHOUSE_PASSWORD = "admin"
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
|
||||
|
||||
|
||||
<TabItem value="cloud" label="Clickhouse.cloud">
|
||||
|
||||
Env Variables for cloud click house
|
||||
|
||||
```shell
|
||||
CLICKHOUSE_HOST = "hjs1z7j37j.us-east1.gcp.clickhouse.cloud"
|
||||
CLICKHOUSE_PORT = "8443"
|
||||
CLICKHOUSE_USERNAME = "default"
|
||||
CLICKHOUSE_PASSWORD = "M~PimRs~c3Z6b"
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
|
||||
|
||||
|
||||
**Step 3**: Start the proxy, make a test request
|
||||
|
||||
Start proxy
|
||||
```shell
|
||||
litellm --config config.yaml --debug
|
||||
```
|
||||
|
||||
Test Request
|
||||
```
|
||||
litellm --test
|
||||
```
|
||||
|
||||
|
||||
## Logging Proxy Input/Output - s3 Buckets
|
||||
|
||||
We will use the `--config` to set
|
||||
|
|
|
@ -16,6 +16,7 @@ google-generativeai==0.3.2 # for vertex ai calls
|
|||
async_generator==1.10.0 # for async ollama calls
|
||||
traceloop-sdk==0.5.3 # for open telemetry logging
|
||||
langfuse>=2.6.3 # for langfuse self-hosted logging
|
||||
clickhouse_connect==0.7.0
|
||||
orjson==3.9.7 # fast /embedding responses
|
||||
apscheduler==3.10.4 # for resetting budget in background
|
||||
fastapi-sso==0.10.0 # admin UI, SSO
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue