docs: refactor team based logging in docs

This commit is contained in:
Krrish Dholakia 2024-03-13 12:26:39 -07:00
parent b3493269b3
commit dbc7552d15
2 changed files with 40 additions and 34 deletions

View file

@ -678,34 +678,6 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
Your logs should be available on the specified s3 Bucket
## Team-based Logging
Set success callbacks (e.g. langfuse), for a specific team-id.
```yaml
litellm_settings:
default_team_settings:
- team_id: my-secret-project
success_callback: ["langfuse"]
langfuse_public_key: os.environ/LANGFUSE_PUB_KEY_2
langfuse_secret: os.environ/LANGFUSE_PRIVATE_KEY_2
- team_id: ishaans-secret-project
success_callback: ["langfuse"]
langfuse_public_key: os.environ/LANGFUSE_PUB_KEY_3
langfuse_secret: os.environ/LANGFUSE_SECRET_3
```
Now, when you [generate keys](./virtual_keys.md) for this team-id
```bash
curl -X POST 'http://0.0.0.0:4000/key/generate' \
-H 'Authorization: Bearer sk-1234' \
-H 'Content-Type: application/json' \
-D '{"team_id": "ishaans-secret-project"}'
```
All requests made with these keys will log data to their team-specific logging.
## Logging Proxy Input/Output - DynamoDB
We will use the `--config` to set

View file

@ -1,8 +1,9 @@
# 👥 Team-based Routing
# 👥 Team-based Routing + Logging
## Routing
Route calls to different model groups based on the team-id
## Config with model group
### Config with model group
Create a config.yaml with 2 model groups + connected postgres db
@ -32,7 +33,7 @@ Start proxy
litellm --config /path/to/config.yaml
```
## Create Team with Model Alias
### Create Team with Model Alias
```bash
curl --location 'http://0.0.0.0:4000/team/new' \
@ -46,7 +47,7 @@ curl --location 'http://0.0.0.0:4000/team/new' \
# Returns team_id: my-team-id
```
## Create Team Key
### Create Team Key
```bash
curl --location 'http://localhost:4000/key/generate' \
@ -57,7 +58,7 @@ curl --location 'http://localhost:4000/key/generate' \
}'
```
## Call Model with alias
### Call Model with alias
```bash
curl --location 'http://0.0.0.0:4000/v1/chat/completions' \
@ -68,4 +69,37 @@ curl --location 'http://0.0.0.0:4000/v1/chat/completions' \
"messages": [{"role": "system", "content": "You'\''re an expert at writing poems"}, {"role": "user", "content": "Write me a poem"}, {"role": "user", "content": "What'\''s your name?"}],
"user": "usha"
}'
```
```
## Logging / Caching
Turn on/off logging and caching for a specific team id.
**Example:**
This config would send langfuse logs to 2 different langfuse projects, based on the team id
```yaml
litellm_settings:
default_team_settings:
- team_id: my-secret-project
success_callback: ["langfuse"]
langfuse_public_key: os.environ/LANGFUSE_PUB_KEY_1 # Project 1
langfuse_secret: os.environ/LANGFUSE_PRIVATE_KEY_1 # Project 1
- team_id: ishaans-secret-project
success_callback: ["langfuse"]
langfuse_public_key: os.environ/LANGFUSE_PUB_KEY_2 # Project 2
langfuse_secret: os.environ/LANGFUSE_SECRET_2 # Project 2
```
Now, when you [generate keys](./virtual_keys.md) for this team-id
```bash
curl -X POST 'http://0.0.0.0:4000/key/generate' \
-H 'Authorization: Bearer sk-1234' \
-H 'Content-Type: application/json' \
-D '{"team_id": "ishaans-secret-project"}'
```
All requests made with these keys will log data to their team-specific logging.