mirror of
https://github.com/BerriAI/litellm.git
synced 2025-04-27 11:43:54 +00:00
513874858b
* fix(model_hub.tsx): clarify cost in model hub is per 1m tokens * docs: test blog * docs: improve release note docs * docs(docs/): new stable release doc * docs(docs/): specify date in all posts * docs(docs/): add git diff to stable release docs
117 lines
No EOL
3 KiB
Markdown
117 lines
No EOL
3 KiB
Markdown
---
|
|
title: v1.56.3
|
|
slug: v1.56.3
|
|
date: 2024-12-28T10:00:00
|
|
authors:
|
|
- name: Krrish Dholakia
|
|
title: CEO, LiteLLM
|
|
url: https://www.linkedin.com/in/krish-d/
|
|
image_url: https://media.licdn.com/dms/image/v2/C5103AQHYMXJfHTf4Ng/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1517455593871?e=1741824000&v=beta&t=udmat6jS-s3EQZp1DTykf7NZmf-3sefD_I9B1aMjE5Y
|
|
- name: Ishaan Jaffer
|
|
title: CTO, LiteLLM
|
|
url: https://www.linkedin.com/in/reffajnaahsi/
|
|
image_url: https://media.licdn.com/dms/image/v2/D4D03AQGiM7ZrUwqu_Q/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1675971026692?e=1741824000&v=beta&t=eQnRdXPJo4eiINWTZARoYTfqh064pgZ-E21pQTSy8jc
|
|
tags: [guardrails, logging, virtual key management, new models]
|
|
hide_table_of_contents: false
|
|
---
|
|
|
|
import Image from '@theme/IdealImage';
|
|
|
|
`guardrails`, `logging`, `virtual key management`, `new models`
|
|
|
|
:::info
|
|
|
|
Get a 7 day free trial for LiteLLM Enterprise [here](https://litellm.ai/#trial).
|
|
|
|
**no call needed**
|
|
|
|
:::
|
|
|
|
## New Features
|
|
|
|
### ✨ Log Guardrail Traces
|
|
|
|
Track guardrail failure rate and if a guardrail is going rogue and failing requests. [Start here](https://docs.litellm.ai/docs/proxy/guardrails/quick_start)
|
|
|
|
|
|
#### Traced Guardrail Success
|
|
|
|
<Image img={require('../../img/gd_success.png')} />
|
|
|
|
#### Traced Guardrail Failure
|
|
|
|
<Image img={require('../../img/gd_fail.png')} />
|
|
|
|
|
|
### `/guardrails/list`
|
|
|
|
`/guardrails/list` allows clients to view available guardrails + supported guardrail params
|
|
|
|
|
|
```shell
|
|
curl -X GET 'http://0.0.0.0:4000/guardrails/list'
|
|
```
|
|
|
|
Expected response
|
|
|
|
```json
|
|
{
|
|
"guardrails": [
|
|
{
|
|
"guardrail_name": "aporia-post-guard",
|
|
"guardrail_info": {
|
|
"params": [
|
|
{
|
|
"name": "toxicity_score",
|
|
"type": "float",
|
|
"description": "Score between 0-1 indicating content toxicity level"
|
|
},
|
|
{
|
|
"name": "pii_detection",
|
|
"type": "boolean"
|
|
}
|
|
]
|
|
}
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
|
|
### ✨ Guardrails with Mock LLM
|
|
|
|
|
|
Send `mock_response` to test guardrails without making an LLM call. More info on `mock_response` [here](https://docs.litellm.ai/docs/proxy/guardrails/quick_start)
|
|
|
|
```shell
|
|
curl -i http://localhost:4000/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
|
|
-d '{
|
|
"model": "gpt-3.5-turbo",
|
|
"messages": [
|
|
{"role": "user", "content": "hi my email is ishaan@berri.ai"}
|
|
],
|
|
"mock_response": "This is a mock response",
|
|
"guardrails": ["aporia-pre-guard", "aporia-post-guard"]
|
|
}'
|
|
```
|
|
|
|
|
|
|
|
### Assign Keys to Users
|
|
|
|
You can now assign keys to users via Proxy UI
|
|
|
|
|
|
<Image img={require('../../img/ui_key.png')} />
|
|
|
|
## New Models
|
|
|
|
- `openrouter/openai/o1`
|
|
- `vertex_ai/mistral-large@2411`
|
|
|
|
## Fixes
|
|
|
|
- Fix `vertex_ai/` mistral model pricing: https://github.com/BerriAI/litellm/pull/7345
|
|
- Missing model_group field in logs for aspeech call types https://github.com/BerriAI/litellm/pull/7392 |