mirror of
https://github.com/BerriAI/litellm.git
synced 2025-04-25 18:54:30 +00:00
docs(index.md): update changelog with realtime api cost tracking details
This commit is contained in:
parent
44368389f4
commit
65e18f6abe
4 changed files with 37 additions and 24 deletions
|
@ -205,28 +205,6 @@ curl -X POST \
|
||||||
{"message":"Spend for all API Keys and Teams reset successfully","status":"success"}
|
{"message":"Spend for all API Keys and Teams reset successfully","status":"success"}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
## Set 'base_model' for Cost Tracking (e.g. Azure deployments)
|
|
||||||
|
|
||||||
**Problem**: Azure returns `gpt-4` in the response when `azure/gpt-4-1106-preview` is used. This leads to inaccurate cost tracking
|
|
||||||
|
|
||||||
**Solution** ✅ : Set `base_model` on your config so litellm uses the correct model for calculating azure cost
|
|
||||||
|
|
||||||
Get the base model name from [here](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json)
|
|
||||||
|
|
||||||
Example config with `base_model`
|
|
||||||
```yaml
|
|
||||||
model_list:
|
|
||||||
- model_name: azure-gpt-3.5
|
|
||||||
litellm_params:
|
|
||||||
model: azure/chatgpt-v-2
|
|
||||||
api_base: os.environ/AZURE_API_BASE
|
|
||||||
api_key: os.environ/AZURE_API_KEY
|
|
||||||
api_version: "2023-07-01-preview"
|
|
||||||
model_info:
|
|
||||||
base_model: azure/gpt-4-1106-preview
|
|
||||||
```
|
|
||||||
|
|
||||||
## Daily Spend Breakdown API
|
## Daily Spend Breakdown API
|
||||||
|
|
||||||
Retrieve granular daily usage data for a user (by model, provider, and API key) with a single endpoint.
|
Retrieve granular daily usage data for a user (by model, provider, and API key) with a single endpoint.
|
||||||
|
|
|
@ -83,6 +83,28 @@ model_list:
|
||||||
cache_read_input_token_cost: 0.0000006
|
cache_read_input_token_cost: 0.0000006
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Set 'base_model' for Cost Tracking (e.g. Azure deployments)
|
||||||
|
|
||||||
|
**Problem**: Azure returns `gpt-4` in the response when `azure/gpt-4-1106-preview` is used. This leads to inaccurate cost tracking
|
||||||
|
|
||||||
|
**Solution** ✅ : Set `base_model` on your config so litellm uses the correct model for calculating azure cost
|
||||||
|
|
||||||
|
Get the base model name from [here](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json)
|
||||||
|
|
||||||
|
Example config with `base_model`
|
||||||
|
```yaml
|
||||||
|
model_list:
|
||||||
|
- model_name: azure-gpt-3.5
|
||||||
|
litellm_params:
|
||||||
|
model: azure/chatgpt-v-2
|
||||||
|
api_base: os.environ/AZURE_API_BASE
|
||||||
|
api_key: os.environ/AZURE_API_KEY
|
||||||
|
api_version: "2023-07-01-preview"
|
||||||
|
model_info:
|
||||||
|
base_model: azure/gpt-4-1106-preview
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
## Debugging
|
## Debugging
|
||||||
|
|
||||||
If you're custom pricing is not being used or you're seeing errors, please check the following:
|
If you're custom pricing is not being used or you're seeing errors, please check the following:
|
||||||
|
|
BIN
docs/my-website/img/realtime_api.png
Normal file
BIN
docs/my-website/img/realtime_api.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 182 KiB |
|
@ -12,7 +12,7 @@ authors:
|
||||||
url: https://www.linkedin.com/in/reffajnaahsi/
|
url: https://www.linkedin.com/in/reffajnaahsi/
|
||||||
image_url: https://pbs.twimg.com/profile_images/1613813310264340481/lz54oEiB_400x400.jpg
|
image_url: https://pbs.twimg.com/profile_images/1613813310264340481/lz54oEiB_400x400.jpg
|
||||||
|
|
||||||
tags: []
|
tags: ["sso", "unified_file_id", "cost_tracking", "security"]
|
||||||
hide_table_of_contents: false
|
hide_table_of_contents: false
|
||||||
---
|
---
|
||||||
|
|
||||||
|
@ -69,7 +69,20 @@ This release adds support for auto-syncing groups and members on Microsoft Entra
|
||||||
|
|
||||||
Get started with this [here](https://docs.litellm.ai/docs/tutorials/msft_sso)
|
Get started with this [here](https://docs.litellm.ai/docs/tutorials/msft_sso)
|
||||||
|
|
||||||
## Unified File ID
|
## Realtime API Cost Tracking
|
||||||
|
|
||||||
|
<Image
|
||||||
|
img={require('../../img/realtime_api.png')}
|
||||||
|
style={{width: '100%', display: 'block'}}
|
||||||
|
/>
|
||||||
|
|
||||||
|
|
||||||
|
This release adds Realtime API logging + cost tracking.
|
||||||
|
- **Logging**: LiteLLM now logs the complete response from realtime calls to all logging integrations (DB, S3, Langfuse, etc.)
|
||||||
|
- **Cost Tracking**: You can now set 'base_model' and custom pricing for realtime models. [Custom Pricing](../../docs/proxy/custom_pricing)
|
||||||
|
- **Budgets**: Your key/user/team budgets now work for realtime models as well.
|
||||||
|
|
||||||
|
Start [here](https://docs.litellm.ai/docs/realtime)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue