phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-24 18:24:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs(index.md): update changelog with realtime api cost tracking details

Browse source
This commit is contained in:
Krrish Dholakia 2025-04-12 19:15:40 -07:00
parent 44368389f4
commit 65e18f6abe
4 changed files with 37 additions and 24 deletions
Download patch file Download diff file Expand all files Collapse all files

22
docs/my-website/docs/proxy/cost_tracking.md
View file

@ -205,28 +205,6 @@ curl -X POST \
{"message":"Spend for all API Keys and Teams reset successfully","status":"success"}
```
## Set 'base_model' for Cost Tracking (e.g. Azure deployments)
**Problem**: Azure returns `gpt-4` in the response when `azure/gpt-4-1106-preview` is used. This leads to inaccurate cost tracking
**Solution** ✅ : Set `base_model` on your config so litellm uses the correct model for calculating azure cost
Get the base model name from [here](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json)
Example config with `base_model`
```yaml
model_list:
- model_name: azure-gpt-3.5
litellm_params:
model: azure/chatgpt-v-2
api_base: os.environ/AZURE_API_BASE
api_key: os.environ/AZURE_API_KEY
api_version: "2023-07-01-preview"
model_info:
base_model: azure/gpt-4-1106-preview
```
## Daily Spend Breakdown API
Retrieve granular daily usage data for a user (by model, provider, and API key) with a single endpoint.

22
docs/my-website/docs/proxy/custom_pricing.md
View file

@ -83,6 +83,28 @@ model_list:
cache_read_input_token_cost: 0.0000006
```
## Set 'base_model' for Cost Tracking (e.g. Azure deployments)
**Problem**: Azure returns `gpt-4` in the response when `azure/gpt-4-1106-preview` is used. This leads to inaccurate cost tracking
**Solution** ✅ : Set `base_model` on your config so litellm uses the correct model for calculating azure cost
Get the base model name from [here](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json)
Example config with `base_model`
```yaml
model_list:
- model_name: azure-gpt-3.5
litellm_params:
model: azure/chatgpt-v-2
api_base: os.environ/AZURE_API_BASE
api_key: os.environ/AZURE_API_KEY
api_version: "2023-07-01-preview"
model_info:
base_model: azure/gpt-4-1106-preview
```
## Debugging
If you're custom pricing is not being used or you're seeing errors, please check the following:

BIN
docs/my-website/img/realtime_api.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 182 KiB

17
docs/my-website/release_notes/v1.66.0-stable/index.md
View file

@ -12,7 +12,7 @@ authors:
url: https://www.linkedin.com/in/reffajnaahsi/
image_url: https://pbs.twimg.com/profile_images/1613813310264340481/lz54oEiB_400x400.jpg
tags: []
tags: ["sso", "unified_file_id", "cost_tracking", "security"]
hide_table_of_contents: false
---
@ -69,7 +69,20 @@ This release adds support for auto-syncing groups and members on Microsoft Entra
Get started with this [here](https://docs.litellm.ai/docs/tutorials/msft_sso)
## Unified File ID
## Realtime API Cost Tracking
<Image
img={require('../../img/realtime_api.png')}
style={{width: '100%', display: 'block'}}
/>
This release adds Realtime API logging + cost tracking.
- **Logging**: LiteLLM now logs the complete response from realtime calls to all logging integrations (DB, S3, Langfuse, etc.)
- **Cost Tracking**: You can now set 'base_model' and custom pricing for realtime models. [Custom Pricing](../../docs/proxy/custom_pricing)
- **Budgets**: Your key/user/team budgets now work for realtime models as well.
Start [here](https://docs.litellm.ai/docs/realtime)