diff --git a/docs/my-website/docs/proxy/cost_tracking.md b/docs/my-website/docs/proxy/cost_tracking.md index 784de1da81..5b17e565a5 100644 --- a/docs/my-website/docs/proxy/cost_tracking.md +++ b/docs/my-website/docs/proxy/cost_tracking.md @@ -205,28 +205,6 @@ curl -X POST \ {"message":"Spend for all API Keys and Teams reset successfully","status":"success"} ``` - -## Set 'base_model' for Cost Tracking (e.g. Azure deployments) - -**Problem**: Azure returns `gpt-4` in the response when `azure/gpt-4-1106-preview` is used. This leads to inaccurate cost tracking - -**Solution** ✅ : Set `base_model` on your config so litellm uses the correct model for calculating azure cost - -Get the base model name from [here](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json) - -Example config with `base_model` -```yaml -model_list: - - model_name: azure-gpt-3.5 - litellm_params: - model: azure/chatgpt-v-2 - api_base: os.environ/AZURE_API_BASE - api_key: os.environ/AZURE_API_KEY - api_version: "2023-07-01-preview" - model_info: - base_model: azure/gpt-4-1106-preview -``` - ## Daily Spend Breakdown API Retrieve granular daily usage data for a user (by model, provider, and API key) with a single endpoint. diff --git a/docs/my-website/docs/proxy/custom_pricing.md b/docs/my-website/docs/proxy/custom_pricing.md index 26d7ffb1f5..792d5c26dd 100644 --- a/docs/my-website/docs/proxy/custom_pricing.md +++ b/docs/my-website/docs/proxy/custom_pricing.md @@ -83,6 +83,28 @@ model_list: cache_read_input_token_cost: 0.0000006 ``` +## Set 'base_model' for Cost Tracking (e.g. Azure deployments) + +**Problem**: Azure returns `gpt-4` in the response when `azure/gpt-4-1106-preview` is used. This leads to inaccurate cost tracking + +**Solution** ✅ : Set `base_model` on your config so litellm uses the correct model for calculating azure cost + +Get the base model name from [here](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json) + +Example config with `base_model` +```yaml +model_list: + - model_name: azure-gpt-3.5 + litellm_params: + model: azure/chatgpt-v-2 + api_base: os.environ/AZURE_API_BASE + api_key: os.environ/AZURE_API_KEY + api_version: "2023-07-01-preview" + model_info: + base_model: azure/gpt-4-1106-preview +``` + + ## Debugging If you're custom pricing is not being used or you're seeing errors, please check the following: diff --git a/docs/my-website/img/realtime_api.png b/docs/my-website/img/realtime_api.png new file mode 100644 index 0000000000..798525278c Binary files /dev/null and b/docs/my-website/img/realtime_api.png differ diff --git a/docs/my-website/release_notes/v1.66.0-stable/index.md b/docs/my-website/release_notes/v1.66.0-stable/index.md index f54dfb8190..4acea0d547 100644 --- a/docs/my-website/release_notes/v1.66.0-stable/index.md +++ b/docs/my-website/release_notes/v1.66.0-stable/index.md @@ -12,7 +12,7 @@ authors: url: https://www.linkedin.com/in/reffajnaahsi/ image_url: https://pbs.twimg.com/profile_images/1613813310264340481/lz54oEiB_400x400.jpg -tags: [] +tags: ["sso", "unified_file_id", "cost_tracking", "security"] hide_table_of_contents: false --- @@ -69,7 +69,20 @@ This release adds support for auto-syncing groups and members on Microsoft Entra Get started with this [here](https://docs.litellm.ai/docs/tutorials/msft_sso) -## Unified File ID +## Realtime API Cost Tracking + + + + +This release adds Realtime API logging + cost tracking. +- **Logging**: LiteLLM now logs the complete response from realtime calls to all logging integrations (DB, S3, Langfuse, etc.) +- **Cost Tracking**: You can now set 'base_model' and custom pricing for realtime models. [Custom Pricing](../../docs/proxy/custom_pricing) +- **Budgets**: Your key/user/team budgets now work for realtime models as well. + +Start [here](https://docs.litellm.ai/docs/realtime)