diff --git a/docs/my-website/docs/providers/azure.md b/docs/my-website/docs/providers/azure.md index f345c3db36..e58d8a7b5d 100644 --- a/docs/my-website/docs/providers/azure.md +++ b/docs/my-website/docs/providers/azure.md @@ -1011,8 +1011,7 @@ Expected Response: | Supported Operations | `/v1/responses`| | Azure OpenAI Responses API | [Azure OpenAI Responses API ↗](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/responses?tabs=python-secure) | | Cost Tracking, Logging Support | ✅ LiteLLM will log, track cost for Responses API Requests | - - +| Supported OpenAI Params | ✅ All OpenAI params are supported, [See here](https://github.com/BerriAI/litellm/blob/0717369ae6969882d149933da48eeb8ab0e691bd/litellm/llms/openai/responses/transformation.py#L23) | ## Usage diff --git a/docs/my-website/docs/response_api.md b/docs/my-website/docs/response_api.md index 47b409ebfa..85f18164fa 100644 --- a/docs/my-website/docs/response_api.md +++ b/docs/my-website/docs/response_api.md @@ -122,4 +122,4 @@ for event in response: | Provider | Link to Usage | |-------------|--------------------| | OpenAI| [Usage](#usage) | -| Azure OpenAI| [Usage](../docs/providers/azure#responses-api) | \ No newline at end of file +| Azure OpenAI| [Usage](../docs/providers/azure#azure-responses-api) | \ No newline at end of file diff --git a/docs/my-website/release_notes/v1.67.0-stable/index.md b/docs/my-website/release_notes/v1.67.0-stable/index.md index 12b2369506..183fff7e89 100644 --- a/docs/my-website/release_notes/v1.67.0-stable/index.md +++ b/docs/my-website/release_notes/v1.67.0-stable/index.md @@ -19,63 +19,63 @@ hide_table_of_contents: false ## New Models / Updated Models - **OpenAI** - 1. gpt-4.1 pricing - https://github.com/BerriAI/litellm/pull/9990 - 2. o4 - correctly map o4 to openai o_series model + 1. gpt-4.1 pricing - [Get Started](ADD DOCS HERE), [PR](https://github.com/BerriAI/litellm/pull/9990) + 2. o4 - correctly map o4 to openai o_series model [Get Started](ADD DOCS HERE) - **Azure AI** - 1. Phi-4 output cost per token fix - https://github.com/BerriAI/litellm/pull/9880 - 2. Responses API support https://github.com/BerriAI/litellm/pull/10116 + 1. Phi-4 output cost per token fix - [PR](https://github.com/BerriAI/litellm/pull/9880) + 2. Responses API support [Get Started](../../docs/providers/azure#azure-responses-api),[PR](https://github.com/BerriAI/litellm/pull/10116) - **Anthropic** - 1. redacted message thinking support - https://github.com/BerriAI/litellm/pull/10129 + 1. redacted message thinking support - [Get Started](../../docs/providers/anthropic#usage---thinking--reasoning_content),[PR](https://github.com/BerriAI/litellm/pull/10129) - **Cohere** - 1. `/v2/chat` Passthrough endpoint support w/ cost tracking - https://github.com/BerriAI/litellm/pull/9997 + 1. `/v2/chat` Passthrough endpoint support w/ cost tracking - [ADD DOCS HERE], [PR](https://github.com/BerriAI/litellm/pull/9997) - **Azure** - 1. Support azure tenant_id/client_id env vars - https://github.com/BerriAI/litellm/pull/9993 - 2. Fix response_format check for 2025+ api versions - https://github.com/BerriAI/litellm/pull/9993 + 1. Support azure tenant_id/client_id env vars - [Get Started](../../docs/providers/azure#entra-id---use-tenant_id-client_id-client_secret), [PR](https://github.com/BerriAI/litellm/pull/9993) + 2. Fix response_format check for 2025+ api versions - [PR](https://github.com/BerriAI/litellm/pull/9993) - **VLLM** - 1. Files - Support 'file' message type for VLLM video url's - https://github.com/BerriAI/litellm/pull/10129 - 2. Passthrough - new `/vllm/` passthrough endpoint support https://github.com/BerriAI/litellm/pull/10002 + 1. Files - Support 'file' message type for VLLM video url's - [ADD DOCS HERE], [PR](https://github.com/BerriAI/litellm/pull/10129) + 2. Passthrough - new `/vllm/` passthrough endpoint support [ADD DOCS HERE], [PR](https://github.com/BerriAI/litellm/pull/10002) - **Mistral** - 1. new `/mistral` passthrough endpoint support https://github.com/BerriAI/litellm/pull/10002 + 1. new `/mistral` passthrough endpoint support [ADD DOCS HERE], [PR](https://github.com/BerriAI/litellm/pull/10002) - **AWS** - 1. New mapped bedrock regions - https://github.com/BerriAI/litellm/pull/9430 + 1. New mapped bedrock regions - [PR](https://github.com/BerriAI/litellm/pull/9430) - **VertexAI / Google AI Studio** - 1. Gemini - Response format - Retain schema field ordering for google gemini and vertex by specifying propertyOrdering - https://github.com/BerriAI/litellm/pull/9828 - 2. Gemini-2.5-flash - return reasoning content - 3. Gemini-2.5-flash - pricing + model information - https://github.com/BerriAI/litellm/pull/10125 - 4. Passthrough - new `/vertex_ai/discovery` route - enables calling AgentBuilder API routes https://github.com/BerriAI/litellm/pull/10084 + 1. Gemini - Response format - Retain schema field ordering for google gemini and vertex by specifying propertyOrdering - [Get Started](../../docs/providers/vertex#json-schema), [PR](https://github.com/BerriAI/litellm/pull/9828) + 2. Gemini-2.5-flash - return reasoning content [ADD DOCS HERE], [PR](https://github.com/BerriAI/litellm/pull/10125) + 3. Gemini-2.5-flash - pricing + model information [PR](https://github.com/BerriAI/litellm/pull/10125) + 4. Passthrough - new `/vertex_ai/discovery` route - enables calling AgentBuilder API routes [ADD DOCS HERE], [PR](https://github.com/BerriAI/litellm/pull/10084) - **Fireworks AI** - 1. return tool calling responses in `tool_calls` field (fireworks incorrectly returns this as a json str in content) - https://github.com/BerriAI/litellm/pull/10130 + 1. return tool calling responses in `tool_calls` field (fireworks incorrectly returns this as a json str in content) - [Get Started](ADD DOCS HERE), [PR](https://github.com/BerriAI/litellm/pull/10130) - **Triton** - 1. Remove fixed remove bad_words / stop words from `/generate` call - https://github.com/BerriAI/litellm/pull/10163 + 1. Remove fixed remove bad_words / stop words from `/generate` call - [Get Started](../../docs/providers/triton-inference-server#triton-generate---chat-completion), [PR](https://github.com/BerriAI/litellm/pull/10163) - **Other** - 1. Support for all litellm providers on Responses API (works with Codex) - https://github.com/BerriAI/litellm/pull/10132 - 2. Fix combining multiple tool calls in streaming response - https://github.com/BerriAI/litellm/pull/10040 + 1. Support for all litellm providers on Responses API (works with Codex) - [Get Started](ADD DOCS HERE), [PR](https://github.com/BerriAI/litellm/pull/10132) + 2. Fix combining multiple tool calls in streaming response - [Get Started](../../docs/completion/stream#helper-function), [PR](https://github.com/BerriAI/litellm/pull/10040) ## Spend Tracking Improvements -- **Cost Control** - inject cache control points in prompt for cost reduction https://github.com/BerriAI/litellm/pull/10000 -- **Spend Tags** - spend tags in headers - support x-litellm-tags even if tag based routing not enabled -- **Gemini-2.5-flash** - support cost calculation for reasoning tokens +- **Cost Control** - inject cache control points in prompt for cost reduction [Get Started](../../docs/tutorials/prompt_caching), [PR](https://github.com/BerriAI/litellm/pull/10000) +- **Spend Tags** - spend tags in headers - support x-litellm-tags even if tag based routing not enabled [Get Started](ADD DOCS HERE), [PR](https://github.com/BerriAI/litellm/pull/10000) +- **Gemini-2.5-flash** - support cost calculation for reasoning tokens [PR](https://github.com/BerriAI/litellm/pull/10141) ## Management Endpoints / UI - **Users** - 1. Show created_at and updated_at on users page - https://github.com/BerriAI/litellm/pull/10033 + 1. Show created_at and updated_at on users page - [PR](https://github.com/BerriAI/litellm/pull/10033) - **Virtual Keys** 1. Filter by key alias - https://github.com/BerriAI/litellm/pull/10085 - **Usage Tab** + [ADD IMAGE/GIF HERE] 1. Team based usage - 1. New `LiteLLM_DailyTeamSpend` Table for aggregate team based usage logging - https://github.com/BerriAI/litellm/pull/10039 - 2. New Team based usage dashboard + new `/team/daily/activity` API - https://github.com/BerriAI/litellm/pull/10081 - 3. Return team alias on /team/daily/activity API - https://github.com/BerriAI/litellm/pull/10157 - 4. allow internal user view spend for teams they belong to - https://github.com/BerriAI/litellm/pull/10157 + 1. New `LiteLLM_DailyTeamSpend` Table for aggregate team based usage logging - [PR](https://github.com/BerriAI/litellm/pull/10039) + 2. New Team based usage dashboard + new `/team/daily/activity` API - [PR](https://github.com/BerriAI/litellm/pull/10081) + 3. Return team alias on /team/daily/activity API - [PR](https://github.com/BerriAI/litellm/pull/10157) + 4. allow internal user view spend for teams they belong to - [PR](https://github.com/BerriAI/litellm/pull/10157) 5. allow viewing top keys by team 2. Tag Based Usage - 1. New `LiteLLM_DailyTagSpend` Table for aggregate tag based usage logging - https://github.com/BerriAI/litellm/pull/10071 - 2. Restrict to only Proxy Admins - https://github.com/BerriAI/litellm/pull/10157 + 1. New `LiteLLM_DailyTagSpend` Table for aggregate tag based usage logging - [PR](https://github.com/BerriAI/litellm/pull/10071) + 2. Restrict to only Proxy Admins - [PR](https://github.com/BerriAI/litellm/pull/10157) 3. allow viewing top keys by tag 4. Return tags passed in request (i.e. dynamic tags) on `/tag/list` - 5. 3. Track prompt caching metrics in daily user, team, tag tables - https://github.com/BerriAI/litellm/pull/10029 4. Show usage by key (on all up, team, and tag usage dashboards) - https://github.com/BerriAI/litellm/pull/10157 5. swap old usage with new usage tab @@ -84,21 +84,21 @@ hide_table_of_contents: false - **API Playground** 1. Allow internal user to call api playground - https://github.com/BerriAI/litellm/pull/10157 - **SCIM** - 1. Add LiteLLM SCIM Integration for Team and User management - https://github.com/BerriAI/litellm/pull/10072 + 1. Add LiteLLM SCIM Integration for Team and User management - [Get Started](ADD DOCS HERE), [PR](https://github.com/BerriAI/litellm/pull/10072) ## Logging / Guardrail Integrations - **GCS** - 1. Fix gcs pub sub logging with env var GCS_PROJECT_ID - https://github.com/BerriAI/litellm/pull/10042 + 1. Fix gcs pub sub logging with env var GCS_PROJECT_ID - [Get Started](../../docs/observability/gcs_bucket_integration#usage), [PR](https://github.com/BerriAI/litellm/pull/10042) - **AIM** - 1. Add litellm call id passing to Aim guardrails on pre and post-hooks calls - https://github.com/BerriAI/litellm/pull/10021 + 1. Add litellm call id passing to Aim guardrails on pre and post-hooks calls - [Get Started](../../docs/proxy/guardrails/aim_security), [PR](https://github.com/BerriAI/litellm/pull/10021) - **Azure blob storage** - 1. Ensure logging works in high throughput scenarios - https://github.com/BerriAI/litellm/pull/9962 + 1. Ensure logging works in high throughput scenarios - [Get Started](../../docs/proxy/logging#azure-blob-storage), [PR](https://github.com/BerriAI/litellm/pull/9962) ## General Proxy Improvements -- **Support setting `litellm.modify_params` via env var** - https://github.com/BerriAI/litellm/pull/9964 -- **Model Discovery** - Check provider’s `/models` endpoints when calling proxy’s `/v1/models` endpoint - https://github.com/BerriAI/litellm/pull/9958 -- **`/utils/token_counter`** - fix retrieving custom tokenizer for db models - https://github.com/BerriAI/litellm/pull/10047 -- **Prisma migrate** - handle existing columns in db table - https://github.com/BerriAI/litellm/pull/10138 +- **Support setting `litellm.modify_params` via env var** [PR](https://github.com/BerriAI/litellm/pull/9964) +- **Model Discovery** - Check provider’s `/models` endpoints when calling proxy’s `/v1/models` endpoint - [Get Started](ADD DOCS HERE), [PR](https://github.com/BerriAI/litellm/pull/9958) +- **`/utils/token_counter`** - fix retrieving custom tokenizer for db models - [Get Started](../../docs/proxy/configs#set-custom-tokenizer), [PR](https://github.com/BerriAI/litellm/pull/10047) +- **Prisma migrate** - handle existing columns in db table - [PR](https://github.com/BerriAI/litellm/pull/10138)