mirror of
https://github.com/BerriAI/litellm.git
synced 2025-04-26 03:04:13 +00:00
docs azure responses API
This commit is contained in:
parent
409dde22f6
commit
19664960eb
2 changed files with 131 additions and 5 deletions
|
@ -1002,9 +1002,127 @@ Expected Response:
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## **Azure Responses API**
|
||||||
|
|
||||||
|
| Property | Details |
|
||||||
|
|-------|-------|
|
||||||
|
| Description | Azure OpenAI Responses API |
|
||||||
|
| `custom_llm_provider` on LiteLLM | `azure/` |
|
||||||
|
| Supported Operations | `/v1/responses`|
|
||||||
|
| Azure OpenAI Responses API | [Azure OpenAI Responses API ↗](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/responses?tabs=python-secure) |
|
||||||
|
| Cost Tracking, Logging Support | ✅ LiteLLM will log, track cost for Responses API Requests |
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
## Create a model response
|
||||||
|
|
||||||
|
<Tabs>
|
||||||
|
<TabItem value="litellm-sdk" label="LiteLLM SDK">
|
||||||
|
|
||||||
|
#### Non-streaming
|
||||||
|
|
||||||
|
```python showLineNumbers title="Azure Responses API"
|
||||||
|
import litellm
|
||||||
|
|
||||||
|
# Non-streaming response
|
||||||
|
response = litellm.responses(
|
||||||
|
model="azure/o1-pro",
|
||||||
|
input="Tell me a three sentence bedtime story about a unicorn.",
|
||||||
|
max_output_tokens=100,
|
||||||
|
api_key=os.getenv("AZURE_RESPONSES_OPENAI_API_KEY"),
|
||||||
|
api_base="https://litellm8397336933.openai.azure.com/",
|
||||||
|
api_version="2023-03-15-preview",
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response)
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Streaming
|
||||||
|
```python showLineNumbers title="Azure Responses API"
|
||||||
|
import litellm
|
||||||
|
|
||||||
|
# Streaming response
|
||||||
|
response = litellm.responses(
|
||||||
|
model="azure/o1-pro",
|
||||||
|
input="Tell me a three sentence bedtime story about a unicorn.",
|
||||||
|
stream=True,
|
||||||
|
api_key=os.getenv("AZURE_RESPONSES_OPENAI_API_KEY"),
|
||||||
|
api_base="https://litellm8397336933.openai.azure.com/",
|
||||||
|
api_version="2023-03-15-preview",
|
||||||
|
)
|
||||||
|
|
||||||
|
for event in response:
|
||||||
|
print(event)
|
||||||
|
```
|
||||||
|
|
||||||
|
</TabItem>
|
||||||
|
<TabItem value="proxy" label="OpenAI SDK with LiteLLM Proxy">
|
||||||
|
|
||||||
|
First, add this to your litellm proxy config.yaml:
|
||||||
|
```yaml showLineNumbers title="Azure Responses API"
|
||||||
|
model_list:
|
||||||
|
- model_name: o1-pro
|
||||||
|
litellm_params:
|
||||||
|
model: azure/o1-pro
|
||||||
|
api_key: os.environ/AZURE_RESPONSES_OPENAI_API_KEY
|
||||||
|
api_base: https://litellm8397336933.openai.azure.com/
|
||||||
|
api_version: 2023-03-15-preview
|
||||||
|
```
|
||||||
|
|
||||||
|
Start your LiteLLM proxy:
|
||||||
|
```bash
|
||||||
|
litellm --config /path/to/config.yaml
|
||||||
|
|
||||||
|
# RUNNING on http://0.0.0.0:4000
|
||||||
|
```
|
||||||
|
|
||||||
|
Then use the OpenAI SDK pointed to your proxy:
|
||||||
|
|
||||||
|
#### Non-streaming
|
||||||
|
```python showLineNumbers
|
||||||
|
from openai import OpenAI
|
||||||
|
|
||||||
|
# Initialize client with your proxy URL
|
||||||
|
client = OpenAI(
|
||||||
|
base_url="http://localhost:4000", # Your proxy URL
|
||||||
|
api_key="your-api-key" # Your proxy API key
|
||||||
|
)
|
||||||
|
|
||||||
|
# Non-streaming response
|
||||||
|
response = client.responses.create(
|
||||||
|
model="o1-pro",
|
||||||
|
input="Tell me a three sentence bedtime story about a unicorn."
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response)
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Streaming
|
||||||
|
```python showLineNumbers
|
||||||
|
from openai import OpenAI
|
||||||
|
|
||||||
|
# Initialize client with your proxy URL
|
||||||
|
client = OpenAI(
|
||||||
|
base_url="http://localhost:4000", # Your proxy URL
|
||||||
|
api_key="your-api-key" # Your proxy API key
|
||||||
|
)
|
||||||
|
|
||||||
|
# Streaming response
|
||||||
|
response = client.responses.create(
|
||||||
|
model="o1-pro",
|
||||||
|
input="Tell me a three sentence bedtime story about a unicorn.",
|
||||||
|
stream=True
|
||||||
|
)
|
||||||
|
|
||||||
|
for event in response:
|
||||||
|
print(event)
|
||||||
|
```
|
||||||
|
|
||||||
|
</TabItem>
|
||||||
|
</Tabs>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## Advanced
|
## Advanced
|
||||||
|
|
|
@ -24,7 +24,7 @@ LiteLLM provides a BETA endpoint in the spec of [OpenAI's `/responses` API](http
|
||||||
<TabItem value="litellm-sdk" label="LiteLLM SDK">
|
<TabItem value="litellm-sdk" label="LiteLLM SDK">
|
||||||
|
|
||||||
#### Non-streaming
|
#### Non-streaming
|
||||||
```python
|
```python showLineNumbers
|
||||||
import litellm
|
import litellm
|
||||||
|
|
||||||
# Non-streaming response
|
# Non-streaming response
|
||||||
|
@ -38,7 +38,7 @@ print(response)
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Streaming
|
#### Streaming
|
||||||
```python
|
```python showLineNumbers
|
||||||
import litellm
|
import litellm
|
||||||
|
|
||||||
# Streaming response
|
# Streaming response
|
||||||
|
@ -56,7 +56,7 @@ for event in response:
|
||||||
<TabItem value="proxy" label="OpenAI SDK with LiteLLM Proxy">
|
<TabItem value="proxy" label="OpenAI SDK with LiteLLM Proxy">
|
||||||
|
|
||||||
First, add this to your litellm proxy config.yaml:
|
First, add this to your litellm proxy config.yaml:
|
||||||
```yaml
|
```yaml showLineNumbers
|
||||||
model_list:
|
model_list:
|
||||||
- model_name: o1-pro
|
- model_name: o1-pro
|
||||||
litellm_params:
|
litellm_params:
|
||||||
|
@ -74,7 +74,7 @@ litellm --config /path/to/config.yaml
|
||||||
Then use the OpenAI SDK pointed to your proxy:
|
Then use the OpenAI SDK pointed to your proxy:
|
||||||
|
|
||||||
#### Non-streaming
|
#### Non-streaming
|
||||||
```python
|
```python showLineNumbers
|
||||||
from openai import OpenAI
|
from openai import OpenAI
|
||||||
|
|
||||||
# Initialize client with your proxy URL
|
# Initialize client with your proxy URL
|
||||||
|
@ -93,7 +93,7 @@ print(response)
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Streaming
|
#### Streaming
|
||||||
```python
|
```python showLineNumbers
|
||||||
from openai import OpenAI
|
from openai import OpenAI
|
||||||
|
|
||||||
# Initialize client with your proxy URL
|
# Initialize client with your proxy URL
|
||||||
|
@ -115,3 +115,11 @@ for event in response:
|
||||||
|
|
||||||
</TabItem>
|
</TabItem>
|
||||||
</Tabs>
|
</Tabs>
|
||||||
|
|
||||||
|
|
||||||
|
## **Supported Providers**
|
||||||
|
|
||||||
|
| Provider | Link to Usage |
|
||||||
|
|-------------|--------------------|
|
||||||
|
| OpenAI| [Usage](#usage) |
|
||||||
|
| Azure OpenAI| [Usage](../docs/providers/azure#responses-api) |
|
Loading…
Add table
Add a link
Reference in a new issue