forked from phoenix/litellm-mirror
add Azure OpenAI entrata id docs (#5985)
This commit is contained in:
Ishaan Jaff
GitHub
parent
30aa04b8c2
commit
ca9c437021
1 changed files with 148 additions and 106 deletions
|
|
@ -82,9 +82,6 @@ export AZURE_API_KEY=""
|
||||||
|
|
||||||
### 2. Start the proxy
|
### 2. Start the proxy
|
||||||
|
|
||||||
<Tabs>
|
|
||||||
<TabItem value="config" label="config.yaml">
|
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
model_list:
|
model_list:
|
||||||
- model_name: gpt-3.5-turbo
|
- model_name: gpt-3.5-turbo
|
||||||
|
|
@ -94,28 +91,9 @@ model_list:
|
||||||
api_version: "2023-05-15"
|
api_version: "2023-05-15"
|
||||||
api_key: os.environ/AZURE_API_KEY # The `os.environ/` prefix tells litellm to read this from the env.
|
api_key: os.environ/AZURE_API_KEY # The `os.environ/` prefix tells litellm to read this from the env.
|
||||||
```
|
```
|
||||||
</TabItem>
|
|
||||||
<TabItem value="config-*" label="config.yaml (Entrata ID) use tenant_id, client_id, client_secret">
|
|
||||||
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
model_list:
|
|
||||||
- model_name: gpt-3.5-turbo
|
|
||||||
litellm_params:
|
|
||||||
model: azure/chatgpt-v-2
|
|
||||||
api_base: https://openai-gpt-4-test-v-1.openai.azure.com/
|
|
||||||
api_version: "2023-05-15"
|
|
||||||
tenant_id: os.environ/AZURE_TENANT_ID
|
|
||||||
client_id: os.environ/AZURE_CLIENT_ID
|
|
||||||
client_secret: os.environ/AZURE_CLIENT_SECRET
|
|
||||||
```
|
|
||||||
</TabItem>
|
|
||||||
|
|
||||||
</Tabs>
|
|
||||||
|
|
||||||
### 3. Test it
|
### 3. Test it
|
||||||
|
|
||||||
|
|
||||||
<Tabs>
|
<Tabs>
|
||||||
<TabItem value="Curl" label="Curl Request">
|
<TabItem value="Curl" label="Curl Request">
|
||||||
|
|
||||||
|
|
@ -360,6 +338,153 @@ response = speech(
|
||||||
response.stream_to_file(speech_file_path)
|
response.stream_to_file(speech_file_path)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## **Authentication**
|
||||||
|
|
||||||
|
|
||||||
|
### Entrata ID - use `azure_ad_token`
|
||||||
|
|
||||||
|
This is a walkthrough on how to use Azure Active Directory Tokens - Microsoft Entra ID to make `litellm.completion()` calls
|
||||||
|
|
||||||
|
Step 1 - Download Azure CLI
|
||||||
|
Installation instructons: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli
|
||||||
|
```shell
|
||||||
|
brew update && brew install azure-cli
|
||||||
|
```
|
||||||
|
Step 2 - Sign in using `az`
|
||||||
|
```shell
|
||||||
|
az login --output table
|
||||||
|
```
|
||||||
|
|
||||||
|
Step 3 - Generate azure ad token
|
||||||
|
```shell
|
||||||
|
az account get-access-token --resource https://cognitiveservices.azure.com
|
||||||
|
```
|
||||||
|
|
||||||
|
In this step you should see an `accessToken` generated
|
||||||
|
```shell
|
||||||
|
{
|
||||||
|
"accessToken": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IjlHbW55RlBraGMzaE91UjIybXZTdmduTG83WSIsImtpZCI6IjlHbW55RlBraGMzaE91UjIybXZTdmduTG83WSJ9",
|
||||||
|
"expiresOn": "2023-11-14 15:50:46.000000",
|
||||||
|
"expires_on": 1700005846,
|
||||||
|
"subscription": "db38de1f-4bb3..",
|
||||||
|
"tenant": "bdfd79b3-8401-47..",
|
||||||
|
"tokenType": "Bearer"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Step 4 - Make litellm.completion call with Azure AD token
|
||||||
|
|
||||||
|
Set `azure_ad_token` = `accessToken` from step 3 or set `os.environ['AZURE_AD_TOKEN']`
|
||||||
|
|
||||||
|
|
||||||
|
<Tabs>
|
||||||
|
<TabItem value="sdk" label="SDK">
|
||||||
|
|
||||||
|
|
||||||
|
```python
|
||||||
|
response = litellm.completion(
|
||||||
|
model = "azure/<your deployment name>", # model = azure/<your deployment name>
|
||||||
|
api_base = "", # azure api base
|
||||||
|
api_version = "", # azure api version
|
||||||
|
azure_ad_token="", # your accessToken from step 3
|
||||||
|
messages = [{"role": "user", "content": "good morning"}],
|
||||||
|
)
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
</TabItem>
|
||||||
|
<TabItem value="proxy" label="PROXY config.yaml">
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
model_list:
|
||||||
|
- model_name: gpt-3.5-turbo
|
||||||
|
litellm_params:
|
||||||
|
model: azure/chatgpt-v-2
|
||||||
|
api_base: https://openai-gpt-4-test-v-1.openai.azure.com/
|
||||||
|
api_version: "2023-05-15"
|
||||||
|
azure_ad_token: os.environ/AZURE_AD_TOKEN
|
||||||
|
```
|
||||||
|
|
||||||
|
</TabItem>
|
||||||
|
</Tabs>
|
||||||
|
|
||||||
|
### Entrata ID - use tenant_id, client_id, client_secret
|
||||||
|
|
||||||
|
Here is an example of setting up `tenant_id`, `client_id`, `client_secret` in your litellm proxy `config.yaml`
|
||||||
|
```yaml
|
||||||
|
model_list:
|
||||||
|
- model_name: gpt-3.5-turbo
|
||||||
|
litellm_params:
|
||||||
|
model: azure/chatgpt-v-2
|
||||||
|
api_base: https://openai-gpt-4-test-v-1.openai.azure.com/
|
||||||
|
api_version: "2023-05-15"
|
||||||
|
tenant_id: os.environ/AZURE_TENANT_ID
|
||||||
|
client_id: os.environ/AZURE_CLIENT_ID
|
||||||
|
client_secret: os.environ/AZURE_CLIENT_SECRET
|
||||||
|
```
|
||||||
|
|
||||||
|
Test it
|
||||||
|
|
||||||
|
```shell
|
||||||
|
curl --location 'http://0.0.0.0:4000/chat/completions' \
|
||||||
|
--header 'Content-Type: application/json' \
|
||||||
|
--data ' {
|
||||||
|
"model": "gpt-3.5-turbo",
|
||||||
|
"messages": [
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": "what llm are you"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
'
|
||||||
|
```
|
||||||
|
|
||||||
|
Example video of using `tenant_id`, `client_id`, `client_secret` with LiteLLM Proxy Server
|
||||||
|
|
||||||
|
<iframe width="840" height="500" src="https://www.loom.com/embed/70d3f219ee7f4e5d84778b7f17bba506?sid=04b8ff29-485f-4cb8-929e-6b392722f36d" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
|
||||||
|
|
||||||
|
### Azure AD Token Refresh - `DefaultAzureCredential`
|
||||||
|
|
||||||
|
Use this if you want to use Azure `DefaultAzureCredential` for Authentication on your requests
|
||||||
|
|
||||||
|
<Tabs>
|
||||||
|
<TabItem value="sdk" label="SDK">
|
||||||
|
|
||||||
|
```python
|
||||||
|
from litellm import completion
|
||||||
|
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
|
||||||
|
|
||||||
|
token_provider = get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default")
|
||||||
|
|
||||||
|
|
||||||
|
response = completion(
|
||||||
|
model = "azure/<your deployment name>", # model = azure/<your deployment name>
|
||||||
|
api_base = "", # azure api base
|
||||||
|
api_version = "", # azure api version
|
||||||
|
azure_ad_token_provider=token_provider
|
||||||
|
messages = [{"role": "user", "content": "good morning"}],
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
</TabItem>
|
||||||
|
<TabItem value="proxy" label="PROXY config.yaml">
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
model_list:
|
||||||
|
- model_name: gpt-3.5-turbo
|
||||||
|
litellm_params:
|
||||||
|
model: azure/your-deployment-name
|
||||||
|
api_base: https://openai-gpt-4-test-v-1.openai.azure.com/
|
||||||
|
|
||||||
|
litellm_settings:
|
||||||
|
enable_azure_ad_token_refresh: true # 👈 KEY CHANGE
|
||||||
|
```
|
||||||
|
|
||||||
|
</TabItem>
|
||||||
|
</Tabs>
|
||||||
|
|
||||||
|
|
||||||
## Advanced
|
## Advanced
|
||||||
### Azure API Load-Balancing
|
### Azure API Load-Balancing
|
||||||
|
|
||||||
|
|
@ -486,87 +611,4 @@ print("\nLLM Response1:\n", response)
|
||||||
response_message = response.choices[0].message
|
response_message = response.choices[0].message
|
||||||
tool_calls = response.choices[0].message.tool_calls
|
tool_calls = response.choices[0].message.tool_calls
|
||||||
print("\nTool Choice:\n", tool_calls)
|
print("\nTool Choice:\n", tool_calls)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
### Authentication with Azure Active Directory Tokens (Microsoft Entra ID)
|
|
||||||
This is a walkthrough on how to use Azure Active Directory Tokens - Microsoft Entra ID to make `litellm.completion()` calls
|
|
||||||
|
|
||||||
Step 1 - Download Azure CLI
|
|
||||||
Installation instructons: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli
|
|
||||||
```shell
|
|
||||||
brew update && brew install azure-cli
|
|
||||||
```
|
|
||||||
Step 2 - Sign in using `az`
|
|
||||||
```shell
|
|
||||||
az login --output table
|
|
||||||
```
|
|
||||||
|
|
||||||
Step 3 - Generate azure ad token
|
|
||||||
```shell
|
|
||||||
az account get-access-token --resource https://cognitiveservices.azure.com
|
|
||||||
```
|
|
||||||
|
|
||||||
In this step you should see an `accessToken` generated
|
|
||||||
```shell
|
|
||||||
{
|
|
||||||
"accessToken": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IjlHbW55RlBraGMzaE91UjIybXZTdmduTG83WSIsImtpZCI6IjlHbW55RlBraGMzaE91UjIybXZTdmduTG83WSJ9",
|
|
||||||
"expiresOn": "2023-11-14 15:50:46.000000",
|
|
||||||
"expires_on": 1700005846,
|
|
||||||
"subscription": "db38de1f-4bb3..",
|
|
||||||
"tenant": "bdfd79b3-8401-47..",
|
|
||||||
"tokenType": "Bearer"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Step 4 - Make litellm.completion call with Azure AD token
|
|
||||||
|
|
||||||
Set `azure_ad_token` = `accessToken` from step 3 or set `os.environ['AZURE_AD_TOKEN']`
|
|
||||||
|
|
||||||
```python
|
|
||||||
response = litellm.completion(
|
|
||||||
model = "azure/<your deployment name>", # model = azure/<your deployment name>
|
|
||||||
api_base = "", # azure api base
|
|
||||||
api_version = "", # azure api version
|
|
||||||
azure_ad_token="", # your accessToken from step 3
|
|
||||||
messages = [{"role": "user", "content": "good morning"}],
|
|
||||||
)
|
|
||||||
|
|
||||||
```
|
|
||||||
### Azure AD Token Refresh
|
|
||||||
|
|
||||||
<Tabs>
|
|
||||||
<TabItem value="sdk" label="SDK">
|
|
||||||
|
|
||||||
```python
|
|
||||||
from litellm import completion
|
|
||||||
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
|
|
||||||
|
|
||||||
token_provider = get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default")
|
|
||||||
|
|
||||||
|
|
||||||
response = completion(
|
|
||||||
model = "azure/<your deployment name>", # model = azure/<your deployment name>
|
|
||||||
api_base = "", # azure api base
|
|
||||||
api_version = "", # azure api version
|
|
||||||
azure_ad_token_provider=token_provider
|
|
||||||
messages = [{"role": "user", "content": "good morning"}],
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
</TabItem>
|
|
||||||
<TabItem value="proxy" label="PROXY config.yaml">
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
model_list:
|
|
||||||
- model_name: gpt-3.5-turbo
|
|
||||||
litellm_params:
|
|
||||||
model: azure/your-deployment-name
|
|
||||||
api_base: https://openai-gpt-4-test-v-1.openai.azure.com/
|
|
||||||
|
|
||||||
litellm_settings:
|
|
||||||
enable_azure_ad_token_refresh: true # 👈 KEY CHANGE
|
|
||||||
```
|
|
||||||
|
|
||||||
</TabItem>
|
|
||||||
</Tabs>
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue