forked from phoenix/litellm-mirror
docs(azure_ai.md): add rerank api endpoint to docs
This commit is contained in:
parent
388e946df0
commit
920280155b
2 changed files with 87 additions and 1 deletions
|
@ -313,3 +313,88 @@ LiteLLM supports **ALL** azure ai models. Here's a few examples:
|
||||||
| AI21-Jamba-Instruct | `completion(model="azure_ai/ai21-jamba-instruct", messages)` |
|
| AI21-Jamba-Instruct | `completion(model="azure_ai/ai21-jamba-instruct", messages)` |
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## Rerank Endpoint
|
||||||
|
|
||||||
|
### Usage
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
<Tabs>
|
||||||
|
<TabItem value="sdk" label="LiteLLM SDK Usage">
|
||||||
|
|
||||||
|
```python
|
||||||
|
from litellm import rerank
|
||||||
|
import os
|
||||||
|
|
||||||
|
os.environ["AZURE_AI_API_KEY"] = "sk-.."
|
||||||
|
os.environ["AZURE_AI_API_BASE"] = "https://.."
|
||||||
|
|
||||||
|
query = "What is the capital of the United States?"
|
||||||
|
documents = [
|
||||||
|
"Carson City is the capital city of the American state of Nevada.",
|
||||||
|
"The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
|
||||||
|
"Washington, D.C. is the capital of the United States.",
|
||||||
|
"Capital punishment has existed in the United States since before it was a country.",
|
||||||
|
]
|
||||||
|
|
||||||
|
response = rerank(
|
||||||
|
model="azure_ai/rerank-english-v3.0",
|
||||||
|
query=query,
|
||||||
|
documents=documents,
|
||||||
|
top_n=3,
|
||||||
|
)
|
||||||
|
print(response)
|
||||||
|
```
|
||||||
|
</TabItem>
|
||||||
|
|
||||||
|
<TabItem value="proxy" label="LiteLLM Proxy Usage">
|
||||||
|
|
||||||
|
LiteLLM provides an cohere api compatible `/rerank` endpoint for Rerank calls.
|
||||||
|
|
||||||
|
**Setup**
|
||||||
|
|
||||||
|
Add this to your litellm proxy config.yaml
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
model_list:
|
||||||
|
- model_name: Salesforce/Llama-Rank-V1
|
||||||
|
litellm_params:
|
||||||
|
model: together_ai/Salesforce/Llama-Rank-V1
|
||||||
|
api_key: os.environ/TOGETHERAI_API_KEY
|
||||||
|
- model_name: rerank-english-v3.0
|
||||||
|
litellm_params:
|
||||||
|
model: azure_ai/rerank-english-v3.0
|
||||||
|
api_key: os.environ/AZURE_AI_API_KEY
|
||||||
|
api_base: os.environ/AZURE_AI_API_BASE
|
||||||
|
```
|
||||||
|
|
||||||
|
Start litellm
|
||||||
|
|
||||||
|
```bash
|
||||||
|
litellm --config /path/to/config.yaml
|
||||||
|
|
||||||
|
# RUNNING on http://0.0.0.0:4000
|
||||||
|
```
|
||||||
|
|
||||||
|
Test request
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://0.0.0.0:4000/rerank \
|
||||||
|
-H "Authorization: Bearer sk-1234" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{
|
||||||
|
"model": "rerank-english-v3.0",
|
||||||
|
"query": "What is the capital of the United States?",
|
||||||
|
"documents": [
|
||||||
|
"Carson City is the capital city of the American state of Nevada.",
|
||||||
|
"The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
|
||||||
|
"Washington, D.C. is the capital of the United States.",
|
||||||
|
"Capital punishment has existed in the United States since before it was a country."
|
||||||
|
],
|
||||||
|
"top_n": 3
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
</TabItem>
|
||||||
|
</Tabs>
|
|
@ -113,3 +113,4 @@ curl http://0.0.0.0:4000/rerank \
|
||||||
|-------------|--------------------|
|
|-------------|--------------------|
|
||||||
| Cohere | [Usage](#quick-start) |
|
| Cohere | [Usage](#quick-start) |
|
||||||
| Together AI| [Usage](../docs/providers/togetherai) |
|
| Together AI| [Usage](../docs/providers/togetherai) |
|
||||||
|
| Azure AI| [Usage](../docs/providers/azure_ai) |
|
Loading…
Add table
Add a link
Reference in a new issue