mirror of
https://github.com/BerriAI/litellm.git
synced 2025-04-26 19:24:27 +00:00
docs(batches.md): add more examples to docs
This commit is contained in:
parent
b95d4bb61a
commit
09cfa948c9
2 changed files with 74 additions and 2 deletions
|
@ -150,4 +150,76 @@ Just add the azure env vars to your environment.
|
||||||
```bash
|
```bash
|
||||||
export AZURE_API_KEY=""
|
export AZURE_API_KEY=""
|
||||||
export AZURE_API_BASE=""
|
export AZURE_API_BASE=""
|
||||||
```
|
```
|
||||||
|
|
||||||
|
AND use `/azure/*` for the Batches API calls
|
||||||
|
|
||||||
|
```bash
|
||||||
|
http://0.0.0.0:4000/azure/v1/batches
|
||||||
|
```
|
||||||
|
### Usage
|
||||||
|
|
||||||
|
**Setup**
|
||||||
|
|
||||||
|
- Add Azure API Keys to your environment
|
||||||
|
|
||||||
|
#### 1. Upload a File
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:4000/azure/v1/files \
|
||||||
|
-H "Authorization: Bearer sk-1234" \
|
||||||
|
-F purpose="batch" \
|
||||||
|
-F file="@mydata.jsonl"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example File**
|
||||||
|
|
||||||
|
Note: `model` should be your azure deployment name.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"custom_id": "task-0", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "When was Microsoft founded?"}]}}
|
||||||
|
{"custom_id": "task-1", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "When was the first XBOX released?"}]}}
|
||||||
|
{"custom_id": "task-2", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "What is Altair Basic?"}]}}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 2. Create a batch
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://0.0.0.0:4000/azure/v1/batches \
|
||||||
|
-H "Authorization: Bearer $LITELLM_API_KEY" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{
|
||||||
|
"input_file_id": "file-abc123",
|
||||||
|
"endpoint": "/v1/chat/completions",
|
||||||
|
"completion_window": "24h"
|
||||||
|
}'
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 3. Retrieve batch
|
||||||
|
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://0.0.0.0:4000/azure/v1/batches/batch_abc123 \
|
||||||
|
-H "Authorization: Bearer $LITELLM_API_KEY" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 4. Cancel batch
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://0.0.0.0:4000/azure/v1/batches/batch_abc123/cancel \
|
||||||
|
-H "Authorization: Bearer $LITELLM_API_KEY" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-X POST
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 5. List Batch
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://0.0.0.0:4000/v1/batches?limit=2 \
|
||||||
|
-H "Authorization: Bearer $LITELLM_API_KEY" \
|
||||||
|
-H "Content-Type: application/json"
|
||||||
|
```
|
||||||
|
|
||||||
|
### [👉 Health Check Azure Batch models](./proxy/health.md#batch-models-azure-only)
|
|
@ -123,7 +123,7 @@ For Azure models deployed as 'batch' models, set `mode: batch`.
|
||||||
model_list:
|
model_list:
|
||||||
- model_name: "batch-gpt-4o-mini"
|
- model_name: "batch-gpt-4o-mini"
|
||||||
litellm_params:
|
litellm_params:
|
||||||
model: "azure/gpt-4o-mini"
|
model: "azure/batch-gpt-4o-mini"
|
||||||
api_key: os.environ/AZURE_API_KEY
|
api_key: os.environ/AZURE_API_KEY
|
||||||
api_base: os.environ/AZURE_API_BASE
|
api_base: os.environ/AZURE_API_BASE
|
||||||
model_info:
|
model_info:
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue