mirror of
https://github.com/BerriAI/litellm.git
synced 2025-04-26 19:24:27 +00:00
docs(batches.md): add more examples to docs
This commit is contained in:
parent
b95d4bb61a
commit
09cfa948c9
2 changed files with 74 additions and 2 deletions
|
@ -151,3 +151,75 @@ Just add the azure env vars to your environment.
|
|||
export AZURE_API_KEY=""
|
||||
export AZURE_API_BASE=""
|
||||
```
|
||||
|
||||
AND use `/azure/*` for the Batches API calls
|
||||
|
||||
```bash
|
||||
http://0.0.0.0:4000/azure/v1/batches
|
||||
```
|
||||
### Usage
|
||||
|
||||
**Setup**
|
||||
|
||||
- Add Azure API Keys to your environment
|
||||
|
||||
#### 1. Upload a File
|
||||
|
||||
```bash
|
||||
curl http://localhost:4000/azure/v1/files \
|
||||
-H "Authorization: Bearer sk-1234" \
|
||||
-F purpose="batch" \
|
||||
-F file="@mydata.jsonl"
|
||||
```
|
||||
|
||||
**Example File**
|
||||
|
||||
Note: `model` should be your azure deployment name.
|
||||
|
||||
```json
|
||||
{"custom_id": "task-0", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "When was Microsoft founded?"}]}}
|
||||
{"custom_id": "task-1", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "When was the first XBOX released?"}]}}
|
||||
{"custom_id": "task-2", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "What is Altair Basic?"}]}}
|
||||
```
|
||||
|
||||
#### 2. Create a batch
|
||||
|
||||
```bash
|
||||
curl http://0.0.0.0:4000/azure/v1/batches \
|
||||
-H "Authorization: Bearer $LITELLM_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"input_file_id": "file-abc123",
|
||||
"endpoint": "/v1/chat/completions",
|
||||
"completion_window": "24h"
|
||||
}'
|
||||
|
||||
```
|
||||
|
||||
#### 3. Retrieve batch
|
||||
|
||||
|
||||
```bash
|
||||
curl http://0.0.0.0:4000/azure/v1/batches/batch_abc123 \
|
||||
-H "Authorization: Bearer $LITELLM_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
```
|
||||
|
||||
#### 4. Cancel batch
|
||||
|
||||
```bash
|
||||
curl http://0.0.0.0:4000/azure/v1/batches/batch_abc123/cancel \
|
||||
-H "Authorization: Bearer $LITELLM_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-X POST
|
||||
```
|
||||
|
||||
#### 5. List Batch
|
||||
|
||||
```bash
|
||||
curl http://0.0.0.0:4000/v1/batches?limit=2 \
|
||||
-H "Authorization: Bearer $LITELLM_API_KEY" \
|
||||
-H "Content-Type: application/json"
|
||||
```
|
||||
|
||||
### [👉 Health Check Azure Batch models](./proxy/health.md#batch-models-azure-only)
|
|
@ -123,7 +123,7 @@ For Azure models deployed as 'batch' models, set `mode: batch`.
|
|||
model_list:
|
||||
- model_name: "batch-gpt-4o-mini"
|
||||
litellm_params:
|
||||
model: "azure/gpt-4o-mini"
|
||||
model: "azure/batch-gpt-4o-mini"
|
||||
api_key: os.environ/AZURE_API_KEY
|
||||
api_base: os.environ/AZURE_API_BASE
|
||||
model_info:
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue