docs(batches.md): add more examples to docs

This commit is contained in:
Krrish Dholakia 2024-08-22 18:51:44 -07:00
parent b95d4bb61a
commit 09cfa948c9
2 changed files with 74 additions and 2 deletions

View file

@ -151,3 +151,75 @@ Just add the azure env vars to your environment.
export AZURE_API_KEY=""
export AZURE_API_BASE=""
```
AND use `/azure/*` for the Batches API calls
```bash
http://0.0.0.0:4000/azure/v1/batches
```
### Usage
**Setup**
- Add Azure API Keys to your environment
#### 1. Upload a File
```bash
curl http://localhost:4000/azure/v1/files \
-H "Authorization: Bearer sk-1234" \
-F purpose="batch" \
-F file="@mydata.jsonl"
```
**Example File**
Note: `model` should be your azure deployment name.
```json
{"custom_id": "task-0", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "When was Microsoft founded?"}]}}
{"custom_id": "task-1", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "When was the first XBOX released?"}]}}
{"custom_id": "task-2", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "What is Altair Basic?"}]}}
```
#### 2. Create a batch
```bash
curl http://0.0.0.0:4000/azure/v1/batches \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"input_file_id": "file-abc123",
"endpoint": "/v1/chat/completions",
"completion_window": "24h"
}'
```
#### 3. Retrieve batch
```bash
curl http://0.0.0.0:4000/azure/v1/batches/batch_abc123 \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-H "Content-Type: application/json" \
```
#### 4. Cancel batch
```bash
curl http://0.0.0.0:4000/azure/v1/batches/batch_abc123/cancel \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-H "Content-Type: application/json" \
-X POST
```
#### 5. List Batch
```bash
curl http://0.0.0.0:4000/v1/batches?limit=2 \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-H "Content-Type: application/json"
```
### [👉 Health Check Azure Batch models](./proxy/health.md#batch-models-azure-only)

View file

@ -123,7 +123,7 @@ For Azure models deployed as 'batch' models, set `mode: batch`.
model_list:
- model_name: "batch-gpt-4o-mini"
litellm_params:
model: "azure/gpt-4o-mini"
model: "azure/batch-gpt-4o-mini"
api_key: os.environ/AZURE_API_KEY
api_base: os.environ/AZURE_API_BASE
model_info: