phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

docs - passing extra_headers to anthropic api

Browse source
This commit is contained in:
Ishaan Jaff 2024-07-15 21:15:39 -07:00
parent 155ee7e99c
commit eaee90b02e
2 changed files with 68 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

13
docs/my-website/docs/providers/anthropic.md
View file

@ -183,6 +183,19 @@ print(response)
| claude-instant-1.2 | `completion('claude-instant-1.2', messages)` | `os.environ['ANTHROPIC_API_KEY']` |
| claude-instant-1 | `completion('claude-instant-1', messages)` | `os.environ['ANTHROPIC_API_KEY']` |
## Passing Extra Headers to Anthropic API
Pass `extra_headers: dict` to `litellm.completion`
```python
from litellm import completion
messages = [{"role": "user", "content": "What is Anthropic?"}]
response = completion(
model="claude-3-5-sonnet-20240620",
messages=messages,
extra_headers={"anthropic-beta": "max-tokens-3-5-sonnet-2024-07-15"}
)
```
## Advanced
## Usage - Function Calling

55
litellm/model_prices_and_context_window_backup.json
View file

@ -4085,6 +4085,61 @@
"input_cost_per_request": 0.005,
"litellm_provider": "perplexity",
"mode": "chat"
},
"fireworks_ai/firefunction-v2": {
"max_tokens": 8192,
"max_input_tokens": 8192,
"max_output_tokens": 8192,
"input_cost_per_token": 0.0000009,
"output_cost_per_token": 0.0000009,
"litellm_provider": "fireworks_ai",
"mode": "chat",
"supports_function_calling": true,
"source": "https://fireworks.ai/pricing"
},
"fireworks_ai/mixtral-8x22b-instruct-hf": {
"max_tokens": 65536,
"max_input_tokens": 65536,
"max_output_tokens": 65536,
"input_cost_per_token": 0.0000012,
"output_cost_per_token": 0.0000012,
"litellm_provider": "fireworks_ai",
"mode": "chat",
"supports_function_calling": true,
"source": "https://fireworks.ai/pricing"
},
"fireworks_ai/qwen2-72b-instruct": {
"max_tokens": 32768,
"max_input_tokens": 32768,
"max_output_tokens": 32768,
"input_cost_per_token": 0.0000009,
"output_cost_per_token": 0.0000009,
"litellm_provider": "fireworks_ai",
"mode": "chat",
"supports_function_calling": true,
"source": "https://fireworks.ai/pricing"
},
"fireworks_ai/yi-large": {
"max_tokens": 32768,
"max_input_tokens": 32768,
"max_output_tokens": 32768,
"input_cost_per_token": 0.000003,
"output_cost_per_token": 0.000003,
"litellm_provider": "fireworks_ai",
"mode": "chat",
"supports_function_calling": true,
"source": "https://fireworks.ai/pricing"
},
"fireworks_ai/deepseek-coder-v2-instruct": {
"max_tokens": 65536,
"max_input_tokens": 65536,
"max_output_tokens": 8192,
"input_cost_per_token": 0.0000012,
"output_cost_per_token": 0.0000012,
"litellm_provider": "fireworks_ai",
"mode": "chat",
"supports_function_calling": true,
"source": "https://fireworks.ai/pricing"
},
"anyscale/mistralai/Mistral-7B-Instruct-v0.1": {
"max_tokens": 16384,