phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

(docs) add deepinfra models + pricing

Browse source
This commit is contained in:
ishaan-jaff 2023-10-14 10:04:49 -07:00
parent 1e7bae4aea
commit 8251e71a2a
2 changed files with 60 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

22
docs/my-website/docs/providers/deepinfra.md
View file

@ -1,5 +1,12 @@
# DeepInfra
https://deepinfra.com/
## API Key
```python
# env variable
os.environ['DEEPINFRA_API_KEY']
```
## Sample Usage
```python
from litellm import completion
@ -29,10 +36,15 @@ for chunk in response:
```
## Chat Models
| Model Name | Function Call | Required OS Variables |
|------------------|--------------------------------------|-------------------------|
| meta-llama/Llama-2-70b-chat-hf | `completion(model="deepinfra/meta-llama/Llama-2-70b-chat-hf", messages)` | `os.environ['DEEPINFRA_API_KEY']` |
| meta-llama/Llama-2-7b-chat-hf | `completion(model="deepinfra/meta-llama/Llama-2-7b-chat-hf", messages)` | `os.environ['DEEPINFRA_API_KEY']` |
| codellama/CodeLlama-34b-Instruct-hf | `completion(model="deepinfra/codellama/CodeLlama-34b-Instruct-hf", messages)` | `os.environ['DEEPINFRA_API_KEY']` |
| Model Name | Function Call |
|------------------|--------------------------------------|
| meta-llama/Llama-2-70b-chat-hf | `completion(model="deepinfra/meta-llama/Llama-2-70b-chat-hf", messages)` |
| meta-llama/Llama-2-7b-chat-hf | `completion(model="deepinfra/meta-llama/Llama-2-7b-chat-hf", messages)` |
| meta-llama/Llama-2-13b-chat-hf | `completion(model="deepinfra/meta-llama/Llama-2-13b-chat-hf", messages)` |
| codellama/CodeLlama-34b-Instruct-hf | `completion(model="deepinfra/codellama/CodeLlama-34b-Instruct-hf", messages)` |
| mistralai/Mistral-7B-Instruct-v0.1 | `completion(model="deepinfra/mistralai/Mistral-7B-Instruct-v0.1", messages)` |
| jondurbin/airoboros-l2-70b-gpt4-1.4.1 | `completion(model="deepinfra/jondurbin/airoboros-l2-70b-gpt4-1.4.1", messages)` |

44
model_prices_and_context_window.json
View file

@ -570,5 +570,47 @@
"output_cost_per_token": 0.0,
"litellm_provider": "ollama",
"mode": "completion"
}
},
"deepinfra/meta-llama/Llama-2-70b-chat-hf": {
"max_tokens": 6144,
"input_cost_per_token": 0.000001875,
"output_cost_per_token": 0.000001875,
"litellm_provider": "openrouter",
"mode": "chat"
},
"deepinfra/codellama/CodeLlama-34b-Instruct-hf": {
"max_tokens": 4096,
"input_cost_per_token": 0.0000006,
"output_cost_per_token": 0.0000006,
"litellm_provider": "deepinfra",
"mode": "chat"
},
"deepinfra/meta-llama/Llama-2-13b-chat-hf": {
"max_tokens": 4096,
"input_cost_per_token": 0.00000035,
"output_cost_per_token": 0.00000035,
"litellm_provider": "deepinfra",
"mode": "chat"
},
"deepinfra/meta-llama/Llama-2-7b-chat-hf": {
"max_tokens": 4096,
"input_cost_per_token": 0.0000002,
"output_cost_per_token": 0.0000002,
"litellm_provider": "deepinfra",
"mode": "chat"
},
"deepinfra/mistralai/Mistral-7B-Instruct-v0.1": {
"max_tokens": 4096,
"input_cost_per_token": 0.0000002,
"output_cost_per_token": 0.0000002,
"litellm_provider": "deepinfra",
"mode": "chat"
},
"deepinfra/jondurbin/airoboros-l2-70b-gpt4-1.4.1": {
"max_tokens": 4096,
"input_cost_per_token": 0.0000007,
"output_cost_per_token": 0.00000095,
"litellm_provider": "deepinfra",
"mode": "chat"
}
}