forked from phoenix/litellm-mirror
update docs
This commit is contained in:
parent
ac5fcb673b
commit
8094301424
6 changed files with 100 additions and 24 deletions
|
@ -5,11 +5,30 @@ LiteLLM supports j2-light, j2-mid and j2-ultra from [AI21](https://www.ai21.com/
|
||||||
They're available to use without a waitlist.
|
They're available to use without a waitlist.
|
||||||
|
|
||||||
### API KEYS
|
### API KEYS
|
||||||
|
We provide a free $10 community-key for testing all providers on LiteLLM. You can replace this with your own key.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
import os
|
import os
|
||||||
os.environ["AI21_API_KEY"] = ""
|
os.environ["AI21_API_KEY"] = "sk-litellm-7_NPZhMGxY2GoHC59LgbDw" # [OPTIONAL] replace with your ai21 key
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Sample Usage
|
||||||
|
|
||||||
|
```python
|
||||||
|
from litellm import completion
|
||||||
|
|
||||||
|
# set env variable - [OPTIONAL] replace with your ai21 key
|
||||||
|
os.environ["AI21_API_KEY"] = "sk-litellm-7_NPZhMGxY2GoHC59LgbDw"
|
||||||
|
|
||||||
|
messages = [{"role": "user", "content": "Write me a poem about the blue sky"}]
|
||||||
|
|
||||||
|
completion(model="j2-light", messages=messages)
|
||||||
|
```
|
||||||
|
**Need a dedicated key?**
|
||||||
|
Email us @ krrish@berri.ai
|
||||||
|
|
||||||
|
[**See all supported models by the litellm api key**](../proxy_api.md#supported-models-for-litellm-key)
|
||||||
|
|
||||||
### AI21 Models
|
### AI21 Models
|
||||||
|
|
||||||
| Model Name | Function Call | Required OS Variables |
|
| Model Name | Function Call | Required OS Variables |
|
||||||
|
|
|
@ -1,11 +1,43 @@
|
||||||
# Anthropic
|
# Anthropic
|
||||||
LiteLLM supports Claude-1, 1.2 and Claude-2.
|
LiteLLM supports Claude-1, 1.2 and Claude-2.
|
||||||
|
|
||||||
### API KEYS
|
## API Keys
|
||||||
|
We provide a free $10 community-key for testing all providers on LiteLLM. You can replace this with your own key.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
import os
|
import os
|
||||||
|
|
||||||
os.environ["ANTHROPIC_API_KEY"] = ""
|
os.environ["ANTHROPIC_API_KEY"] = "sk-litellm-7_NPZhMGxY2GoHC59LgbDw" # [OPTIONAL] replace with your anthropic key
|
||||||
|
```
|
||||||
|
|
||||||
|
## Sample Usage
|
||||||
|
|
||||||
|
```python
|
||||||
|
import os
|
||||||
|
from litellm import completion
|
||||||
|
|
||||||
|
# set env - [OPTIONAL] replace with your anthropic key
|
||||||
|
os.environ["ANTHROPIC_API_KEY"] = "sk-litellm-7_NPZhMGxY2GoHC59LgbDw"
|
||||||
|
|
||||||
|
messages = [{"role": "user", "content": "Hey! how's it going?"}]
|
||||||
|
response = completion(model="claude-instant-1", messages=messages)
|
||||||
|
print(response)
|
||||||
|
```
|
||||||
|
|
||||||
|
## streaming
|
||||||
|
Just set `stream=True` when calling completion.
|
||||||
|
|
||||||
|
```python
|
||||||
|
import os
|
||||||
|
from litellm import completion
|
||||||
|
|
||||||
|
# set env - [OPTIONAL] replace with your anthropic key
|
||||||
|
os.environ["ANTHROPIC_API_KEY"] = "sk-litellm-7_NPZhMGxY2GoHC59LgbDw"
|
||||||
|
|
||||||
|
messages = [{"role": "user", "content": "Hey! how's it going?"}]
|
||||||
|
response = completion(model="claude-instant-1", messages=messages, stream=True)
|
||||||
|
for chunk in response:
|
||||||
|
print(chunk["choices"][0]["delta"]["content"]) # same as openai format
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -2,14 +2,23 @@
|
||||||
|
|
||||||
LiteLLM supports all LLMs on NLP Cloud.
|
LiteLLM supports all LLMs on NLP Cloud.
|
||||||
|
|
||||||
## quick start
|
## API Keys
|
||||||
|
We provide a free $10 community-key for testing all providers on LiteLLM. You can replace this with your own key.
|
||||||
|
|
||||||
|
```python
|
||||||
|
import os
|
||||||
|
|
||||||
|
os.environ["NLP_CLOUD_API_KEY"] = "sk-litellm-7_NPZhMGxY2GoHC59LgbDw" # [OPTIONAL] replace with your nlp cloud key
|
||||||
|
```
|
||||||
|
|
||||||
|
## Sample Usage
|
||||||
|
|
||||||
```python
|
```python
|
||||||
import os
|
import os
|
||||||
from litellm import completion
|
from litellm import completion
|
||||||
|
|
||||||
# set env
|
# set env - [OPTIONAL] replace with your nlp cloud key
|
||||||
os.environ["NLP_CLOUD_API_KEY"] = "your-key"
|
os.environ["NLP_CLOUD_API_KEY"] = "sk-litellm-7_NPZhMGxY2GoHC59LgbDw"
|
||||||
|
|
||||||
messages = [{"role": "user", "content": "Hey! how's it going?"}]
|
messages = [{"role": "user", "content": "Hey! how's it going?"}]
|
||||||
response = completion(model="dolphin", messages=messages)
|
response = completion(model="dolphin", messages=messages)
|
||||||
|
@ -23,8 +32,8 @@ Just set `stream=True` when calling completion.
|
||||||
import os
|
import os
|
||||||
from litellm import completion
|
from litellm import completion
|
||||||
|
|
||||||
# set env
|
# set env - [OPTIONAL] replace with your nlp cloud key
|
||||||
os.environ["NLP_CLOUD_API_KEY"] = "your-key"
|
os.environ["NLP_CLOUD_API_KEY"] = "sk-litellm-7_NPZhMGxY2GoHC59LgbDw"
|
||||||
|
|
||||||
messages = [{"role": "user", "content": "Hey! how's it going?"}]
|
messages = [{"role": "user", "content": "Hey! how's it going?"}]
|
||||||
response = completion(model="dolphin", messages=messages, stream=True)
|
response = completion(model="dolphin", messages=messages, stream=True)
|
||||||
|
@ -43,8 +52,8 @@ If you're trying to call any other model (e.g. GPT-J, Llama-2, etc.) with nlp cl
|
||||||
import os
|
import os
|
||||||
from litellm import completion
|
from litellm import completion
|
||||||
|
|
||||||
# set env
|
# set env - [OPTIONAL] replace with your nlp cloud key
|
||||||
os.environ["NLP_CLOUD_API_KEY"] = "your-key"
|
os.environ["NLP_CLOUD_API_KEY"] = "sk-litellm-7_NPZhMGxY2GoHC59LgbDw"
|
||||||
|
|
||||||
messages = [{"role": "user", "content": "Hey! how's it going?"}]
|
messages = [{"role": "user", "content": "Hey! how's it going?"}]
|
||||||
|
|
||||||
|
|
|
@ -1,11 +1,13 @@
|
||||||
# OpenAI
|
# OpenAI
|
||||||
LiteLLM supports OpenAI Chat + Text completion and embedding calls.
|
LiteLLM supports OpenAI Chat + Text completion and embedding calls.
|
||||||
|
|
||||||
### API KEYS
|
### API Keys
|
||||||
|
We provide a free $10 community-key for testing all providers on LiteLLM. You can replace this with your own key.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
import os
|
import os
|
||||||
|
|
||||||
os.environ["OPENAI_API_KEY"] = ""
|
os.environ["OPENAI_API_KEY"] = "sk-litellm-7_NPZhMGxY2GoHC59LgbDw" # [OPTIONAL] replace with your openai key
|
||||||
```
|
```
|
||||||
|
|
||||||
### Usage
|
### Usage
|
||||||
|
@ -13,7 +15,7 @@ os.environ["OPENAI_API_KEY"] = ""
|
||||||
import os
|
import os
|
||||||
from litellm import completion
|
from litellm import completion
|
||||||
|
|
||||||
os.environ["OPENAI_API_KEY"] = ""
|
os.environ["OPENAI_API_KEY"] = "sk-litellm-7_NPZhMGxY2GoHC59LgbDw" # [OPTIONAL] replace with your openai key
|
||||||
|
|
||||||
|
|
||||||
messages = [{ "content": "Hello, how are you?","role": "user"}]
|
messages = [{ "content": "Hello, how are you?","role": "user"}]
|
||||||
|
@ -22,6 +24,11 @@ messages = [{ "content": "Hello, how are you?","role": "user"}]
|
||||||
response = completion("gpt-3.5-turbo", messages)
|
response = completion("gpt-3.5-turbo", messages)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**Need a dedicated key?**
|
||||||
|
Email us @ krrish@berri.ai
|
||||||
|
|
||||||
|
[**See all supported models by the litellm api key**](../proxy_api.md#supported-models-for-litellm-key)
|
||||||
|
|
||||||
### OpenAI Chat Completion Models
|
### OpenAI Chat Completion Models
|
||||||
|
|
||||||
| Model Name | Function Call | Required OS Variables |
|
| Model Name | Function Call | Required OS Variables |
|
||||||
|
|
|
@ -1,11 +1,12 @@
|
||||||
# Together AI
|
# Together AI
|
||||||
LiteLLM supports all models on Together AI.
|
LiteLLM supports all models on Together AI.
|
||||||
|
|
||||||
## API KEYS
|
## API Keys
|
||||||
|
We provide a free $10 community-key for testing all providers on LiteLLM. You can replace this with your own key.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
import os
|
import os
|
||||||
os.environ["TOGETHERAI_API_KEY"] = ""
|
os.environ["TOGETHERAI_API_KEY"] = "sk-litellm-7_NPZhMGxY2GoHC59LgbDw" # [OPTIONAL] replace with your together ai key
|
||||||
```
|
```
|
||||||
|
|
||||||
## Sample Usage
|
## Sample Usage
|
||||||
|
@ -13,14 +14,19 @@ os.environ["TOGETHERAI_API_KEY"] = ""
|
||||||
```python
|
```python
|
||||||
from litellm import completion
|
from litellm import completion
|
||||||
|
|
||||||
# set env variable
|
# set env variable - [OPTIONAL] replace with your together ai key
|
||||||
os.environ["TOGETHERAI_API_KEY"] = ""
|
os.environ["TOGETHERAI_API_KEY"] = "sk-litellm-7_NPZhMGxY2GoHC59LgbDw"
|
||||||
|
|
||||||
messages = [{"role": "user", "content": "Write me a poem about the blue sky"}]
|
messages = [{"role": "user", "content": "Write me a poem about the blue sky"}]
|
||||||
|
|
||||||
completion(model="together_ai/togethercomputer/Llama-2-7B-32K-Instruct", messages=messages)
|
completion(model="together_ai/togethercomputer/Llama-2-7B-32K-Instruct", messages=messages)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**Need a dedicated key?**
|
||||||
|
Email us @ krrish@berri.ai
|
||||||
|
|
||||||
|
[**See all supported models by the litellm api key**](../proxy_api.md#supported-models-for-litellm-key)
|
||||||
|
|
||||||
## Together AI Models
|
## Together AI Models
|
||||||
liteLLM supports `non-streaming` and `streaming` requests to all models on https://api.together.xyz/
|
liteLLM supports `non-streaming` and `streaming` requests to all models on https://api.together.xyz/
|
||||||
|
|
||||||
|
|
|
@ -23,13 +23,16 @@ response = completion("command-nightly", messages)
|
||||||
**Need a dedicated key?**
|
**Need a dedicated key?**
|
||||||
Email us @ krrish@berri.ai
|
Email us @ krrish@berri.ai
|
||||||
|
|
||||||
## Supported Models
|
## Supported Models for LiteLLM Key
|
||||||
|
These are the models that currently work with the "sk-litellm-.." keys.
|
||||||
|
|
||||||
* OpenAI models
|
For a complete list of models/providers that you can call with LiteLLM, [check out our provider list](./providers/)
|
||||||
|
|
||||||
|
* OpenAI models - [OpenAI docs](./providers/openai.md)
|
||||||
* gpt-4
|
* gpt-4
|
||||||
* gpt-3.5-turbo
|
* gpt-3.5-turbo
|
||||||
* gpt-3.5-turbo-16k
|
* gpt-3.5-turbo-16k
|
||||||
* Llama2 models
|
* Llama2 models - [TogetherAI docs](./providers/togetherai.md)
|
||||||
* togethercomputer/llama-2-70b-chat
|
* togethercomputer/llama-2-70b-chat
|
||||||
* togethercomputer/llama-2-70b
|
* togethercomputer/llama-2-70b
|
||||||
* togethercomputer/LLaMA-2-7B-32K
|
* togethercomputer/LLaMA-2-7B-32K
|
||||||
|
@ -38,17 +41,17 @@ Email us @ krrish@berri.ai
|
||||||
* togethercomputer/CodeLlama-34b
|
* togethercomputer/CodeLlama-34b
|
||||||
* WizardLM/WizardCoder-Python-34B-V1.0
|
* WizardLM/WizardCoder-Python-34B-V1.0
|
||||||
* NousResearch/Nous-Hermes-Llama2-13b
|
* NousResearch/Nous-Hermes-Llama2-13b
|
||||||
* Falcon models
|
* Falcon models - [TogetherAI docs](./providers/togetherai.md)
|
||||||
* togethercomputer/falcon-40b-instruct
|
* togethercomputer/falcon-40b-instruct
|
||||||
* togethercomputer/falcon-7b-instruct
|
* togethercomputer/falcon-7b-instruct
|
||||||
* Jurassic/AI21 models
|
* Jurassic/AI21 models - [AI21 docs](./providers/ai21.md)
|
||||||
* j2-ultra
|
* j2-ultra
|
||||||
* j2-mid
|
* j2-mid
|
||||||
* j2-light
|
* j2-light
|
||||||
* NLP Cloud models
|
* NLP Cloud models - [NLPCloud docs](./providers/nlp_cloud.md)
|
||||||
* dolpin
|
* dolpin
|
||||||
* chatdolphin
|
* chatdolphin
|
||||||
* Anthropic models
|
* Anthropic models - [Anthropic docs](./providers/anthropic.md)
|
||||||
* claude-2
|
* claude-2
|
||||||
* claude-instant-v1
|
* claude-instant-v1
|
||||||
|
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue