forked from phoenix/litellm-mirror
bedrock docs
This commit is contained in:
parent
4b61bcee6c
commit
0c96b61fce
1 changed files with 35 additions and 0 deletions
35
docs/my-website/docs/providers/bedrock.md
Normal file
35
docs/my-website/docs/providers/bedrock.md
Normal file
|
@ -0,0 +1,35 @@
|
||||||
|
# AWS Bedrock
|
||||||
|
|
||||||
|
### API KEYS
|
||||||
|
```python
|
||||||
|
!pip install boto3
|
||||||
|
|
||||||
|
os.environ["AWS_ACCESS_KEY_ID"] = ""
|
||||||
|
os.environ["AWS_SECRET_ACCESS_KEY"] = ""
|
||||||
|
os.environ["AWS_REGION_NAME"] = ""
|
||||||
|
```
|
||||||
|
|
||||||
|
### Usage
|
||||||
|
```python
|
||||||
|
import os
|
||||||
|
from litellm import completion
|
||||||
|
|
||||||
|
os.environ["AWS_ACCESS_KEY_ID"] = ""
|
||||||
|
os.environ["AWS_SECRET_ACCESS_KEY"] = ""
|
||||||
|
os.environ["AWS_REGION_NAME"] = ""
|
||||||
|
|
||||||
|
response = completion(
|
||||||
|
model="bedrock/amazon.titan-tg1-large",
|
||||||
|
messages=[{ "content": "Hello, how are you?","role": "user"}],
|
||||||
|
temperature=0.2,
|
||||||
|
max_tokens=80,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Supported AWS Bedrock Models
|
||||||
|
Here's an example of using a bedrock model with LiteLLM
|
||||||
|
|
||||||
|
| Model Name | Function Call | Required OS Variables |
|
||||||
|
|------------------|--------------------------------------------|------------------------------------|
|
||||||
|
| Llama2 7B | `completion(model='sagemaker/jumpstart-dft-meta-textgeneration-llama-2-7b, messages=messages)` | `os.environ['AWS_ACCESS_KEY_ID']`, `os.environ['AWS_SECRET_ACCESS_KEY']`, `os.environ['AWS_REGION_NAME']` |
|
||||||
|
| Custom LLM Endpoint | `completion(model='sagemaker/your-endpoint, messages=messages)` | `os.environ['AWS_ACCESS_KEY_ID']`, `os.environ['AWS_SECRET_ACCESS_KEY']`, `os.environ['AWS_REGION_NAME']` |
|
Loading…
Add table
Add a link
Reference in a new issue