litellm/docs/my-website/docs/providers/bedrock.md

AWS Bedrock

API KEYS

os.environ["AWS_ACCESS_KEY_ID"] = ""
os.environ["AWS_SECRET_ACCESS_KEY"] = ""
os.environ["AWS_REGION_NAME"] = ""

Usage

import os 
from litellm import completion

os.environ["AWS_ACCESS_KEY_ID"] = ""
os.environ["AWS_SECRET_ACCESS_KEY"] = ""
os.environ["AWS_REGION_NAME"] = ""

response = completion(
            model="bedrock/amazon.titan-tg1-large", 
            messages=[{ "content": "Hello, how are you?","role": "user"}],
            temperature=0.2,
            max_tokens=80,
)

Supported AWS Bedrock Models

Here's an example of using a bedrock model with LiteLLM

Model Name	Function Call	Required OS Variables
Titan Text Large	completion(model='bedrock/amazon.titan-tg1-large', messages=messages)	os.environ['AWS_ACCESS_KEY_ID'], os.environ['AWS_SECRET_ACCESS_KEY'], os.environ['AWS_REGION_NAME']
AI21 J2-Ultra	completion(model='bedrock/ai21.j2-ultra', messages=messages)	os.environ['AWS_ACCESS_KEY_ID'], os.environ['AWS_SECRET_ACCESS_KEY'], os.environ['AWS_REGION_NAME']
AI21 J2-Mid	completion(model='bedrock/ai21.j2-mid', messages=messages)	os.environ['AWS_ACCESS_KEY_ID'], os.environ['AWS_SECRET_ACCESS_KEY'], os.environ['AWS_REGION_NAME']

Troubleshooting

If creating a boto3 bedrock client fails litellm will retry pip installing boto3 on your system

pip install https://github.com/BerriAI/litellm/raw/main/cookbook/bedrock_resources/boto3-1.28.21-py3-none-any.whl
pip install https://github.com/BerriAI/litellm/raw/main/cookbook/bedrock_resources/botocore-1.31.21-py3-none-any.whl