diff --git a/docs/my-website/docs/providers/bedrock.md b/docs/my-website/docs/providers/bedrock.md index 608bc9d1f..ffe258e97 100644 --- a/docs/my-website/docs/providers/bedrock.md +++ b/docs/my-website/docs/providers/bedrock.md @@ -361,47 +361,6 @@ response = completion( ) ``` -### Passing an external BedrockRuntime.Client as a parameter - Completion() -Pass an external BedrockRuntime.Client object as a parameter to litellm.completion. Useful when using an AWS credentials profile, SSO session, assumed role session, or if environment variables are not available for auth. - -Create a client from session credentials: -```python -import boto3 -from litellm import completion - -bedrock = boto3.client( - service_name="bedrock-runtime", - region_name="us-east-1", - aws_access_key_id="", - aws_secret_access_key="", - aws_session_token="", -) - -response = completion( - model="bedrock/anthropic.claude-instant-v1", - messages=[{ "content": "Hello, how are you?","role": "user"}], - aws_bedrock_client=bedrock, -) -``` - -Create a client from AWS profile in `~/.aws/config`: -```python -import boto3 -from litellm import completion - -dev_session = boto3.Session(profile_name="dev-profile") -bedrock = dev_session.client( - service_name="bedrock-runtime", - region_name="us-east-1", -) - -response = completion( - model="bedrock/anthropic.claude-instant-v1", - messages=[{ "content": "Hello, how are you?","role": "user"}], - aws_bedrock_client=bedrock, -) -``` - ### SSO Login (AWS Profile) - Set `AWS_PROFILE` environment variable - Make bedrock completion call @@ -464,6 +423,56 @@ response = completion( ) ``` + +### Passing an external BedrockRuntime.Client as a parameter - Completion() + +:::warning + +This is a deprecated flow. Boto3 is not async. And boto3.client does not let us make the http call through httpx. Pass in your aws params through the method above 👆. [See Auth Code](https://github.com/BerriAI/litellm/blob/55a20c7cce99a93d36a82bf3ae90ba3baf9a7f89/litellm/llms/bedrock_httpx.py#L284) [Add new auth flow](https://github.com/BerriAI/litellm/issues) + +::: + +Pass an external BedrockRuntime.Client object as a parameter to litellm.completion. Useful when using an AWS credentials profile, SSO session, assumed role session, or if environment variables are not available for auth. + +Create a client from session credentials: +```python +import boto3 +from litellm import completion + +bedrock = boto3.client( + service_name="bedrock-runtime", + region_name="us-east-1", + aws_access_key_id="", + aws_secret_access_key="", + aws_session_token="", +) + +response = completion( + model="bedrock/anthropic.claude-instant-v1", + messages=[{ "content": "Hello, how are you?","role": "user"}], + aws_bedrock_client=bedrock, +) +``` + +Create a client from AWS profile in `~/.aws/config`: +```python +import boto3 +from litellm import completion + +dev_session = boto3.Session(profile_name="dev-profile") +bedrock = dev_session.client( + service_name="bedrock-runtime", + region_name="us-east-1", +) + +response = completion( + model="bedrock/anthropic.claude-instant-v1", + messages=[{ "content": "Hello, how are you?","role": "user"}], + aws_bedrock_client=bedrock, +) +``` + + ## Provisioned throughput models To use provisioned throughput Bedrock models pass - `model=bedrock/`, example `model=bedrock/anthropic.claude-v2`. Set `model` to any of the [Supported AWS models](#supported-aws-bedrock-models)