From fbc52c87eeef7bf146e9c6880c891fe688c59cb0 Mon Sep 17 00:00:00 2001 From: Krrish Dholakia Date: Thu, 22 Aug 2024 22:10:00 -0700 Subject: [PATCH] docs(bedrock.md): add docs on alternating user/assistant messages --- docs/my-website/docs/providers/bedrock.md | 39 +++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/docs/my-website/docs/providers/bedrock.md b/docs/my-website/docs/providers/bedrock.md index a1a056d415..bbb5e9ea73 100644 --- a/docs/my-website/docs/providers/bedrock.md +++ b/docs/my-website/docs/providers/bedrock.md @@ -577,6 +577,45 @@ for chunk in response: } ``` +## Alternate user/assistant messages + +Use `user_continue_message` to add a default user message, for cases (e.g. Autogen) where the client might not follow alternating user/assistant messages starting and ending with a user message. + + +```yaml +model_list: + - model_name: "bedrock-claude" + litellm_params: + model: "bedrock/anthropic.claude-instant-v1" + user_continue_message: {"role": "user", "content": "Please continue"} +``` + +OR + +just set `litellm.modify_params=True` and LiteLLM will automatically handle this with a default user_continue_message. + +```yaml +model_list: + - model_name: "bedrock-claude" + litellm_params: + model: "bedrock/anthropic.claude-instant-v1" + +litellm_settings: + modify_params: true +``` + +Test it! + +```bash +curl -X POST 'http://0.0.0.0:4000/chat/completions' \ +-H 'Content-Type: application/json' \ +-H 'Authorization: Bearer sk-1234' \ +-d '{ + "model": "bedrock-claude", + "messages": [{"role": "assistant", "content": "Hey, how's it going?"}] +}' +``` + ## Boto3 - Authentication ### Passing credentials as parameters - Completion()