docs(bedrock.md): add docs on alternating user/assistant messages

This commit is contained in:
Krrish Dholakia 2024-08-22 22:10:00 -07:00
parent 0abdd3b587
commit fbc52c87ee

View file

@ -577,6 +577,45 @@ for chunk in response:
}
```
## Alternate user/assistant messages
Use `user_continue_message` to add a default user message, for cases (e.g. Autogen) where the client might not follow alternating user/assistant messages starting and ending with a user message.
```yaml
model_list:
- model_name: "bedrock-claude"
litellm_params:
model: "bedrock/anthropic.claude-instant-v1"
user_continue_message: {"role": "user", "content": "Please continue"}
```
OR
just set `litellm.modify_params=True` and LiteLLM will automatically handle this with a default user_continue_message.
```yaml
model_list:
- model_name: "bedrock-claude"
litellm_params:
model: "bedrock/anthropic.claude-instant-v1"
litellm_settings:
modify_params: true
```
Test it!
```bash
curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-d '{
"model": "bedrock-claude",
"messages": [{"role": "assistant", "content": "Hey, how's it going?"}]
}'
```
## Boto3 - Authentication
### Passing credentials as parameters - Completion()