mirror of
https://github.com/BerriAI/litellm.git
synced 2025-04-27 11:43:54 +00:00
docs(bedrock.md): add docs on alternating user/assistant messages
This commit is contained in:
parent
0abdd3b587
commit
fbc52c87ee
1 changed files with 39 additions and 0 deletions
|
@ -577,6 +577,45 @@ for chunk in response:
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Alternate user/assistant messages
|
||||||
|
|
||||||
|
Use `user_continue_message` to add a default user message, for cases (e.g. Autogen) where the client might not follow alternating user/assistant messages starting and ending with a user message.
|
||||||
|
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
model_list:
|
||||||
|
- model_name: "bedrock-claude"
|
||||||
|
litellm_params:
|
||||||
|
model: "bedrock/anthropic.claude-instant-v1"
|
||||||
|
user_continue_message: {"role": "user", "content": "Please continue"}
|
||||||
|
```
|
||||||
|
|
||||||
|
OR
|
||||||
|
|
||||||
|
just set `litellm.modify_params=True` and LiteLLM will automatically handle this with a default user_continue_message.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
model_list:
|
||||||
|
- model_name: "bedrock-claude"
|
||||||
|
litellm_params:
|
||||||
|
model: "bedrock/anthropic.claude-instant-v1"
|
||||||
|
|
||||||
|
litellm_settings:
|
||||||
|
modify_params: true
|
||||||
|
```
|
||||||
|
|
||||||
|
Test it!
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'http://0.0.0.0:4000/chat/completions' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-H 'Authorization: Bearer sk-1234' \
|
||||||
|
-d '{
|
||||||
|
"model": "bedrock-claude",
|
||||||
|
"messages": [{"role": "assistant", "content": "Hey, how's it going?"}]
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
## Boto3 - Authentication
|
## Boto3 - Authentication
|
||||||
|
|
||||||
### Passing credentials as parameters - Completion()
|
### Passing credentials as parameters - Completion()
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue