forked from phoenix/litellm-mirror
Add/fix samples for Claude pre-fill and system messages
This commit is contained in:
parent
5d074f5b56
commit
767679ffcf
2 changed files with 51 additions and 2 deletions
|
@ -50,6 +50,44 @@ response = completion(model="claude-2.1", messages=messages)
|
||||||
print(response)
|
print(response)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Example prompt sent to Claude
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
Human: How do you say 'Hello' in German? Return your answer as a JSON object, like this:
|
||||||
|
|
||||||
|
{ "Hello": "Hallo" }
|
||||||
|
|
||||||
|
Assistant: {
|
||||||
|
```
|
||||||
|
|
||||||
|
## Usage - "System" messages
|
||||||
|
If you're using Anthropic's Claude 2.1 with Bedrock, `system` role messages are properly formatted for you.
|
||||||
|
|
||||||
|
```python
|
||||||
|
import os
|
||||||
|
from litellm import completion
|
||||||
|
|
||||||
|
# set env - [OPTIONAL] replace with your anthropic key
|
||||||
|
os.environ["ANTHROPIC_API_KEY"] = "your-api-key"
|
||||||
|
|
||||||
|
messages = [
|
||||||
|
{"role": "system", "content": "You are a snarky assistant."},
|
||||||
|
{"role": "user", "content": "How do I boil water?"},
|
||||||
|
]
|
||||||
|
response = completion(model="claude-2.1", messages=messages)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Example prompt sent to Claude
|
||||||
|
|
||||||
|
```
|
||||||
|
You are a snarky assistant.
|
||||||
|
|
||||||
|
Human: How do I boil water?
|
||||||
|
|
||||||
|
Assistant:
|
||||||
|
```
|
||||||
|
|
||||||
## Streaming
|
## Streaming
|
||||||
Just set `stream=True` when calling completion.
|
Just set `stream=True` when calling completion.
|
||||||
|
|
||||||
|
|
|
@ -56,6 +56,17 @@ messages = [
|
||||||
response = completion(model="anthropic.claude-v2", messages=messages)
|
response = completion(model="anthropic.claude-v2", messages=messages)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Example prompt sent to Claude
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
Human: How do you say 'Hello' in German? Return your answer as a JSON object, like this:
|
||||||
|
|
||||||
|
{ "Hello": "Hallo" }
|
||||||
|
|
||||||
|
Assistant: {
|
||||||
|
```
|
||||||
|
|
||||||
## Usage - "System" messages
|
## Usage - "System" messages
|
||||||
If you're using Anthropic's Claude 2.1 with Bedrock, `system` role messages are properly formatted for you.
|
If you're using Anthropic's Claude 2.1 with Bedrock, `system` role messages are properly formatted for you.
|
||||||
|
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue