phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

Merge pull request #2701 from rmann-nflx/main

Browse source 
Updating the default Claude3 max tokens
This commit is contained in:
Krish Dholakia 2024-03-27 10:14:20 -07:00 • committed by GitHub
commit d259c754ef
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 5 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

8
litellm/llms/bedrock.py
View file

@ -79,11 +79,13 @@ class AmazonTitanConfig:
class AmazonAnthropicClaude3Config: class AmazonAnthropicClaude3Config:
""" """
Reference: https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/providers?model=claude Reference:
https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/providers?model=claude
https://docs.anthropic.com/claude/docs/models-overview#model-comparison
Supported Params for the Amazon / Anthropic Claude 3 models: Supported Params for the Amazon / Anthropic Claude 3 models:
- `max_tokens` Required (integer) max tokens, - `max_tokens` Required (integer) max tokens. Default is 4096
- `anthropic_version` Required (string) version of anthropic for bedrock - e.g. "bedrock-2023-05-31" - `anthropic_version` Required (string) version of anthropic for bedrock - e.g. "bedrock-2023-05-31"
- `system` Optional (string) the system prompt, conversion from openai format to this is handled in factory.py - `system` Optional (string) the system prompt, conversion from openai format to this is handled in factory.py
- `temperature` Optional (float) The amount of randomness injected into the response - `temperature` Optional (float) The amount of randomness injected into the response
@ -92,7 +94,7 @@ class AmazonAnthropicClaude3Config:
- `stop_sequences` Optional (List[str]) Custom text sequences that cause the model to stop generating - `stop_sequences` Optional (List[str]) Custom text sequences that cause the model to stop generating
""" """
max_tokens: Optional[int] = litellm.max_tokens max_tokens: Optional[int] = 4096 # Opus, Sonnet, and Haiku default
anthropic_version: Optional[str] = "bedrock-2023-05-31" anthropic_version: Optional[str] = "bedrock-2023-05-31"
system: Optional[str] = None system: Optional[str] = None
temperature: Optional[float] = None temperature: Optional[float] = None