phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-25 18:54:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Allow zero temperature for Sagemaker models based on config

Browse source 
Since Sagemaker can host any kind of model, some models allow
zero temperature. However, this is not enabled by default and
only allowed based on config
This commit is contained in:
Ravi N 2024-07-28 21:55:53 -04:00
parent c0717133a9
commit b321f2988b
1 changed files with 2 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
litellm/utils.py
View file

@ -3114,7 +3114,8 @@ def get_optional_params(
if temperature == 0.0 or temperature == 0:
# hugging face exception raised when temp==0
# Failed: Error occurred: HuggingfaceException - Input validation error: `temperature` must be strictly positive
temperature = 0.01
if not passed_params.get("aws_sagemaker_allow_zero_temp", False):
temperature = 0.01
optional_params["temperature"] = temperature
if top_p is not None:
optional_params["top_p"] = top_p