phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-27 03:34:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Allow zero temperature for Sagemaker models based on config

Browse source 
Since Sagemaker can host any kind of model, some models allow
zero temperature. However, this is not enabled by default and
only allowed based on config
This commit is contained in:
Ravi N 2024-07-28 21:55:53 -04:00
parent cd0ec17a29
commit 5cf0667d38
1 changed files with 2 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
litellm/utils.py
View file

@ -3114,7 +3114,8 @@ def get_optional_params(
if temperature == 0.0 or temperature == 0:
# hugging face exception raised when temp==0
# Failed: Error occurred: HuggingfaceException - Input validation error: `temperature` must be strictly positive
temperature = 0.01
if not passed_params.get("aws_sagemaker_allow_zero_temp", False):
temperature = 0.01
optional_params["temperature"] = temperature
if top_p is not None:
optional_params["top_p"] = top_p