phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-03 09:53:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix: Avoid BadRequestError due to invalid max_tokens (#3667)

Browse source 
This patch ensures if max tokens is not defined, then is set to None
instead of 0 when calling openai_chat_completion. This way some
providers (like gemini) that cannot handle the `max_tokens = 0` will not
fail

Issue: #3666
This commit is contained in:
Luis Tomas Bolivar 2025-10-27 17:27:21 +01:00 • committed by GitHub
parent 00d8414597
commit f18b5eb537
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
171 changed files with 12728 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

2
tests/integration/eval/test_eval.py
View file

@ -55,6 +55,7 @@ def test_evaluate_rows(llama_stack_client, text_model_id, scoring_fn_id):
"model": text_model_id,
"sampling_params": {
"temperature": 0.0,
"max_tokens": 512,
},
},
},
@ -88,6 +89,7 @@ def test_evaluate_benchmark(llama_stack_client, text_model_id, scoring_fn_id):
"model": text_model_id,
"sampling_params": {
"temperature": 0.0,
"max_tokens": 512,
},
},
},