phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-03 09:53:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix: Avoid BadRequestError due to invalid max_tokens (#3667)

Browse source 
This patch ensures if max tokens is not defined, then is set to None
instead of 0 when calling openai_chat_completion. This way some
providers (like gemini) that cannot handle the `max_tokens = 0` will not
fail

Issue: #3666
This commit is contained in:
Luis Tomas Bolivar 2025-10-27 17:27:21 +01:00 • committed by GitHub
parent 00d8414597
commit f18b5eb537
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
171 changed files with 12728 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

1
docs/static/experimental-llama-stack-spec.yaml vendored
View file

@ -1695,7 +1695,6 @@ components:
description: The sampling strategy.
max_tokens:
type: integer
default: 0
description: >-
The maximum number of tokens that can be generated in the completion.
The token count of your prompt plus max_tokens cannot exceed the model's