phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-03 18:00:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix: Avoid BadRequestError due to invalid max_tokens (#3667)

Browse source 
This patch ensures if max tokens is not defined, then is set to None
instead of 0 when calling openai_chat_completion. This way some
providers (like gemini) that cannot handle the `max_tokens = 0` will not
fail

Issue: #3666
This commit is contained in:
Luis Tomas Bolivar 2025-10-27 17:27:21 +01:00 • committed by GitHub
parent 00d8414597
commit f18b5eb537
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
171 changed files with 12728 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

2
llama_stack/apis/inference/inference.py
View file

@ -97,7 +97,7 @@ class SamplingParams(BaseModel):
strategy: SamplingStrategy = Field(default_factory=GreedySamplingStrategy)
max_tokens: int | None = 0
max_tokens: int | None = None
repetition_penalty: float | None = 1.0
stop: list[str] | None = None