phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-10-04 20:14:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

chore: handle missing finish_reason

Browse source 
# What does this PR do?
Sometimes the stream don't have chunks with finish_reason, e.g. canceled stream, which throws a pydantic error as OpenAIChoice.finish_reason: str

## Test Plan
observe no more such error when benchmarking
This commit is contained in:
Eric Huang 2025-09-03 21:54:45 -07:00
parent 02f6e0f531
commit df873b8b27
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
llama_stack/core/routers/inference.py
View file

@ -755,7 +755,7 @@ class InferenceRouter(Inference):
choices_data[idx] = { choices_data[idx] = {
"content_parts": [], "content_parts": [],
"tool_calls_builder": {}, "tool_calls_builder": {},
"finish_reason": None, "finish_reason": "stop",
"logprobs_content_parts": [], "logprobs_content_parts": [],
} }
current_choice_data = choices_data[idx] current_choice_data = choices_data[idx]