mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-12-03 01:48:05 +00:00
b1851dc9ef
Add support for reasoning fields in OpenAI-compatible chat completion messages to enable compatibility with vLLM reasoning parsers. Changes: - Add `reasoning_content` and `reasoning` fields to OpenAIAssistantMessageParam - Add `reasoning` field to OpenAIChoiceDelta (reasoning_content already existed) Both field names are supported for maximum compatibility: - `reasoning_content`: Used by vLLM ≤ v0.8.4 - `reasoning`: New field name in vLLM ≥ v0.9.x vLLM documentation recommends migrating to the shorter `reasoning` field name, but maintains backward compatibility with `reasoning_content`. These fields allow reasoning models to return their chain-of-thought process alongside the final answer, which is crucial for transparency and debugging with reasoning models. References: - vLLM Reasoning Outputs: https://docs.vllm.ai/en/stable/features/reasoning_outputs/ - vLLM Issue #12468: https://github.com/vllm-project/vllm/issues/12468 Signed-off-by: Charlie Doern <cdoern@redhat.com> |
||
|---|---|---|
| .. | ||
| config.yml | ||
| openapi.yml | ||
| README.md | ||
These are the source-of-truth configuration files used to generate the Stainless client SDKs via Stainless.
openapi.yml: this is the OpenAPI specification for the Llama Stack API.config.yml: this is the Stainless configuration which instructs Stainless how to generate the client SDKs.
A small side note: notice the .yml suffixes since Stainless uses that suffix typically for its configuration files.
These files go hand-in-hand. Both openapi.yml and config.yml are generated by scripts/run_openapi_generator.sh:
openapi.ymlcomes from the FastAPI-based generator.config.ymlis rendered fromscripts/openapi_generator/stainless_config/config_data.pyso the Stainless config stays in lock-step with the spec.