phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-28 13:01:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix: openai_compat messages system/assistant non-str content

Browse source 
When converting OpenAI message content for the "system" and
"assistant" roles to Llama Stack inference APIs (used for some
providers when dealing with Llama models via OpenAI API requests to
get proper prompt / tool handling), we were not properly converting
any non-string content.

I discovered this while running the new Responses AI verification
suite against the Fireworks provider, but instead of fixing it as part
of some ongoing work there split this out into a separate PR.

This fixes that, by using the `openai_content_to_content` helper we
used elsewhere to ensure content parts were mapped properly.

I added a couple of new tests to `test_openai_compat` to reproduce
this issue and validate its fix. I ran those as below:

```
python -m pytest -s -v tests/unit/providers/utils/inference/test_openai_compat.py
```

Signed-off-by: Ben Browning <bbrownin@redhat.com>
This commit is contained in:
Ben Browning 2025-05-02 15:31:22 -04:00
parent 272d3359ee
commit 49148bb26a
2 changed files with 54 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

9
llama_stack/providers/utils/inference/openai_compat.py
View file

@ -108,6 +108,7 @@ from llama_stack.apis.inference.inference import (
OpenAIChatCompletion,
OpenAICompletion,
OpenAICompletionChoice,
OpenAIMessageParam,
OpenAIResponseFormatParam,
ToolConfig,
)
@ -987,7 +988,7 @@ def _convert_openai_sampling_params(
def openai_messages_to_messages(
messages: list[OpenAIChatCompletionMessage],
messages: list[OpenAIMessageParam],
) -> list[Message]:
"""
Convert a list of OpenAIChatCompletionMessage into a list of Message.
@ -995,12 +996,12 @@ def openai_messages_to_messages(
converted_messages = []
for message in messages:
if message.role == "system":
converted_message = SystemMessage(content=message.content)
converted_message = SystemMessage(content=openai_content_to_content(message.content))
elif message.role == "user":
converted_message = UserMessage(content=openai_content_to_content(message.content))
elif message.role == "assistant":
converted_message = CompletionMessage(
content=message.content,
content=openai_content_to_content(message.content),
tool_calls=_convert_openai_tool_calls(message.tool_calls),
stop_reason=StopReason.end_of_turn,
)
@ -1331,7 +1332,7 @@ class OpenAIChatCompletionToLlamaStackMixin:
async def openai_chat_completion(
self,
model: str,
messages: list[OpenAIChatCompletionMessage],
messages: list[OpenAIMessageParam],
frequency_penalty: float | None = None,
function_call: str | dict[str, Any] | None = None,
functions: list[dict[str, Any]] | None = None,