phoenix-oss/llama-stack
1
0
Fork 0
forked from phoenix-oss/llama-stack-mirror
Code Pull requests Releases Packages 2 Activity Actions

meta reference inference fixes (#797)

Browse source 
Miscellaneous fixes for meta reference inference

Tests for log probs dont pass because meta reference does not support
top_k > 1
This commit is contained in:
Ashwin Bharambe 2025-01-16 18:17:46 -08:00 • committed by GitHub
parent cb41848a2a
commit 9f14382d82
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 20 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

6
llama_stack/providers/utils/inference/prompt_adapter.py
View file

@ -227,9 +227,11 @@ async def completion_request_to_prompt_model_input_info(
def augment_content_with_response_format_prompt(response_format, content):
if fmt_prompt := response_format_prompt(response_format):
if isinstance(content, list):
return content + [fmt_prompt]
return content + [TextContentItem(text=fmt_prompt)]
elif isinstance(content, str):
return [TextContentItem(text=content), TextContentItem(text=fmt_prompt)]
else:
return [content, fmt_prompt]
return [content, TextContentItem(text=fmt_prompt)]
return content