OpenAI completion prompt can also be an array

The OpenAI completion prompt field can be a string or an array, so
update things to use and pass that properly.

This also stubs in a basic conversion of OpenAI non-streaming
completion requests to Llama Stack completion calls, for those
providers that don't actually have an OpenAI backend to allow them to
still accept requests via the OpenAI APIs.

Signed-off-by: Ben Browning <bbrownin@redhat.com>
This commit is contained in:
Ben Browning 2025-04-09 09:28:50 -04:00
parent 24cfa1ef1a
commit a6cf8fa12b
10 changed files with 95 additions and 12 deletions

View file

@ -780,7 +780,7 @@ class Inference(Protocol):
async def openai_completion(
self,
model: str,
prompt: str,
prompt: Union[str, List[str]],
best_of: Optional[int] = None,
echo: Optional[bool] = None,
frequency_penalty: Optional[float] = None,