phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-10-04 12:07:34 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat: introduce API leveling, post_training to v1alpha

Browse source 
Rather than have a single `LLAMA_STACK_VERSION`, we need to have a `_V1`, `_V1ALPHA`, and `_V1BETA` constant.

This also necessitated addition of `level` to the `WebMethod` so that routing can be handeled properly.

For backwards compat, the `v1` routes are being kept around and marked as `deprecated`. When used, the server will log a deprecation warning.

move:

post_training to v1alpha as it is under heavy development and not near its final state
eval: job scheduling is not implemented. Relies heavily on the datasetio API which is under development missing implementations of specific routes indicating the structure of those routes might change. Additionally eval depends on the inference API which is going to be deprecated, eval will likely need a major API surface change to conform to using completions properly

Signed-off-by: Charlie Doern <cdoern@redhat.com>
This commit is contained in:
Charlie Doern 2025-09-12 13:23:57 -04:00
parent a50b63906c
commit 03399cebf3
35 changed files with 1507 additions and 260 deletions
Download patch file Download diff file Expand all files Collapse all files

5
llama_stack/apis/batch_inference/batch_inference.py
View file

@ -17,6 +17,7 @@ from llama_stack.apis.inference import (
ToolDefinition,
ToolPromptFormat,
)
from llama_stack.apis.version import LLAMA_STACK_API_V1
from llama_stack.schema_utils import webmethod
@ -30,7 +31,7 @@ class BatchInference(Protocol):
including (post-training, evals, etc).
"""
@webmethod(route="/batch-inference/completion", method="POST")
@webmethod(route="/batch-inference/completion", method="POST", level=LLAMA_STACK_API_V1)
async def completion(
self,
model: str,
@ -50,7 +51,7 @@ class BatchInference(Protocol):
"""
...
@webmethod(route="/batch-inference/chat-completion", method="POST")
@webmethod(route="/batch-inference/chat-completion", method="POST", level=LLAMA_STACK_API_V1)
async def chat_completion(
self,
model: str,