mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-12-03 01:48:05 +00:00
eb3cab1eec
This commit introduces a new FastAPI router-based system for defining API endpoints, enabling a migration path away from the legacy @webmethod decorator system. The implementation includes router infrastructure, migration of the Batches API as the first example, and updates to server, OpenAPI generation, and inspection systems to support both routing approaches. The router infrastructure consists of a router registry system that allows APIs to register FastAPI router factories, which are then automatically discovered and included in the server application. Standard error responses are centralized in router_utils to ensure consistent OpenAPI specification generation with proper $ref references to component responses. The Batches API has been migrated to demonstrate the new pattern. The protocol definition and models remain in llama_stack_api/batches, maintaining clear separation between API contracts and server implementation. The FastAPI router implementation lives in llama_stack/core/server/routers/batches, following the established pattern where API contracts are defined in llama_stack_api and server routing logic lives in llama_stack/core/server. The server now checks for registered routers before falling back to the legacy webmethod-based route discovery, ensuring backward compatibility during the migration period. The OpenAPI generator has been updated to handle both router-based and webmethod-based routes, correctly extracting metadata from FastAPI route decorators and Pydantic Field descriptions. The inspect endpoint now includes routes from both systems, with proper filtering for deprecated routes and API levels. Response descriptions are now explicitly defined in router decorators, ensuring the generated OpenAPI specification matches the previous format. Error responses use $ref references to component responses (BadRequest400, TooManyRequests429, etc.) as required by the specification. This is neat and will allow us to remove a lot of boiler plate code from our generator once the migration is done. This implementation provides a foundation for incrementally migrating other APIs to the router system while maintaining full backward compatibility with existing webmethod-based APIs. Closes: https://github.com/llamastack/llama-stack/issues/4188 Signed-off-by: Sébastien Han <seb@redhat.com>
37 lines
1.4 KiB
Python
37 lines
1.4 KiB
Python
# Copyright (c) Meta Platforms, Inc. and affiliates.
|
|
# All rights reserved.
|
|
#
|
|
# This source code is licensed under the terms described in the LICENSE file in
|
|
# the root directory of this source tree.
|
|
|
|
"""Pydantic models for Batches API requests and responses.
|
|
|
|
This module defines the request and response models for the Batches API
|
|
using Pydantic with Field descriptions for OpenAPI schema generation.
|
|
"""
|
|
|
|
from typing import Literal
|
|
|
|
from pydantic import BaseModel, Field
|
|
|
|
from llama_stack_api.batches import BatchObject, ListBatchesResponse
|
|
from llama_stack_api.schema_utils import json_schema_type
|
|
|
|
|
|
@json_schema_type
|
|
class CreateBatchRequest(BaseModel):
|
|
"""Request model for creating a batch."""
|
|
|
|
input_file_id: str = Field(..., description="The ID of an uploaded file containing requests for the batch.")
|
|
endpoint: str = Field(..., description="The endpoint to be used for all requests in the batch.")
|
|
completion_window: Literal["24h"] = Field(
|
|
..., description="The time window within which the batch should be processed."
|
|
)
|
|
metadata: dict[str, str] | None = Field(default=None, description="Optional metadata for the batch.")
|
|
idempotency_key: str | None = Field(
|
|
default=None, description="Optional idempotency key. When provided, enables idempotent behavior."
|
|
)
|
|
|
|
|
|
# Re-export response models for convenience
|
|
__all__ = ["CreateBatchRequest", "BatchObject", "ListBatchesResponse"]
|