Refactored the router to accept the implementation directly instead of
using the impl_getter pattern.
The caller already knows which API it's building a router for.for
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit refactors the Batches protocol to use Pydantic request
models for both create_batch and list_batches methods, improving
consistency, readability, and maintainability.
- create_batch now accepts a single CreateBatchRequest parameter instead
of individual arguments. This aligns the protocol with FastAPI’s
request model pattern, allowing the router to pass the request object
directly without unpacking parameters. Provider implementations now
access fields via request.input_file_id, request.endpoint, etc.
- list_batches now accepts a single ListBatchesRequest parameter,
replacing individual query parameters. The model includes after and
limit fields with proper OpenAPI descriptions. FastAPI automatically
parses query parameters into the model for GET requests, keeping
router code clean. Provider implementations access fields via
request.after and request.limit.
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit migrates the Batches API to use FastAPI routers directly in the
API package, removing the need for custom decorator systems and manual router
registration. The API package now defines FastAPI routers using standard
FastAPI route decorators, making it self-sufficient and eliminating dependencies
on the server package.
The router implementation has been moved from llama_stack/core/server/routers/batches.py
to llama_stack_api/batches/routes.py, where it belongs alongside the protocol
and models.
Standard error responses (standard_responses) have been moved from the server
package to llama_stack_api/router_utils.py, ensuring the API package can
define complete routers without server dependencies. FastAPI has been added
as an explicit dependency to the llama-stack-api package, making it an
intentional dependency rather than an implicit one.
Router discovery is now fully automatic. The server discovers routers by
checking for routes modules in each API package and looking for a create_router
function. This eliminates the need for manual registration and makes the system
scalable - new APIs with router modules are automatically discovered and used.
The router registry has been simplified to use automatic discovery instead of
maintaining a manual registry. The build_router function (renamed from
create_router to better reflect its purpose) discovers and combines router
factories with implementations to create the final router instances.
Exposing Routers from the API is nice for the Bring Your Own API use
case too.
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit introduces a new FastAPI router-based system for defining
API endpoints, enabling a migration path away from the legacy @webmethod
decorator system. The implementation includes router infrastructure,
migration of the Batches API as the first example, and updates
to server, OpenAPI generation, and inspection systems to
support both routing approaches.
The router infrastructure consists of a router registry system
that allows APIs to register FastAPI router factories, which
are then automatically discovered and included in the server
application. Standard error responses are centralized in
router_utils to ensure consistent OpenAPI specification
generation with proper $ref references to component responses.
The Batches API has been migrated to demonstrate the new
pattern. The protocol definition and models remain in
llama_stack_api/batches, maintaining clear separation between
API contracts and server implementation. The FastAPI router
implementation lives in
llama_stack/core/server/routers/batches, following the
established pattern where API contracts are defined in
llama_stack_api and server routing logic lives in
llama_stack/core/server.
The server now checks for registered routers before falling
back to the legacy webmethod-based route discovery, ensuring
backward compatibility during the migration period. The
OpenAPI generator has been updated to handle both router-based
and webmethod-based routes, correctly extracting metadata from
FastAPI route decorators and Pydantic Field descriptions. The
inspect endpoint now includes routes from both systems, with
proper filtering for deprecated routes and API levels.
Response descriptions are now explicitly defined in router decorators,
ensuring the generated OpenAPI specification matches the
previous format. Error responses use $ref references to
component responses (BadRequest400, TooManyRequests429, etc.)
as required by the specification. This is neat and will allow us to
remove a lot of boiler plate code from our generator once the
migration is done.
This implementation provides a foundation for incrementally migrating
other APIs to the router system while maintaining full backward
compatibility with existing webmethod-based APIs.
Closes: https://github.com/llamastack/llama-stack/issues/4188
Signed-off-by: Sébastien Han <seb@redhat.com>
# What does this PR do?
It was referencing strong_typing which was removed in
https://github.com/llamastack/llama-stack/pull/3944
## Test Plan
New CI build test.
Signed-off-by: Sébastien Han <seb@redhat.com>
# What does this PR do?
This replaces the legacy "pyopenapi + strong_typing" pipeline with a
FastAPI-backed generator that has an explicit schema registry inside
`llama_stack_api`. The key changes:
1. **New generator architecture.** FastAPI now builds the OpenAPI schema
directly from the real routes, while helper modules
(`schema_collection`, `endpoints`, `schema_transforms`, etc.)
post-process the result. The old pyopenapi stack and its strong_typing
helpers are removed entirely, so we no longer rely on fragile AST
analysis or top-level import side effects.
2. **Schema registry in `llama_stack_api`.** `schema_utils.py` keeps a
`SchemaInfo` record for every `@json_schema_type`, `register_schema`,
and dynamically created request model. The OpenAPI generator and other
tooling query this registry instead of scanning the package tree,
producing deterministic names (e.g., `{MethodName}Request`), capturing
all optional/nullable fields, and making schema discovery testable. A
new unit test covers the registry behavior.
3. **Regenerated specs + CI alignment.** All docs/Stainless specs are
regenerated from the new pipeline, so optional/nullable fields now match
reality (expect the API Conformance workflow to report breaking
changes—this PR establishes the new baseline). The workflow itself is
back to the stock oasdiff invocation so future regressions surface
normally.
*Conformance will be RED on this PR; we choose to accept the
deviations.*
## Test Plan
- `uv run pytest tests/unit/server/test_schema_registry.py`
- `uv run python -m scripts.openapi_generator.main docs/static`
---------
Signed-off-by: Sébastien Han <seb@redhat.com>
Co-authored-by: Ashwin Bharambe <ashwin.bharambe@gmail.com>
# What does this PR do?
Adding a user-facing `authorization ` parameter to MCP tool definitions
that allows users to explicitly configure credentials per MCP server,
addressing GitHub Issue #4034 in a secure manner.
## Test Plan
tests/integration/responses/test_mcp_authentication.py
---------
Co-authored-by: Omar Abdelwahab <omara@fb.com>
Co-authored-by: Ashwin Bharambe <ashwin.bharambe@gmail.com>
# What does this PR do?
the directory structure was src/llama-stack-api/llama_stack_api
instead it should just be src/llama_stack_api to match the other
packages.
update the structure and pyproject/linting config
---------
Signed-off-by: Charlie Doern <cdoern@redhat.com>
Co-authored-by: Ashwin Bharambe <ashwin.bharambe@gmail.com>
