phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-15 07:12:37 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge

Browse source 
Signed-off-by: Bill Murdock <bmurdock@redhat.com>
This commit is contained in:
Bill Murdock 2025-10-06 16:19:57 -04:00
commit e77b7a127c
854 changed files with 165238 additions and 99099 deletions
Download patch file Download diff file Expand all files Collapse all files

20
docs/docs/building_applications/telemetry.mdx
View file

@ -187,21 +187,21 @@ Configure telemetry behavior using environment variables:
- **`OTEL_SERVICE_NAME`**: Service name for telemetry (default: empty string)
- **`TELEMETRY_SINKS`**: Comma-separated list of sinks (default: `console,sqlite`)
## Visualization with Jaeger
### Quick Setup: Complete Telemetry Stack
The `otel_trace` sink works with any service compatible with the OpenTelemetry collector. Traces and metrics use separate endpoints but can share the same collector.
### Starting Jaeger
Start a Jaeger instance with OTLP HTTP endpoint at 4318 and the Jaeger UI at 16686:
Use the automated setup script to launch the complete telemetry stack (Jaeger, OpenTelemetry Collector, Prometheus, and Grafana):
```bash
docker run --pull always --rm --name jaeger \
-p 16686:16686 -p 4318:4318 \
jaegertracing/jaeger:2.1.0
./scripts/telemetry/setup_telemetry.sh
```
Once running, you can visualize traces by navigating to [http://localhost:16686/](http://localhost:16686/).
This sets up:
- **Jaeger UI**: http://localhost:16686 (traces visualization)
- **Prometheus**: http://localhost:9090 (metrics)
- **Grafana**: http://localhost:3000 (dashboards with auto-configured data sources)
- **OTEL Collector**: http://localhost:4318 (OTLP endpoint)
Once running, you can visualize traces by navigating to [Grafana](http://localhost:3000/) and login with login `admin` and password `admin`.
## Querying Metrics

2
docs/docs/concepts/apis/external.mdx
View file

@ -357,7 +357,7 @@ server:
8. Run the server:
```bash
python -m llama_stack.core.server.server --yaml-config ~/.llama/run-byoa.yaml
llama stack run ~/.llama/run-byoa.yaml
```
9. Test the API:

2
docs/docs/deploying/kubernetes_deployment.mdx
View file

@ -170,7 +170,7 @@ spec:
- name: llama-stack
image: localhost/llama-stack-run-k8s:latest
imagePullPolicy: IfNotPresent
command: ["python", "-m", "llama_stack.core.server.server", "--config", "/app/config.yaml"]
command: ["llama", "stack", "run", "/app/config.yaml"]
ports:
- containerPort: 5000
volumeMounts:

2
docs/docs/distributions/k8s/stack-k8s.yaml.template
View file

@ -52,7 +52,7 @@ spec:
value: "${SAFETY_MODEL}"
- name: TAVILY_SEARCH_API_KEY
value: "${TAVILY_SEARCH_API_KEY}"
command: ["python", "-m", "llama_stack.core.server.server", "/etc/config/stack_run_config.yaml", "--port", "8321"]
command: ["llama", "stack", "run", "/etc/config/stack_run_config.yaml", "--port", "8321"]
ports:
- containerPort: 8321
volumeMounts:

7
docs/docs/providers/files/index.mdx
View file

@ -1,4 +1,7 @@
---
description: "Files
This API is used to upload documents that can be used with other Llama Stack APIs."
sidebar_label: Files
title: Files
---
@ -7,4 +10,8 @@ title: Files
## Overview
Files
This API is used to upload documents that can be used with other Llama Stack APIs.
This section contains documentation for all available providers for the **files** API.

8
docs/docs/providers/inference/index.mdx
View file

@ -1,5 +1,7 @@
---
description: "Llama Stack Inference API for generating completions, chat completions, and embeddings.
description: "Inference
Llama Stack Inference API for generating completions, chat completions, and embeddings.
This API provides the raw interface to the underlying models. Two kinds of models are supported:
- LLM models: these models generate \"raw\" and \"chat\" (conversational) completions.
@ -12,7 +14,9 @@ title: Inference
## Overview
Llama Stack Inference API for generating completions, chat completions, and embeddings.
Inference
Llama Stack Inference API for generating completions, chat completions, and embeddings.
This API provides the raw interface to the underlying models. Two kinds of models are supported:
- LLM models: these models generate "raw" and "chat" (conversational) completions.

2
docs/docs/providers/inference/remote_databricks.mdx
View file

@ -15,7 +15,7 @@ Databricks inference provider for running models on Databricks' unified analytic
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `allowed_models` | `list[str \| None` | No | | List of models that should be registered with the model registry. If None, all models are allowed. |
| `url` | `<class 'str'>` | No | | The URL for the Databricks model serving endpoint |
| `url` | `str \| None` | No | | The URL for the Databricks model serving endpoint |
| `api_token` | `<class 'pydantic.types.SecretStr'>` | No | | The Databricks API token |
## Sample Configuration

7
docs/docs/providers/safety/index.mdx
View file

@ -1,4 +1,7 @@
---
description: "Safety
OpenAI-compatible Moderations API."
sidebar_label: Safety
title: Safety
---
@ -7,4 +10,8 @@ title: Safety
## Overview
Safety
OpenAI-compatible Moderations API.
This section contains documentation for all available providers for the **safety** API.

23
docs/openapi_generator/pyopenapi/generator.py
View file

@ -50,6 +50,7 @@ from .specification import (
Document,
Example,
ExampleRef,
ExtraBodyParameter,
MediaType,
Operation,
Parameter,
@ -677,6 +678,27 @@ class Generator:
# parameters passed anywhere
parameters = path_parameters + query_parameters
# Build extra body parameters documentation
extra_body_parameters = []
for param_name, param_type, description in op.extra_body_params:
if is_type_optional(param_type):
inner_type: type = unwrap_optional_type(param_type)
required = False
else:
inner_type = param_type
required = True
# Use description from ExtraBodyField if available, otherwise from docstring
param_description = description or doc_params.get(param_name)
extra_body_param = ExtraBodyParameter(
name=param_name,
schema=self.schema_builder.classdef_to_ref(inner_type),
description=param_description,
required=required,
)
extra_body_parameters.append(extra_body_param)
webmethod = getattr(op.func_ref, "__webmethod__", None)
raw_bytes_request_body = False
if webmethod:
@ -898,6 +920,7 @@ class Generator:
deprecated=getattr(op.webmethod, "deprecated", False)
or "DEPRECATED" in op.func_name,
security=[] if op.public else None,
extraBodyParameters=extra_body_parameters if extra_body_parameters else None,
)
def _get_api_stability_priority(self, api_level: str) -> int:

42
docs/openapi_generator/pyopenapi/operations.py
View file

@ -19,10 +19,12 @@ from llama_stack.strong_typing.inspection import get_signature
from typing import get_origin, get_args
from fastapi import UploadFile
from fastapi import UploadFile
from fastapi.params import File, Form
from typing import Annotated
from llama_stack.schema_utils import ExtraBodyField
def split_prefix(
s: str, sep: str, prefix: Union[str, Iterable[str]]
@ -89,6 +91,7 @@ class EndpointOperation:
:param query_params: Parameters of the operation signature that are passed in the query string as `key=value` pairs.
:param request_params: The parameter that corresponds to the data transmitted in the request body.
:param multipart_params: Parameters that indicate multipart/form-data request body.
:param extra_body_params: Parameters that arrive via extra_body and are documented but not in SDK.
:param event_type: The Python type of the data that is transmitted out-of-band (e.g. via websockets) while the operation is in progress.
:param response_type: The Python type of the data that is transmitted in the response body.
:param http_method: The HTTP method used to invoke the endpoint such as POST, GET or PUT.
@ -106,6 +109,7 @@ class EndpointOperation:
query_params: List[OperationParameter]
request_params: Optional[OperationParameter]
multipart_params: List[OperationParameter]
extra_body_params: List[tuple[str, type, str | None]]
event_type: Optional[type]
response_type: type
http_method: HTTPMethod
@ -265,6 +269,7 @@ def get_endpoint_operations(
query_params = []
request_params = []
multipart_params = []
extra_body_params = []
for param_name, parameter in signature.parameters.items():
param_type = _get_annotation_type(parameter.annotation, func_ref)
@ -279,6 +284,13 @@ def get_endpoint_operations(
f"parameter '{param_name}' in function '{func_name}' has no type annotation"
)
# Check if this is an extra_body parameter
is_extra_body, extra_body_desc = _is_extra_body_param(param_type)
if is_extra_body:
# Store in a separate list for documentation
extra_body_params.append((param_name, param_type, extra_body_desc))
continue # Skip adding to request_params
is_multipart = _is_multipart_param(param_type)
if prefix in ["get", "delete"]:
@ -351,6 +363,7 @@ def get_endpoint_operations(
query_params=query_params,
request_params=request_params,
multipart_params=multipart_params,
extra_body_params=extra_body_params,
event_type=event_type,
response_type=response_type,
http_method=http_method,
@ -403,7 +416,7 @@ def get_endpoint_events(endpoint: type) -> Dict[str, type]:
def _is_multipart_param(param_type: type) -> bool:
"""
Check if a parameter type indicates multipart form data.
Returns True if the type is:
- UploadFile
- Annotated[UploadFile, File()]
@ -413,19 +426,38 @@ def _is_multipart_param(param_type: type) -> bool:
"""
if param_type is UploadFile:
return True
# Check for Annotated types
origin = get_origin(param_type)
if origin is None:
return False
if origin is Annotated:
args = get_args(param_type)
if len(args) < 2:
return False
# Check the annotations for File() or Form()
for annotation in args[1:]:
if isinstance(annotation, (File, Form)):
return True
return False
def _is_extra_body_param(param_type: type) -> tuple[bool, str | None]:
"""
Check if parameter is marked as coming from extra_body.
Returns:
(is_extra_body, description): Tuple of boolean and optional description
"""
origin = get_origin(param_type)
if origin is Annotated:
args = get_args(param_type)
for annotation in args[1:]:
if isinstance(annotation, ExtraBodyField):
return True, annotation.description
# Also check by type name for cases where import matters
if type(annotation).__name__ == 'ExtraBodyField':
return True, getattr(annotation, 'description', None)
return False, None

10
docs/openapi_generator/pyopenapi/specification.py
View file

@ -106,6 +106,15 @@ class Parameter:
example: Optional[Any] = None
@dataclass
class ExtraBodyParameter:
"""Represents a parameter that arrives via extra_body in the request."""
name: str
schema: SchemaOrRef
description: Optional[str] = None
required: Optional[bool] = None
@dataclass
class Operation:
responses: Dict[str, Union[Response, ResponseRef]]
@ -118,6 +127,7 @@ class Operation:
callbacks: Optional[Dict[str, "Callback"]] = None
security: Optional[List["SecurityRequirement"]] = None
deprecated: Optional[bool] = None
extraBodyParameters: Optional[List[ExtraBodyParameter]] = None
@dataclass

11
docs/openapi_generator/pyopenapi/utility.py
View file

@ -52,6 +52,17 @@ class Specification:
if display_name:
tag["x-displayName"] = display_name
# Handle operations to rename extraBodyParameters -> x-llama-stack-extra-body-params
paths = json_doc.get("paths", {})
for path_item in paths.values():
if isinstance(path_item, dict):
for method in ["get", "post", "put", "delete", "patch"]:
operation = path_item.get(method)
if operation and isinstance(operation, dict):
extra_body_params = operation.pop("extraBodyParameters", None)
if extra_body_params:
operation["x-llama-stack-extra-body-params"] = extra_body_params
return json_doc
def get_json_string(self, pretty_print: bool = False) -> str:

111
docs/static/deprecated-llama-stack-spec.html vendored
View file

@ -1443,8 +1443,8 @@
"tags": [
"Inference"
],
"summary": "List all chat completions.",
"description": "List all chat completions.",
"summary": "List chat completions.",
"description": "List chat completions.",
"parameters": [
{
"name": "after",
@ -1520,8 +1520,8 @@
"tags": [
"Inference"
],
"summary": "Generate an OpenAI-compatible chat completion for the given messages using the specified model.",
"description": "Generate an OpenAI-compatible chat completion for the given messages using the specified model.",
"summary": "Create chat completions.",
"description": "Create chat completions.\nGenerate an OpenAI-compatible chat completion for the given messages using the specified model.",
"parameters": [],
"requestBody": {
"content": {
@ -1565,8 +1565,8 @@
"tags": [
"Inference"
],
"summary": "Describe a chat completion by its ID.",
"description": "Describe a chat completion by its ID.",
"summary": "Get chat completion.",
"description": "Get chat completion.\nDescribe a chat completion by its ID.",
"parameters": [
{
"name": "completion_id",
@ -1610,8 +1610,8 @@
"tags": [
"Inference"
],
"summary": "Generate an OpenAI-compatible completion for the given prompt using the specified model.",
"description": "Generate an OpenAI-compatible completion for the given prompt using the specified model.",
"summary": "Create completion.",
"description": "Create completion.\nGenerate an OpenAI-compatible completion for the given prompt using the specified model.",
"parameters": [],
"requestBody": {
"content": {
@ -1655,8 +1655,8 @@
"tags": [
"Inference"
],
"summary": "Generate OpenAI-compatible embeddings for the given input using the specified model.",
"description": "Generate OpenAI-compatible embeddings for the given input using the specified model.",
"summary": "Create embeddings.",
"description": "Create embeddings.\nGenerate OpenAI-compatible embeddings for the given input using the specified model.",
"parameters": [],
"requestBody": {
"content": {
@ -1700,8 +1700,8 @@
"tags": [
"Files"
],
"summary": "Returns a list of files that belong to the user's organization.",
"description": "Returns a list of files that belong to the user's organization.",
"summary": "List files.",
"description": "List files.\nReturns a list of files that belong to the user's organization.",
"parameters": [
{
"name": "after",
@ -1770,8 +1770,8 @@
"tags": [
"Files"
],
"summary": "Upload a file that can be used across various endpoints.",
"description": "Upload a file that can be used across various endpoints.\nThe file upload should be a multipart form request with:\n- file: The File object (not file name) to be uploaded.\n- purpose: The intended purpose of the uploaded file.\n- expires_after: Optional form values describing expiration for the file.",
"summary": "Upload file.",
"description": "Upload file.\nUpload a file that can be used across various endpoints.\n\nThe file upload should be a multipart form request with:\n- file: The File object (not file name) to be uploaded.\n- purpose: The intended purpose of the uploaded file.\n- expires_after: Optional form values describing expiration for the file.",
"parameters": [],
"requestBody": {
"content": {
@ -1831,8 +1831,8 @@
"tags": [
"Files"
],
"summary": "Returns information about a specific file.",
"description": "Returns information about a specific file.",
"summary": "Retrieve file.",
"description": "Retrieve file.\nReturns information about a specific file.",
"parameters": [
{
"name": "file_id",
@ -1874,8 +1874,8 @@
"tags": [
"Files"
],
"summary": "Delete a file.",
"description": "Delete a file.",
"summary": "Delete file.",
"description": "Delete file.",
"parameters": [
{
"name": "file_id",
@ -1919,8 +1919,8 @@
"tags": [
"Files"
],
"summary": "Returns the contents of the specified file.",
"description": "Returns the contents of the specified file.",
"summary": "Retrieve file content.",
"description": "Retrieve file content.\nReturns the contents of the specified file.",
"parameters": [
{
"name": "file_id",
@ -1999,8 +1999,8 @@
"tags": [
"Safety"
],
"summary": "Classifies if text and/or image inputs are potentially harmful.",
"description": "Classifies if text and/or image inputs are potentially harmful.",
"summary": "Create moderation.",
"description": "Create moderation.\nClassifies if text and/or image inputs are potentially harmful.",
"parameters": [],
"requestBody": {
"content": {
@ -2044,8 +2044,8 @@
"tags": [
"Agents"
],
"summary": "List all OpenAI responses.",
"description": "List all OpenAI responses.",
"summary": "List all responses.",
"description": "List all responses.",
"parameters": [
{
"name": "after",
@ -2119,8 +2119,8 @@
"tags": [
"Agents"
],
"summary": "Create a new OpenAI response.",
"description": "Create a new OpenAI response.",
"summary": "Create a model response.",
"description": "Create a model response.",
"parameters": [],
"requestBody": {
"content": {
@ -2132,7 +2132,27 @@
},
"required": true
},
"deprecated": true
"deprecated": true,
"x-llama-stack-extra-body-params": [
{
"name": "shields",
"schema": {
"type": "array",
"items": {
"oneOf": [
{
"type": "string"
},
{
"$ref": "#/components/schemas/ResponseShieldSpec"
}
]
}
},
"description": "List of shields to apply during response generation. Shields provide safety and content moderation.",
"required": false
}
]
}
},
"/v1/openai/v1/responses/{response_id}": {
@ -2164,8 +2184,8 @@
"tags": [
"Agents"
],
"summary": "Retrieve an OpenAI response by its ID.",
"description": "Retrieve an OpenAI response by its ID.",
"summary": "Get a model response.",
"description": "Get a model response.",
"parameters": [
{
"name": "response_id",
@ -2207,8 +2227,8 @@
"tags": [
"Agents"
],
"summary": "Delete an OpenAI response by its ID.",
"description": "Delete an OpenAI response by its ID.",
"summary": "Delete a response.",
"description": "Delete a response.",
"parameters": [
{
"name": "response_id",
@ -2252,8 +2272,8 @@
"tags": [
"Agents"
],
"summary": "List input items for a given OpenAI response.",
"description": "List input items for a given OpenAI response.",
"summary": "List input items.",
"description": "List input items.",
"parameters": [
{
"name": "response_id",
@ -9521,6 +9541,21 @@
"title": "OpenAIResponseText",
"description": "Text response configuration for OpenAI responses."
},
"ResponseShieldSpec": {
"type": "object",
"properties": {
"type": {
"type": "string",
"description": "The type/identifier of the shield."
}
},
"additionalProperties": false,
"required": [
"type"
],
"title": "ResponseShieldSpec",
"description": "Specification for a shield to apply during response generation."
},
"OpenAIResponseInputTool": {
"oneOf": [
{
@ -13331,12 +13366,13 @@
},
{
"name": "Files",
"description": ""
"description": "This API is used to upload documents that can be used with other Llama Stack APIs.",
"x-displayName": "Files"
},
{
"name": "Inference",
"description": "This API provides the raw interface to the underlying models. Two kinds of models are supported:\n- LLM models: these models generate \"raw\" and \"chat\" (conversational) completions.\n- Embedding models: these models generate embeddings to be used for semantic search.",
"x-displayName": "Llama Stack Inference API for generating completions, chat completions, and embeddings."
"description": "Llama Stack Inference API for generating completions, chat completions, and embeddings.\n\nThis API provides the raw interface to the underlying models. Two kinds of models are supported:\n- LLM models: these models generate \"raw\" and \"chat\" (conversational) completions.\n- Embedding models: these models generate embeddings to be used for semantic search.",
"x-displayName": "Inference"
},
{
"name": "Models",
@ -13348,7 +13384,8 @@
},
{
"name": "Safety",
"description": ""
"description": "OpenAI-compatible Moderations API.",
"x-displayName": "Safety"
},
{
"name": "Telemetry",

121
docs/static/deprecated-llama-stack-spec.yaml vendored
View file

@ -1033,8 +1033,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: List all chat completions.
description: List all chat completions.
summary: List chat completions.
description: List chat completions.
parameters:
- name: after
in: query
@ -1087,10 +1087,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: >-
Generate an OpenAI-compatible chat completion for the given messages using
the specified model.
summary: Create chat completions.
description: >-
Create chat completions.
Generate an OpenAI-compatible chat completion for the given messages using
the specified model.
parameters: []
@ -1122,8 +1122,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: Describe a chat completion by its ID.
description: Describe a chat completion by its ID.
summary: Get chat completion.
description: >-
Get chat completion.
Describe a chat completion by its ID.
parameters:
- name: completion_id
in: path
@ -1153,10 +1156,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: >-
Generate an OpenAI-compatible completion for the given prompt using the specified
model.
summary: Create completion.
description: >-
Create completion.
Generate an OpenAI-compatible completion for the given prompt using the specified
model.
parameters: []
@ -1189,10 +1192,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: >-
Generate OpenAI-compatible embeddings for the given input using the specified
model.
summary: Create embeddings.
description: >-
Create embeddings.
Generate OpenAI-compatible embeddings for the given input using the specified
model.
parameters: []
@ -1225,9 +1228,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: >-
Returns a list of files that belong to the user's organization.
summary: List files.
description: >-
List files.
Returns a list of files that belong to the user's organization.
parameters:
- name: after
@ -1285,11 +1289,13 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: >-
Upload a file that can be used across various endpoints.
summary: Upload file.
description: >-
Upload file.
Upload a file that can be used across various endpoints.
The file upload should be a multipart form request with:
- file: The File object (not file name) to be uploaded.
@ -1338,9 +1344,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: >-
Returns information about a specific file.
summary: Retrieve file.
description: >-
Retrieve file.
Returns information about a specific file.
parameters:
- name: file_id
@ -1372,8 +1379,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: Delete a file.
description: Delete a file.
summary: Delete file.
description: Delete file.
parameters:
- name: file_id
in: path
@ -1405,9 +1412,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: >-
Returns the contents of the specified file.
summary: Retrieve file content.
description: >-
Retrieve file content.
Returns the contents of the specified file.
parameters:
- name: file_id
@ -1464,9 +1472,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Safety
summary: >-
Classifies if text and/or image inputs are potentially harmful.
summary: Create moderation.
description: >-
Create moderation.
Classifies if text and/or image inputs are potentially harmful.
parameters: []
requestBody:
@ -1497,8 +1506,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: List all OpenAI responses.
description: List all OpenAI responses.
summary: List all responses.
description: List all responses.
parameters:
- name: after
in: query
@ -1549,8 +1558,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: Create a new OpenAI response.
description: Create a new OpenAI response.
summary: Create a model response.
description: Create a model response.
parameters: []
requestBody:
content:
@ -1559,6 +1568,18 @@ paths:
$ref: '#/components/schemas/CreateOpenaiResponseRequest'
required: true
deprecated: true
x-llama-stack-extra-body-params:
- name: shields
schema:
type: array
items:
oneOf:
- type: string
- $ref: '#/components/schemas/ResponseShieldSpec'
description: >-
List of shields to apply during response generation. Shields provide safety
and content moderation.
required: false
/v1/openai/v1/responses/{response_id}:
get:
responses:
@ -1580,8 +1601,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: Retrieve an OpenAI response by its ID.
description: Retrieve an OpenAI response by its ID.
summary: Get a model response.
description: Get a model response.
parameters:
- name: response_id
in: path
@ -1611,8 +1632,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: Delete an OpenAI response by its ID.
description: Delete an OpenAI response by its ID.
summary: Delete a response.
description: Delete a response.
parameters:
- name: response_id
in: path
@ -1642,10 +1663,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: >-
List input items for a given OpenAI response.
description: >-
List input items for a given OpenAI response.
summary: List input items.
description: List input items.
parameters:
- name: response_id
in: path
@ -7076,6 +7095,18 @@ components:
title: OpenAIResponseText
description: >-
Text response configuration for OpenAI responses.
ResponseShieldSpec:
type: object
properties:
type:
type: string
description: The type/identifier of the shield.
additionalProperties: false
required:
- type
title: ResponseShieldSpec
description: >-
Specification for a shield to apply during response generation.
OpenAIResponseInputTool:
oneOf:
- $ref: '#/components/schemas/OpenAIResponseInputToolWebSearch'
@ -9987,9 +10018,16 @@ tags:
x-displayName: >-
Llama Stack Evaluation API for running evaluations on model and agent candidates.
- name: Files
description: ''
description: >-
This API is used to upload documents that can be used with other Llama Stack
APIs.
x-displayName: Files
- name: Inference
description: >-
Llama Stack Inference API for generating completions, chat completions, and
embeddings.
This API provides the raw interface to the underlying models. Two kinds of models
are supported:
@ -9997,15 +10035,14 @@ tags:
- Embedding models: these models generate embeddings to be used for semantic
search.
x-displayName: >-
Llama Stack Inference API for generating completions, chat completions, and
embeddings.
x-displayName: Inference
- name: Models
description: ''
- name: PostTraining (Coming Soon)
description: ''
- name: Safety
description: ''
description: OpenAI-compatible Moderations API.
x-displayName: Safety
- name: Telemetry
description: ''
- name: VectorIO

182
docs/static/llama-stack-spec.html vendored
View file

@ -69,8 +69,8 @@
"tags": [
"Inference"
],
"summary": "List all chat completions.",
"description": "List all chat completions.",
"summary": "List chat completions.",
"description": "List chat completions.",
"parameters": [
{
"name": "after",
@ -146,8 +146,8 @@
"tags": [
"Inference"
],
"summary": "Generate an OpenAI-compatible chat completion for the given messages using the specified model.",
"description": "Generate an OpenAI-compatible chat completion for the given messages using the specified model.",
"summary": "Create chat completions.",
"description": "Create chat completions.\nGenerate an OpenAI-compatible chat completion for the given messages using the specified model.",
"parameters": [],
"requestBody": {
"content": {
@ -191,8 +191,8 @@
"tags": [
"Inference"
],
"summary": "Describe a chat completion by its ID.",
"description": "Describe a chat completion by its ID.",
"summary": "Get chat completion.",
"description": "Get chat completion.\nDescribe a chat completion by its ID.",
"parameters": [
{
"name": "completion_id",
@ -236,8 +236,8 @@
"tags": [
"Inference"
],
"summary": "Generate an OpenAI-compatible completion for the given prompt using the specified model.",
"description": "Generate an OpenAI-compatible completion for the given prompt using the specified model.",
"summary": "Create completion.",
"description": "Create completion.\nGenerate an OpenAI-compatible completion for the given prompt using the specified model.",
"parameters": [],
"requestBody": {
"content": {
@ -758,8 +758,8 @@
"tags": [
"Inference"
],
"summary": "Generate OpenAI-compatible embeddings for the given input using the specified model.",
"description": "Generate OpenAI-compatible embeddings for the given input using the specified model.",
"summary": "Create embeddings.",
"description": "Create embeddings.\nGenerate OpenAI-compatible embeddings for the given input using the specified model.",
"parameters": [],
"requestBody": {
"content": {
@ -803,8 +803,8 @@
"tags": [
"Files"
],
"summary": "Returns a list of files that belong to the user's organization.",
"description": "Returns a list of files that belong to the user's organization.",
"summary": "List files.",
"description": "List files.\nReturns a list of files that belong to the user's organization.",
"parameters": [
{
"name": "after",
@ -873,8 +873,8 @@
"tags": [
"Files"
],
"summary": "Upload a file that can be used across various endpoints.",
"description": "Upload a file that can be used across various endpoints.\nThe file upload should be a multipart form request with:\n- file: The File object (not file name) to be uploaded.\n- purpose: The intended purpose of the uploaded file.\n- expires_after: Optional form values describing expiration for the file.",
"summary": "Upload file.",
"description": "Upload file.\nUpload a file that can be used across various endpoints.\n\nThe file upload should be a multipart form request with:\n- file: The File object (not file name) to be uploaded.\n- purpose: The intended purpose of the uploaded file.\n- expires_after: Optional form values describing expiration for the file.",
"parameters": [],
"requestBody": {
"content": {
@ -934,8 +934,8 @@
"tags": [
"Files"
],
"summary": "Returns information about a specific file.",
"description": "Returns information about a specific file.",
"summary": "Retrieve file.",
"description": "Retrieve file.\nReturns information about a specific file.",
"parameters": [
{
"name": "file_id",
@ -977,8 +977,8 @@
"tags": [
"Files"
],
"summary": "Delete a file.",
"description": "Delete a file.",
"summary": "Delete file.",
"description": "Delete file.",
"parameters": [
{
"name": "file_id",
@ -1022,8 +1022,8 @@
"tags": [
"Files"
],
"summary": "Returns the contents of the specified file.",
"description": "Returns the contents of the specified file.",
"summary": "Retrieve file content.",
"description": "Retrieve file content.\nReturns the contents of the specified file.",
"parameters": [
{
"name": "file_id",
@ -1067,8 +1067,8 @@
"tags": [
"Inspect"
],
"summary": "Get the current health status of the service.",
"description": "Get the current health status of the service.",
"summary": "Get health status.",
"description": "Get health status.\nGet the current health status of the service.",
"parameters": [],
"deprecated": false
}
@ -1102,8 +1102,8 @@
"tags": [
"Inspect"
],
"summary": "List all available API routes with their methods and implementing providers.",
"description": "List all available API routes with their methods and implementing providers.",
"summary": "List routes.",
"description": "List routes.\nList all available API routes with their methods and implementing providers.",
"parameters": [],
"deprecated": false
}
@ -1170,8 +1170,8 @@
"tags": [
"Models"
],
"summary": "Register a model.",
"description": "Register a model.",
"summary": "Register model.",
"description": "Register model.\nRegister a model.",
"parameters": [],
"requestBody": {
"content": {
@ -1215,8 +1215,8 @@
"tags": [
"Models"
],
"summary": "Get a model by its identifier.",
"description": "Get a model by its identifier.",
"summary": "Get model.",
"description": "Get model.\nGet a model by its identifier.",
"parameters": [
{
"name": "model_id",
@ -1251,8 +1251,8 @@
"tags": [
"Models"
],
"summary": "Unregister a model.",
"description": "Unregister a model.",
"summary": "Unregister model.",
"description": "Unregister model.\nUnregister a model.",
"parameters": [
{
"name": "model_id",
@ -1296,8 +1296,8 @@
"tags": [
"Safety"
],
"summary": "Classifies if text and/or image inputs are potentially harmful.",
"description": "Classifies if text and/or image inputs are potentially harmful.",
"summary": "Create moderation.",
"description": "Create moderation.\nClassifies if text and/or image inputs are potentially harmful.",
"parameters": [],
"requestBody": {
"content": {
@ -1374,8 +1374,8 @@
"tags": [
"Prompts"
],
"summary": "Create a new prompt.",
"description": "Create a new prompt.",
"summary": "Create prompt.",
"description": "Create prompt.\nCreate a new prompt.",
"parameters": [],
"requestBody": {
"content": {
@ -1419,8 +1419,8 @@
"tags": [
"Prompts"
],
"summary": "Get a prompt by its identifier and optional version.",
"description": "Get a prompt by its identifier and optional version.",
"summary": "Get prompt.",
"description": "Get prompt.\nGet a prompt by its identifier and optional version.",
"parameters": [
{
"name": "prompt_id",
@ -1471,8 +1471,8 @@
"tags": [
"Prompts"
],
"summary": "Update an existing prompt (increments version).",
"description": "Update an existing prompt (increments version).",
"summary": "Update prompt.",
"description": "Update prompt.\nUpdate an existing prompt (increments version).",
"parameters": [
{
"name": "prompt_id",
@ -1517,8 +1517,8 @@
"tags": [
"Prompts"
],
"summary": "Delete a prompt.",
"description": "Delete a prompt.",
"summary": "Delete prompt.",
"description": "Delete prompt.\nDelete a prompt.",
"parameters": [
{
"name": "prompt_id",
@ -1562,8 +1562,8 @@
"tags": [
"Prompts"
],
"summary": "Set which version of a prompt should be the default in get_prompt (latest).",
"description": "Set which version of a prompt should be the default in get_prompt (latest).",
"summary": "Set prompt version.",
"description": "Set prompt version.\nSet which version of a prompt should be the default in get_prompt (latest).",
"parameters": [
{
"name": "prompt_id",
@ -1617,8 +1617,8 @@
"tags": [
"Prompts"
],
"summary": "List all versions of a specific prompt.",
"description": "List all versions of a specific prompt.",
"summary": "List prompt versions.",
"description": "List prompt versions.\nList all versions of a specific prompt.",
"parameters": [
{
"name": "prompt_id",
@ -1662,8 +1662,8 @@
"tags": [
"Providers"
],
"summary": "List all available providers.",
"description": "List all available providers.",
"summary": "List providers.",
"description": "List providers.\nList all available providers.",
"parameters": [],
"deprecated": false
}
@ -1697,8 +1697,8 @@
"tags": [
"Providers"
],
"summary": "Get detailed information about a specific provider.",
"description": "Get detailed information about a specific provider.",
"summary": "Get provider.",
"description": "Get provider.\nGet detailed information about a specific provider.",
"parameters": [
{
"name": "provider_id",
@ -1742,8 +1742,8 @@
"tags": [
"Agents"
],
"summary": "List all OpenAI responses.",
"description": "List all OpenAI responses.",
"summary": "List all responses.",
"description": "List all responses.",
"parameters": [
{
"name": "after",
@ -1817,8 +1817,8 @@
"tags": [
"Agents"
],
"summary": "Create a new OpenAI response.",
"description": "Create a new OpenAI response.",
"summary": "Create a model response.",
"description": "Create a model response.",
"parameters": [],
"requestBody": {
"content": {
@ -1830,7 +1830,27 @@
},
"required": true
},
"deprecated": false
"deprecated": false,
"x-llama-stack-extra-body-params": [
{
"name": "shields",
"schema": {
"type": "array",
"items": {
"oneOf": [
{
"type": "string"
},
{
"$ref": "#/components/schemas/ResponseShieldSpec"
}
]
}
},
"description": "List of shields to apply during response generation. Shields provide safety and content moderation.",
"required": false
}
]
}
},
"/v1/responses/{response_id}": {
@ -1862,8 +1882,8 @@
"tags": [
"Agents"
],
"summary": "Retrieve an OpenAI response by its ID.",
"description": "Retrieve an OpenAI response by its ID.",
"summary": "Get a model response.",
"description": "Get a model response.",
"parameters": [
{
"name": "response_id",
@ -1905,8 +1925,8 @@
"tags": [
"Agents"
],
"summary": "Delete an OpenAI response by its ID.",
"description": "Delete an OpenAI response by its ID.",
"summary": "Delete a response.",
"description": "Delete a response.",
"parameters": [
{
"name": "response_id",
@ -1950,8 +1970,8 @@
"tags": [
"Agents"
],
"summary": "List input items for a given OpenAI response.",
"description": "List input items for a given OpenAI response.",
"summary": "List input items.",
"description": "List input items.",
"parameters": [
{
"name": "response_id",
@ -2043,8 +2063,8 @@
"tags": [
"Safety"
],
"summary": "Run a shield.",
"description": "Run a shield.",
"summary": "Run shield.",
"description": "Run shield.\nRun a shield.",
"parameters": [],
"requestBody": {
"content": {
@ -4176,8 +4196,8 @@
"tags": [
"Inspect"
],
"summary": "Get the version of the service.",
"description": "Get the version of the service.",
"summary": "Get version.",
"description": "Get version.\nGet the version of the service.",
"parameters": [],
"deprecated": false
}
@ -7616,6 +7636,21 @@
"title": "OpenAIResponseText",
"description": "Text response configuration for OpenAI responses."
},
"ResponseShieldSpec": {
"type": "object",
"properties": {
"type": {
"type": "string",
"description": "The type/identifier of the shield."
}
},
"additionalProperties": false,
"required": [
"type"
],
"title": "ResponseShieldSpec",
"description": "Specification for a shield to apply during response generation."
},
"OpenAIResponseInputTool": {
"oneOf": [
{
@ -12879,16 +12914,18 @@
},
{
"name": "Files",
"description": ""
"description": "This API is used to upload documents that can be used with other Llama Stack APIs.",
"x-displayName": "Files"
},
{
"name": "Inference",
"description": "This API provides the raw interface to the underlying models. Two kinds of models are supported:\n- LLM models: these models generate \"raw\" and \"chat\" (conversational) completions.\n- Embedding models: these models generate embeddings to be used for semantic search.",
"x-displayName": "Llama Stack Inference API for generating completions, chat completions, and embeddings."
"description": "Llama Stack Inference API for generating completions, chat completions, and embeddings.\n\nThis API provides the raw interface to the underlying models. Two kinds of models are supported:\n- LLM models: these models generate \"raw\" and \"chat\" (conversational) completions.\n- Embedding models: these models generate embeddings to be used for semantic search.",
"x-displayName": "Inference"
},
{
"name": "Inspect",
"description": ""
"description": "APIs for inspecting the Llama Stack service, including health status, available API routes with methods and implementing providers.",
"x-displayName": "Inspect"
},
{
"name": "Models",
@ -12896,17 +12933,18 @@
},
{
"name": "Prompts",
"description": "",
"x-displayName": "Protocol for prompt management operations."
"description": "Protocol for prompt management operations.",
"x-displayName": "Prompts"
},
{
"name": "Providers",
"description": "",
"x-displayName": "Providers API for inspecting, listing, and modifying providers and their configurations."
"description": "Providers API for inspecting, listing, and modifying providers and their configurations.",
"x-displayName": "Providers"
},
{
"name": "Safety",
"description": ""
"description": "OpenAI-compatible Moderations API.",
"x-displayName": "Safety"
},
{
"name": "Scoring",

227
docs/static/llama-stack-spec.yaml vendored
View file

@ -33,8 +33,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: List all chat completions.
description: List all chat completions.
summary: List chat completions.
description: List chat completions.
parameters:
- name: after
in: query
@ -87,10 +87,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: >-
Generate an OpenAI-compatible chat completion for the given messages using
the specified model.
summary: Create chat completions.
description: >-
Create chat completions.
Generate an OpenAI-compatible chat completion for the given messages using
the specified model.
parameters: []
@ -122,8 +122,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: Describe a chat completion by its ID.
description: Describe a chat completion by its ID.
summary: Get chat completion.
description: >-
Get chat completion.
Describe a chat completion by its ID.
parameters:
- name: completion_id
in: path
@ -153,10 +156,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: >-
Generate an OpenAI-compatible completion for the given prompt using the specified
model.
summary: Create completion.
description: >-
Create completion.
Generate an OpenAI-compatible completion for the given prompt using the specified
model.
parameters: []
@ -603,10 +606,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: >-
Generate OpenAI-compatible embeddings for the given input using the specified
model.
summary: Create embeddings.
description: >-
Create embeddings.
Generate OpenAI-compatible embeddings for the given input using the specified
model.
parameters: []
@ -639,9 +642,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: >-
Returns a list of files that belong to the user's organization.
summary: List files.
description: >-
List files.
Returns a list of files that belong to the user's organization.
parameters:
- name: after
@ -699,11 +703,13 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: >-
Upload a file that can be used across various endpoints.
summary: Upload file.
description: >-
Upload file.
Upload a file that can be used across various endpoints.
The file upload should be a multipart form request with:
- file: The File object (not file name) to be uploaded.
@ -752,9 +758,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: >-
Returns information about a specific file.
summary: Retrieve file.
description: >-
Retrieve file.
Returns information about a specific file.
parameters:
- name: file_id
@ -786,8 +793,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: Delete a file.
description: Delete a file.
summary: Delete file.
description: Delete file.
parameters:
- name: file_id
in: path
@ -819,9 +826,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: >-
Returns the contents of the specified file.
summary: Retrieve file content.
description: >-
Retrieve file content.
Returns the contents of the specified file.
parameters:
- name: file_id
@ -854,9 +862,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inspect
summary: >-
Get the current health status of the service.
summary: Get health status.
description: >-
Get health status.
Get the current health status of the service.
parameters: []
deprecated: false
@ -882,9 +891,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inspect
summary: >-
List all available API routes with their methods and implementing providers.
summary: List routes.
description: >-
List routes.
List all available API routes with their methods and implementing providers.
parameters: []
deprecated: false
@ -933,8 +943,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Models
summary: Register a model.
description: Register a model.
summary: Register model.
description: >-
Register model.
Register a model.
parameters: []
requestBody:
content:
@ -964,8 +977,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Models
summary: Get a model by its identifier.
description: Get a model by its identifier.
summary: Get model.
description: >-
Get model.
Get a model by its identifier.
parameters:
- name: model_id
in: path
@ -990,8 +1006,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Models
summary: Unregister a model.
description: Unregister a model.
summary: Unregister model.
description: >-
Unregister model.
Unregister a model.
parameters:
- name: model_id
in: path
@ -1022,9 +1041,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Safety
summary: >-
Classifies if text and/or image inputs are potentially harmful.
summary: Create moderation.
description: >-
Create moderation.
Classifies if text and/or image inputs are potentially harmful.
parameters: []
requestBody:
@ -1080,8 +1100,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Prompts
summary: Create a new prompt.
description: Create a new prompt.
summary: Create prompt.
description: >-
Create prompt.
Create a new prompt.
parameters: []
requestBody:
content:
@ -1111,9 +1134,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Prompts
summary: >-
Get a prompt by its identifier and optional version.
summary: Get prompt.
description: >-
Get prompt.
Get a prompt by its identifier and optional version.
parameters:
- name: prompt_id
@ -1151,9 +1175,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Prompts
summary: >-
Update an existing prompt (increments version).
summary: Update prompt.
description: >-
Update prompt.
Update an existing prompt (increments version).
parameters:
- name: prompt_id
@ -1185,8 +1210,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Prompts
summary: Delete a prompt.
description: Delete a prompt.
summary: Delete prompt.
description: >-
Delete prompt.
Delete a prompt.
parameters:
- name: prompt_id
in: path
@ -1217,9 +1245,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Prompts
summary: >-
Set which version of a prompt should be the default in get_prompt (latest).
summary: Set prompt version.
description: >-
Set prompt version.
Set which version of a prompt should be the default in get_prompt (latest).
parameters:
- name: prompt_id
@ -1257,8 +1286,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Prompts
summary: List all versions of a specific prompt.
description: List all versions of a specific prompt.
summary: List prompt versions.
description: >-
List prompt versions.
List all versions of a specific prompt.
parameters:
- name: prompt_id
in: path
@ -1290,8 +1322,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Providers
summary: List all available providers.
description: List all available providers.
summary: List providers.
description: >-
List providers.
List all available providers.
parameters: []
deprecated: false
/v1/providers/{provider_id}:
@ -1316,9 +1351,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Providers
summary: >-
Get detailed information about a specific provider.
summary: Get provider.
description: >-
Get provider.
Get detailed information about a specific provider.
parameters:
- name: provider_id
@ -1349,8 +1385,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: List all OpenAI responses.
description: List all OpenAI responses.
summary: List all responses.
description: List all responses.
parameters:
- name: after
in: query
@ -1401,8 +1437,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: Create a new OpenAI response.
description: Create a new OpenAI response.
summary: Create a model response.
description: Create a model response.
parameters: []
requestBody:
content:
@ -1411,6 +1447,18 @@ paths:
$ref: '#/components/schemas/CreateOpenaiResponseRequest'
required: true
deprecated: false
x-llama-stack-extra-body-params:
- name: shields
schema:
type: array
items:
oneOf:
- type: string
- $ref: '#/components/schemas/ResponseShieldSpec'
description: >-
List of shields to apply during response generation. Shields provide safety
and content moderation.
required: false
/v1/responses/{response_id}:
get:
responses:
@ -1432,8 +1480,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: Retrieve an OpenAI response by its ID.
description: Retrieve an OpenAI response by its ID.
summary: Get a model response.
description: Get a model response.
parameters:
- name: response_id
in: path
@ -1463,8 +1511,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: Delete an OpenAI response by its ID.
description: Delete an OpenAI response by its ID.
summary: Delete a response.
description: Delete a response.
parameters:
- name: response_id
in: path
@ -1494,10 +1542,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: >-
List input items for a given OpenAI response.
description: >-
List input items for a given OpenAI response.
summary: List input items.
description: List input items.
parameters:
- name: response_id
in: path
@ -1566,8 +1612,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Safety
summary: Run a shield.
description: Run a shield.
summary: Run shield.
description: >-
Run shield.
Run a shield.
parameters: []
requestBody:
content:
@ -3123,8 +3172,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inspect
summary: Get the version of the service.
description: Get the version of the service.
summary: Get version.
description: >-
Get version.
Get the version of the service.
parameters: []
deprecated: false
jsonSchemaDialect: >-
@ -5739,6 +5791,18 @@ components:
title: OpenAIResponseText
description: >-
Text response configuration for OpenAI responses.
ResponseShieldSpec:
type: object
properties:
type:
type: string
description: The type/identifier of the shield.
additionalProperties: false
required:
- type
title: ResponseShieldSpec
description: >-
Specification for a shield to apply during response generation.
OpenAIResponseInputTool:
oneOf:
- $ref: '#/components/schemas/OpenAIResponseInputToolWebSearch'
@ -9725,9 +9789,16 @@ tags:
x-displayName: >-
Protocol for conversation management operations.
- name: Files
description: ''
description: >-
This API is used to upload documents that can be used with other Llama Stack
APIs.
x-displayName: Files
- name: Inference
description: >-
Llama Stack Inference API for generating completions, chat completions, and
embeddings.
This API provides the raw interface to the underlying models. Two kinds of models
are supported:
@ -9735,23 +9806,25 @@ tags:
- Embedding models: these models generate embeddings to be used for semantic
search.
x-displayName: >-
Llama Stack Inference API for generating completions, chat completions, and
embeddings.
x-displayName: Inference
- name: Inspect
description: ''
description: >-
APIs for inspecting the Llama Stack service, including health status, available
API routes with methods and implementing providers.
x-displayName: Inspect
- name: Models
description: ''
- name: Prompts
description: ''
x-displayName: >-
description: >-
Protocol for prompt management operations.
x-displayName: Prompts
- name: Providers
description: ''
x-displayName: >-
description: >-
Providers API for inspecting, listing, and modifying providers and their configurations.
x-displayName: Providers
- name: Safety
description: ''
description: OpenAI-compatible Moderations API.
x-displayName: Safety
- name: Scoring
description: ''
- name: ScoringFunctions

182
docs/static/stainless-llama-stack-spec.html vendored
View file

@ -69,8 +69,8 @@
"tags": [
"Inference"
],
"summary": "List all chat completions.",
"description": "List all chat completions.",
"summary": "List chat completions.",
"description": "List chat completions.",
"parameters": [
{
"name": "after",
@ -146,8 +146,8 @@
"tags": [
"Inference"
],
"summary": "Generate an OpenAI-compatible chat completion for the given messages using the specified model.",
"description": "Generate an OpenAI-compatible chat completion for the given messages using the specified model.",
"summary": "Create chat completions.",
"description": "Create chat completions.\nGenerate an OpenAI-compatible chat completion for the given messages using the specified model.",
"parameters": [],
"requestBody": {
"content": {
@ -191,8 +191,8 @@
"tags": [
"Inference"
],
"summary": "Describe a chat completion by its ID.",
"description": "Describe a chat completion by its ID.",
"summary": "Get chat completion.",
"description": "Get chat completion.\nDescribe a chat completion by its ID.",
"parameters": [
{
"name": "completion_id",
@ -236,8 +236,8 @@
"tags": [
"Inference"
],
"summary": "Generate an OpenAI-compatible completion for the given prompt using the specified model.",
"description": "Generate an OpenAI-compatible completion for the given prompt using the specified model.",
"summary": "Create completion.",
"description": "Create completion.\nGenerate an OpenAI-compatible completion for the given prompt using the specified model.",
"parameters": [],
"requestBody": {
"content": {
@ -758,8 +758,8 @@
"tags": [
"Inference"
],
"summary": "Generate OpenAI-compatible embeddings for the given input using the specified model.",
"description": "Generate OpenAI-compatible embeddings for the given input using the specified model.",
"summary": "Create embeddings.",
"description": "Create embeddings.\nGenerate OpenAI-compatible embeddings for the given input using the specified model.",
"parameters": [],
"requestBody": {
"content": {
@ -803,8 +803,8 @@
"tags": [
"Files"
],
"summary": "Returns a list of files that belong to the user's organization.",
"description": "Returns a list of files that belong to the user's organization.",
"summary": "List files.",
"description": "List files.\nReturns a list of files that belong to the user's organization.",
"parameters": [
{
"name": "after",
@ -873,8 +873,8 @@
"tags": [
"Files"
],
"summary": "Upload a file that can be used across various endpoints.",
"description": "Upload a file that can be used across various endpoints.\nThe file upload should be a multipart form request with:\n- file: The File object (not file name) to be uploaded.\n- purpose: The intended purpose of the uploaded file.\n- expires_after: Optional form values describing expiration for the file.",
"summary": "Upload file.",
"description": "Upload file.\nUpload a file that can be used across various endpoints.\n\nThe file upload should be a multipart form request with:\n- file: The File object (not file name) to be uploaded.\n- purpose: The intended purpose of the uploaded file.\n- expires_after: Optional form values describing expiration for the file.",
"parameters": [],
"requestBody": {
"content": {
@ -934,8 +934,8 @@
"tags": [
"Files"
],
"summary": "Returns information about a specific file.",
"description": "Returns information about a specific file.",
"summary": "Retrieve file.",
"description": "Retrieve file.\nReturns information about a specific file.",
"parameters": [
{
"name": "file_id",
@ -977,8 +977,8 @@
"tags": [
"Files"
],
"summary": "Delete a file.",
"description": "Delete a file.",
"summary": "Delete file.",
"description": "Delete file.",
"parameters": [
{
"name": "file_id",
@ -1022,8 +1022,8 @@
"tags": [
"Files"
],
"summary": "Returns the contents of the specified file.",
"description": "Returns the contents of the specified file.",
"summary": "Retrieve file content.",
"description": "Retrieve file content.\nReturns the contents of the specified file.",
"parameters": [
{
"name": "file_id",
@ -1067,8 +1067,8 @@
"tags": [
"Inspect"
],
"summary": "Get the current health status of the service.",
"description": "Get the current health status of the service.",
"summary": "Get health status.",
"description": "Get health status.\nGet the current health status of the service.",
"parameters": [],
"deprecated": false
}
@ -1102,8 +1102,8 @@
"tags": [
"Inspect"
],
"summary": "List all available API routes with their methods and implementing providers.",
"description": "List all available API routes with their methods and implementing providers.",
"summary": "List routes.",
"description": "List routes.\nList all available API routes with their methods and implementing providers.",
"parameters": [],
"deprecated": false
}
@ -1170,8 +1170,8 @@
"tags": [
"Models"
],
"summary": "Register a model.",
"description": "Register a model.",
"summary": "Register model.",
"description": "Register model.\nRegister a model.",
"parameters": [],
"requestBody": {
"content": {
@ -1215,8 +1215,8 @@
"tags": [
"Models"
],
"summary": "Get a model by its identifier.",
"description": "Get a model by its identifier.",
"summary": "Get model.",
"description": "Get model.\nGet a model by its identifier.",
"parameters": [
{
"name": "model_id",
@ -1251,8 +1251,8 @@
"tags": [
"Models"
],
"summary": "Unregister a model.",
"description": "Unregister a model.",
"summary": "Unregister model.",
"description": "Unregister model.\nUnregister a model.",
"parameters": [
{
"name": "model_id",
@ -1296,8 +1296,8 @@
"tags": [
"Safety"
],
"summary": "Classifies if text and/or image inputs are potentially harmful.",
"description": "Classifies if text and/or image inputs are potentially harmful.",
"summary": "Create moderation.",
"description": "Create moderation.\nClassifies if text and/or image inputs are potentially harmful.",
"parameters": [],
"requestBody": {
"content": {
@ -1374,8 +1374,8 @@
"tags": [
"Prompts"
],
"summary": "Create a new prompt.",
"description": "Create a new prompt.",
"summary": "Create prompt.",
"description": "Create prompt.\nCreate a new prompt.",
"parameters": [],
"requestBody": {
"content": {
@ -1419,8 +1419,8 @@
"tags": [
"Prompts"
],
"summary": "Get a prompt by its identifier and optional version.",
"description": "Get a prompt by its identifier and optional version.",
"summary": "Get prompt.",
"description": "Get prompt.\nGet a prompt by its identifier and optional version.",
"parameters": [
{
"name": "prompt_id",
@ -1471,8 +1471,8 @@
"tags": [
"Prompts"
],
"summary": "Update an existing prompt (increments version).",
"description": "Update an existing prompt (increments version).",
"summary": "Update prompt.",
"description": "Update prompt.\nUpdate an existing prompt (increments version).",
"parameters": [
{
"name": "prompt_id",
@ -1517,8 +1517,8 @@
"tags": [
"Prompts"
],
"summary": "Delete a prompt.",
"description": "Delete a prompt.",
"summary": "Delete prompt.",
"description": "Delete prompt.\nDelete a prompt.",
"parameters": [
{
"name": "prompt_id",
@ -1562,8 +1562,8 @@
"tags": [
"Prompts"
],
"summary": "Set which version of a prompt should be the default in get_prompt (latest).",
"description": "Set which version of a prompt should be the default in get_prompt (latest).",
"summary": "Set prompt version.",
"description": "Set prompt version.\nSet which version of a prompt should be the default in get_prompt (latest).",
"parameters": [
{
"name": "prompt_id",
@ -1617,8 +1617,8 @@
"tags": [
"Prompts"
],
"summary": "List all versions of a specific prompt.",
"description": "List all versions of a specific prompt.",
"summary": "List prompt versions.",
"description": "List prompt versions.\nList all versions of a specific prompt.",
"parameters": [
{
"name": "prompt_id",
@ -1662,8 +1662,8 @@
"tags": [
"Providers"
],
"summary": "List all available providers.",
"description": "List all available providers.",
"summary": "List providers.",
"description": "List providers.\nList all available providers.",
"parameters": [],
"deprecated": false
}
@ -1697,8 +1697,8 @@
"tags": [
"Providers"
],
"summary": "Get detailed information about a specific provider.",
"description": "Get detailed information about a specific provider.",
"summary": "Get provider.",
"description": "Get provider.\nGet detailed information about a specific provider.",
"parameters": [
{
"name": "provider_id",
@ -1742,8 +1742,8 @@
"tags": [
"Agents"
],
"summary": "List all OpenAI responses.",
"description": "List all OpenAI responses.",
"summary": "List all responses.",
"description": "List all responses.",
"parameters": [
{
"name": "after",
@ -1817,8 +1817,8 @@
"tags": [
"Agents"
],
"summary": "Create a new OpenAI response.",
"description": "Create a new OpenAI response.",
"summary": "Create a model response.",
"description": "Create a model response.",
"parameters": [],
"requestBody": {
"content": {
@ -1830,7 +1830,27 @@
},
"required": true
},
"deprecated": false
"deprecated": false,
"x-llama-stack-extra-body-params": [
{
"name": "shields",
"schema": {
"type": "array",
"items": {
"oneOf": [
{
"type": "string"
},
{
"$ref": "#/components/schemas/ResponseShieldSpec"
}
]
}
},
"description": "List of shields to apply during response generation. Shields provide safety and content moderation.",
"required": false
}
]
}
},
"/v1/responses/{response_id}": {
@ -1862,8 +1882,8 @@
"tags": [
"Agents"
],
"summary": "Retrieve an OpenAI response by its ID.",
"description": "Retrieve an OpenAI response by its ID.",
"summary": "Get a model response.",
"description": "Get a model response.",
"parameters": [
{
"name": "response_id",
@ -1905,8 +1925,8 @@
"tags": [
"Agents"
],
"summary": "Delete an OpenAI response by its ID.",
"description": "Delete an OpenAI response by its ID.",
"summary": "Delete a response.",
"description": "Delete a response.",
"parameters": [
{
"name": "response_id",
@ -1950,8 +1970,8 @@
"tags": [
"Agents"
],
"summary": "List input items for a given OpenAI response.",
"description": "List input items for a given OpenAI response.",
"summary": "List input items.",
"description": "List input items.",
"parameters": [
{
"name": "response_id",
@ -2043,8 +2063,8 @@
"tags": [
"Safety"
],
"summary": "Run a shield.",
"description": "Run a shield.",
"summary": "Run shield.",
"description": "Run shield.\nRun a shield.",
"parameters": [],
"requestBody": {
"content": {
@ -4176,8 +4196,8 @@
"tags": [
"Inspect"
],
"summary": "Get the version of the service.",
"description": "Get the version of the service.",
"summary": "Get version.",
"description": "Get version.\nGet the version of the service.",
"parameters": [],
"deprecated": false
}
@ -9625,6 +9645,21 @@
"title": "OpenAIResponseText",
"description": "Text response configuration for OpenAI responses."
},
"ResponseShieldSpec": {
"type": "object",
"properties": {
"type": {
"type": "string",
"description": "The type/identifier of the shield."
}
},
"additionalProperties": false,
"required": [
"type"
],
"title": "ResponseShieldSpec",
"description": "Specification for a shield to apply during response generation."
},
"OpenAIResponseInputTool": {
"oneOf": [
{
@ -18452,16 +18487,18 @@
},
{
"name": "Files",
"description": ""
"description": "This API is used to upload documents that can be used with other Llama Stack APIs.",
"x-displayName": "Files"
},
{
"name": "Inference",
"description": "This API provides the raw interface to the underlying models. Two kinds of models are supported:\n- LLM models: these models generate \"raw\" and \"chat\" (conversational) completions.\n- Embedding models: these models generate embeddings to be used for semantic search.",
"x-displayName": "Llama Stack Inference API for generating completions, chat completions, and embeddings."
"description": "Llama Stack Inference API for generating completions, chat completions, and embeddings.\n\nThis API provides the raw interface to the underlying models. Two kinds of models are supported:\n- LLM models: these models generate \"raw\" and \"chat\" (conversational) completions.\n- Embedding models: these models generate embeddings to be used for semantic search.",
"x-displayName": "Inference"
},
{
"name": "Inspect",
"description": ""
"description": "APIs for inspecting the Llama Stack service, including health status, available API routes with methods and implementing providers.",
"x-displayName": "Inspect"
},
{
"name": "Models",
@ -18473,17 +18510,18 @@
},
{
"name": "Prompts",
"description": "",
"x-displayName": "Protocol for prompt management operations."
"description": "Protocol for prompt management operations.",
"x-displayName": "Prompts"
},
{
"name": "Providers",
"description": "",
"x-displayName": "Providers API for inspecting, listing, and modifying providers and their configurations."
"description": "Providers API for inspecting, listing, and modifying providers and their configurations.",
"x-displayName": "Providers"
},
{
"name": "Safety",
"description": ""
"description": "OpenAI-compatible Moderations API.",
"x-displayName": "Safety"
},
{
"name": "Scoring",

227
docs/static/stainless-llama-stack-spec.yaml vendored
View file

@ -36,8 +36,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: List all chat completions.
description: List all chat completions.
summary: List chat completions.
description: List chat completions.
parameters:
- name: after
in: query
@ -90,10 +90,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: >-
Generate an OpenAI-compatible chat completion for the given messages using
the specified model.
summary: Create chat completions.
description: >-
Create chat completions.
Generate an OpenAI-compatible chat completion for the given messages using
the specified model.
parameters: []
@ -125,8 +125,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: Describe a chat completion by its ID.
description: Describe a chat completion by its ID.
summary: Get chat completion.
description: >-
Get chat completion.
Describe a chat completion by its ID.
parameters:
- name: completion_id
in: path
@ -156,10 +159,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: >-
Generate an OpenAI-compatible completion for the given prompt using the specified
model.
summary: Create completion.
description: >-
Create completion.
Generate an OpenAI-compatible completion for the given prompt using the specified
model.
parameters: []
@ -606,10 +609,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inference
summary: >-
Generate OpenAI-compatible embeddings for the given input using the specified
model.
summary: Create embeddings.
description: >-
Create embeddings.
Generate OpenAI-compatible embeddings for the given input using the specified
model.
parameters: []
@ -642,9 +645,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: >-
Returns a list of files that belong to the user's organization.
summary: List files.
description: >-
List files.
Returns a list of files that belong to the user's organization.
parameters:
- name: after
@ -702,11 +706,13 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: >-
Upload a file that can be used across various endpoints.
summary: Upload file.
description: >-
Upload file.
Upload a file that can be used across various endpoints.
The file upload should be a multipart form request with:
- file: The File object (not file name) to be uploaded.
@ -755,9 +761,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: >-
Returns information about a specific file.
summary: Retrieve file.
description: >-
Retrieve file.
Returns information about a specific file.
parameters:
- name: file_id
@ -789,8 +796,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: Delete a file.
description: Delete a file.
summary: Delete file.
description: Delete file.
parameters:
- name: file_id
in: path
@ -822,9 +829,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Files
summary: >-
Returns the contents of the specified file.
summary: Retrieve file content.
description: >-
Retrieve file content.
Returns the contents of the specified file.
parameters:
- name: file_id
@ -857,9 +865,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inspect
summary: >-
Get the current health status of the service.
summary: Get health status.
description: >-
Get health status.
Get the current health status of the service.
parameters: []
deprecated: false
@ -885,9 +894,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inspect
summary: >-
List all available API routes with their methods and implementing providers.
summary: List routes.
description: >-
List routes.
List all available API routes with their methods and implementing providers.
parameters: []
deprecated: false
@ -936,8 +946,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Models
summary: Register a model.
description: Register a model.
summary: Register model.
description: >-
Register model.
Register a model.
parameters: []
requestBody:
content:
@ -967,8 +980,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Models
summary: Get a model by its identifier.
description: Get a model by its identifier.
summary: Get model.
description: >-
Get model.
Get a model by its identifier.
parameters:
- name: model_id
in: path
@ -993,8 +1009,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Models
summary: Unregister a model.
description: Unregister a model.
summary: Unregister model.
description: >-
Unregister model.
Unregister a model.
parameters:
- name: model_id
in: path
@ -1025,9 +1044,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Safety
summary: >-
Classifies if text and/or image inputs are potentially harmful.
summary: Create moderation.
description: >-
Create moderation.
Classifies if text and/or image inputs are potentially harmful.
parameters: []
requestBody:
@ -1083,8 +1103,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Prompts
summary: Create a new prompt.
description: Create a new prompt.
summary: Create prompt.
description: >-
Create prompt.
Create a new prompt.
parameters: []
requestBody:
content:
@ -1114,9 +1137,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Prompts
summary: >-
Get a prompt by its identifier and optional version.
summary: Get prompt.
description: >-
Get prompt.
Get a prompt by its identifier and optional version.
parameters:
- name: prompt_id
@ -1154,9 +1178,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Prompts
summary: >-
Update an existing prompt (increments version).
summary: Update prompt.
description: >-
Update prompt.
Update an existing prompt (increments version).
parameters:
- name: prompt_id
@ -1188,8 +1213,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Prompts
summary: Delete a prompt.
description: Delete a prompt.
summary: Delete prompt.
description: >-
Delete prompt.
Delete a prompt.
parameters:
- name: prompt_id
in: path
@ -1220,9 +1248,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Prompts
summary: >-
Set which version of a prompt should be the default in get_prompt (latest).
summary: Set prompt version.
description: >-
Set prompt version.
Set which version of a prompt should be the default in get_prompt (latest).
parameters:
- name: prompt_id
@ -1260,8 +1289,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Prompts
summary: List all versions of a specific prompt.
description: List all versions of a specific prompt.
summary: List prompt versions.
description: >-
List prompt versions.
List all versions of a specific prompt.
parameters:
- name: prompt_id
in: path
@ -1293,8 +1325,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Providers
summary: List all available providers.
description: List all available providers.
summary: List providers.
description: >-
List providers.
List all available providers.
parameters: []
deprecated: false
/v1/providers/{provider_id}:
@ -1319,9 +1354,10 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Providers
summary: >-
Get detailed information about a specific provider.
summary: Get provider.
description: >-
Get provider.
Get detailed information about a specific provider.
parameters:
- name: provider_id
@ -1352,8 +1388,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: List all OpenAI responses.
description: List all OpenAI responses.
summary: List all responses.
description: List all responses.
parameters:
- name: after
in: query
@ -1404,8 +1440,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: Create a new OpenAI response.
description: Create a new OpenAI response.
summary: Create a model response.
description: Create a model response.
parameters: []
requestBody:
content:
@ -1414,6 +1450,18 @@ paths:
$ref: '#/components/schemas/CreateOpenaiResponseRequest'
required: true
deprecated: false
x-llama-stack-extra-body-params:
- name: shields
schema:
type: array
items:
oneOf:
- type: string
- $ref: '#/components/schemas/ResponseShieldSpec'
description: >-
List of shields to apply during response generation. Shields provide safety
and content moderation.
required: false
/v1/responses/{response_id}:
get:
responses:
@ -1435,8 +1483,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: Retrieve an OpenAI response by its ID.
description: Retrieve an OpenAI response by its ID.
summary: Get a model response.
description: Get a model response.
parameters:
- name: response_id
in: path
@ -1466,8 +1514,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: Delete an OpenAI response by its ID.
description: Delete an OpenAI response by its ID.
summary: Delete a response.
description: Delete a response.
parameters:
- name: response_id
in: path
@ -1497,10 +1545,8 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Agents
summary: >-
List input items for a given OpenAI response.
description: >-
List input items for a given OpenAI response.
summary: List input items.
description: List input items.
parameters:
- name: response_id
in: path
@ -1569,8 +1615,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Safety
summary: Run a shield.
description: Run a shield.
summary: Run shield.
description: >-
Run shield.
Run a shield.
parameters: []
requestBody:
content:
@ -3126,8 +3175,11 @@ paths:
$ref: '#/components/responses/DefaultError'
tags:
- Inspect
summary: Get the version of the service.
description: Get the version of the service.
summary: Get version.
description: >-
Get version.
Get the version of the service.
parameters: []
deprecated: false
/v1beta/datasetio/append-rows/{dataset_id}:
@ -7184,6 +7236,18 @@ components:
title: OpenAIResponseText
description: >-
Text response configuration for OpenAI responses.
ResponseShieldSpec:
type: object
properties:
type:
type: string
description: The type/identifier of the shield.
additionalProperties: false
required:
- type
title: ResponseShieldSpec
description: >-
Specification for a shield to apply during response generation.
OpenAIResponseInputTool:
oneOf:
- $ref: '#/components/schemas/OpenAIResponseInputToolWebSearch'
@ -13771,9 +13835,16 @@ tags:
x-displayName: >-
Llama Stack Evaluation API for running evaluations on model and agent candidates.
- name: Files
description: ''
description: >-
This API is used to upload documents that can be used with other Llama Stack
APIs.
x-displayName: Files
- name: Inference
description: >-
Llama Stack Inference API for generating completions, chat completions, and
embeddings.
This API provides the raw interface to the underlying models. Two kinds of models
are supported:
@ -13781,25 +13852,27 @@ tags:
- Embedding models: these models generate embeddings to be used for semantic
search.
x-displayName: >-
Llama Stack Inference API for generating completions, chat completions, and
embeddings.
x-displayName: Inference
- name: Inspect
description: ''
description: >-
APIs for inspecting the Llama Stack service, including health status, available
API routes with methods and implementing providers.
x-displayName: Inspect
- name: Models
description: ''
- name: PostTraining (Coming Soon)
description: ''
- name: Prompts
description: ''
x-displayName: >-
description: >-
Protocol for prompt management operations.
x-displayName: Prompts
- name: Providers
description: ''
x-displayName: >-
description: >-
Providers API for inspecting, listing, and modifying providers and their configurations.
x-displayName: Providers
- name: Safety
description: ''
description: OpenAI-compatible Moderations API.
x-displayName: Safety
- name: Scoring
description: ''
- name: ScoringFunctions