phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-06-27 18:50:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

API Updates (#73)

Browse source 
* API Keys passed from Client instead of distro configuration

* delete distribution registry

* Rename the "package" word away

* Introduce a "Router" layer for providers

Some providers need to be factorized and considered as thin routing
layers on top of other providers. Consider two examples:

- The inference API should be a routing layer over inference providers,
  routed using the "model" key
- The memory banks API is another instance where various memory bank
  types will be provided by independent providers (e.g., a vector store
  is served by Chroma while a keyvalue memory can be served by Redis or
  PGVector)

This commit introduces a generalized routing layer for this purpose.

* update `apis_to_serve`

* llama_toolchain -> llama_stack

* Codemod from llama_toolchain -> llama_stack

- added providers/registry
- cleaned up api/ subdirectories and moved impls away
- restructured api/api.py
- from llama_stack.apis.<api> import foo should work now
- update imports to do llama_stack.apis.<api>
- update many other imports
- added __init__, fixed some registry imports
- updated registry imports
- create_agentic_system -> create_agent
- AgenticSystem -> Agent

* Moved some stuff out of common/; re-generated OpenAPI spec

* llama-toolchain -> llama-stack (hyphens)

* add control plane API

* add redis adapter + sqlite provider

* move core -> distribution

* Some more toolchain -> stack changes

* small naming shenanigans

* Removing custom tool and agent utilities and moving them client side

* Move control plane to distribution server for now

* Remove control plane from API list

* no codeshield dependency randomly plzzzzz

* Add "fire" as a dependency

* add back event loggers

* stack configure fixes

* use brave instead of bing in the example client

* add init file so it gets packaged

* add init files so it gets packaged

* Update MANIFEST

* bug fix

---------

Co-authored-by: Hardik Shah <hjshah@fb.com>
Co-authored-by: Xi Yan <xiyan@meta.com>
Co-authored-by: Ashwin Bharambe <ashwin@meta.com>
This commit is contained in:
Ashwin Bharambe 2024-09-17 19:51:35 -07:00 • committed by GitHub
parent f294eac5f5
commit 9487ad8294
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
213 changed files with 1725 additions and 1204 deletions
Download patch file Download diff file Expand all files Collapse all files

225
rfcs/RFC-0001-llama-stack-assets/llama-stack-spec.html
View file

@ -21,7 +21,7 @@
"info": {
"title": "[DRAFT] Llama Stack Specification",
"version": "0.0.1",
"description": "This is the specification of the llama stack that provides\n a set of endpoints and their corresponding interfaces that are tailored to\n best leverage Llama Models. The specification is still in draft and subject to change.\n Generated at 2024-09-11 16:05:23.016090"
"description": "This is the specification of the llama stack that provides\n a set of endpoints and their corresponding interfaces that are tailored to\n best leverage Llama Models. The specification is still in draft and subject to change.\n Generated at 2024-09-17 12:55:45.538053"
},
"servers": [
{
@ -209,7 +209,7 @@
}
}
},
"/agentic_system/create": {
"/agents/create": {
"post": {
"responses": {
"200": {
@ -217,21 +217,21 @@
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/AgenticSystemCreateResponse"
"$ref": "#/components/schemas/AgentCreateResponse"
}
}
}
}
},
"tags": [
"AgenticSystem"
"Agents"
],
"parameters": [],
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CreateAgenticSystemRequest"
"$ref": "#/components/schemas/CreateAgentRequest"
}
}
},
@ -239,7 +239,7 @@
}
}
},
"/agentic_system/session/create": {
"/agents/session/create": {
"post": {
"responses": {
"200": {
@ -247,21 +247,21 @@
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/AgenticSystemSessionCreateResponse"
"$ref": "#/components/schemas/AgentSessionCreateResponse"
}
}
}
}
},
"tags": [
"AgenticSystem"
"Agents"
],
"parameters": [],
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CreateAgenticSystemSessionRequest"
"$ref": "#/components/schemas/CreateAgentSessionRequest"
}
}
},
@ -269,29 +269,29 @@
}
}
},
"/agentic_system/turn/create": {
"/agents/turn/create": {
"post": {
"responses": {
"200": {
"description": "OK",
"content": {
"text/event-stream": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/AgenticSystemTurnResponseStreamChunk"
"$ref": "#/components/schemas/AgentTurnResponseStreamChunk"
}
}
}
}
},
"tags": [
"AgenticSystem"
"Agents"
],
"parameters": [],
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CreateAgenticSystemTurnRequest"
"$ref": "#/components/schemas/CreateAgentTurnRequest"
}
}
},
@ -352,7 +352,7 @@
}
}
},
"/agentic_system/delete": {
"/agents/delete": {
"post": {
"responses": {
"200": {
@ -360,14 +360,14 @@
}
},
"tags": [
"AgenticSystem"
"Agents"
],
"parameters": [],
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/DeleteAgenticSystemRequest"
"$ref": "#/components/schemas/DeleteAgentsRequest"
}
}
},
@ -375,7 +375,7 @@
}
}
},
"/agentic_system/session/delete": {
"/agents/session/delete": {
"post": {
"responses": {
"200": {
@ -383,14 +383,14 @@
}
},
"tags": [
"AgenticSystem"
"Agents"
],
"parameters": [],
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/DeleteAgenticSystemSessionRequest"
"$ref": "#/components/schemas/DeleteAgentsSessionRequest"
}
}
},
@ -594,7 +594,7 @@
}
}
},
"/agentic_system/session/get": {
"/agents/session/get": {
"post": {
"responses": {
"200": {
@ -609,7 +609,7 @@
}
},
"tags": [
"AgenticSystem"
"Agents"
],
"parameters": [
{
@ -633,7 +633,7 @@
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/GetAgenticSystemSessionRequest"
"$ref": "#/components/schemas/GetAgentsSessionRequest"
}
}
},
@ -641,7 +641,7 @@
}
}
},
"/agentic_system/step/get": {
"/agents/step/get": {
"get": {
"responses": {
"200": {
@ -649,14 +649,14 @@
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/AgenticSystemStepResponse"
"$ref": "#/components/schemas/AgentStepResponse"
}
}
}
}
},
"tags": [
"AgenticSystem"
"Agents"
],
"parameters": [
{
@ -686,7 +686,7 @@
]
}
},
"/agentic_system/turn/get": {
"/agents/turn/get": {
"get": {
"responses": {
"200": {
@ -701,7 +701,7 @@
}
},
"tags": [
"AgenticSystem"
"Agents"
],
"parameters": [
{
@ -2672,7 +2672,7 @@
"type"
]
},
"CreateAgenticSystemRequest": {
"CreateAgentRequest": {
"type": "object",
"properties": {
"agent_config": {
@ -2684,7 +2684,7 @@
"agent_config"
]
},
"AgenticSystemCreateResponse": {
"AgentCreateResponse": {
"type": "object",
"properties": {
"agent_id": {
@ -2696,7 +2696,7 @@
"agent_id"
]
},
"CreateAgenticSystemSessionRequest": {
"CreateAgentSessionRequest": {
"type": "object",
"properties": {
"agent_id": {
@ -2712,7 +2712,7 @@
"session_name"
]
},
"AgenticSystemSessionCreateResponse": {
"AgentSessionCreateResponse": {
"type": "object",
"properties": {
"session_id": {
@ -2753,7 +2753,7 @@
"mime_type"
]
},
"CreateAgenticSystemTurnRequest": {
"CreateAgentTurnRequest": {
"type": "object",
"properties": {
"agent_id": {
@ -2792,25 +2792,25 @@
"messages"
]
},
"AgenticSystemTurnResponseEvent": {
"AgentTurnResponseEvent": {
"type": "object",
"properties": {
"payload": {
"oneOf": [
{
"$ref": "#/components/schemas/AgenticSystemTurnResponseStepStartPayload"
"$ref": "#/components/schemas/AgentTurnResponseStepStartPayload"
},
{
"$ref": "#/components/schemas/AgenticSystemTurnResponseStepProgressPayload"
"$ref": "#/components/schemas/AgentTurnResponseStepProgressPayload"
},
{
"$ref": "#/components/schemas/AgenticSystemTurnResponseStepCompletePayload"
"$ref": "#/components/schemas/AgentTurnResponseStepCompletePayload"
},
{
"$ref": "#/components/schemas/AgenticSystemTurnResponseTurnStartPayload"
"$ref": "#/components/schemas/AgentTurnResponseTurnStartPayload"
},
{
"$ref": "#/components/schemas/AgenticSystemTurnResponseTurnCompletePayload"
"$ref": "#/components/schemas/AgentTurnResponseTurnCompletePayload"
}
]
}
@ -2821,7 +2821,7 @@
],
"title": "Streamed agent execution response."
},
"AgenticSystemTurnResponseStepCompletePayload": {
"AgentTurnResponseStepCompletePayload": {
"type": "object",
"properties": {
"event_type": {
@ -2861,7 +2861,7 @@
"step_details"
]
},
"AgenticSystemTurnResponseStepProgressPayload": {
"AgentTurnResponseStepProgressPayload": {
"type": "object",
"properties": {
"event_type": {
@ -2897,7 +2897,7 @@
"step_id"
]
},
"AgenticSystemTurnResponseStepStartPayload": {
"AgentTurnResponseStepStartPayload": {
"type": "object",
"properties": {
"event_type": {
@ -2949,11 +2949,11 @@
"step_id"
]
},
"AgenticSystemTurnResponseStreamChunk": {
"AgentTurnResponseStreamChunk": {
"type": "object",
"properties": {
"event": {
"$ref": "#/components/schemas/AgenticSystemTurnResponseEvent"
"$ref": "#/components/schemas/AgentTurnResponseEvent"
}
},
"additionalProperties": false,
@ -2961,7 +2961,7 @@
"event"
]
},
"AgenticSystemTurnResponseTurnCompletePayload": {
"AgentTurnResponseTurnCompletePayload": {
"type": "object",
"properties": {
"event_type": {
@ -2978,7 +2978,7 @@
"turn"
]
},
"AgenticSystemTurnResponseTurnStartPayload": {
"AgentTurnResponseTurnStartPayload": {
"type": "object",
"properties": {
"event_type": {
@ -3532,7 +3532,7 @@
"config"
]
},
"DeleteAgenticSystemRequest": {
"DeleteAgentsRequest": {
"type": "object",
"properties": {
"agent_id": {
@ -3544,7 +3544,7 @@
"agent_id"
]
},
"DeleteAgenticSystemSessionRequest": {
"DeleteAgentsSessionRequest": {
"type": "object",
"properties": {
"agent_id": {
@ -3720,7 +3720,7 @@
"metrics"
]
},
"GetAgenticSystemSessionRequest": {
"GetAgentsSessionRequest": {
"type": "object",
"properties": {
"turn_ids": {
@ -3764,7 +3764,7 @@
],
"title": "A single session of an interaction with an Agentic System."
},
"AgenticSystemStepResponse": {
"AgentStepResponse": {
"type": "object",
"properties": {
"step": {
@ -3859,7 +3859,6 @@
"required": [
"document_id",
"content",
"mime_type",
"metadata"
]
},
@ -5142,37 +5141,37 @@
],
"tags": [
{
"name": "SyntheticDataGeneration"
},
{
"name": "Datasets"
},
{
"name": "Evaluations"
"name": "Agents"
},
{
"name": "Safety"
},
{
"name": "Inference"
"name": "SyntheticDataGeneration"
},
{
"name": "Telemetry"
},
{
"name": "PostTraining"
},
{
"name": "Memory"
"name": "Datasets"
},
{
"name": "RewardScoring"
},
{
"name": "Evaluations"
},
{
"name": "PostTraining"
},
{
"name": "Inference"
},
{
"name": "BatchInference"
},
{
"name": "AgenticSystem"
"name": "Memory"
},
{
"name": "BuiltinTool",
@ -5343,56 +5342,56 @@
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/WolframAlphaToolDefinition\" />"
},
{
"name": "CreateAgenticSystemRequest",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/CreateAgenticSystemRequest\" />"
"name": "CreateAgentRequest",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/CreateAgentRequest\" />"
},
{
"name": "AgenticSystemCreateResponse",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgenticSystemCreateResponse\" />"
"name": "AgentCreateResponse",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgentCreateResponse\" />"
},
{
"name": "CreateAgenticSystemSessionRequest",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/CreateAgenticSystemSessionRequest\" />"
"name": "CreateAgentSessionRequest",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/CreateAgentSessionRequest\" />"
},
{
"name": "AgenticSystemSessionCreateResponse",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgenticSystemSessionCreateResponse\" />"
"name": "AgentSessionCreateResponse",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgentSessionCreateResponse\" />"
},
{
"name": "Attachment",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/Attachment\" />"
},
{
"name": "CreateAgenticSystemTurnRequest",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/CreateAgenticSystemTurnRequest\" />"
"name": "CreateAgentTurnRequest",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/CreateAgentTurnRequest\" />"
},
{
"name": "AgenticSystemTurnResponseEvent",
"description": "Streamed agent execution response.\n\n<SchemaDefinition schemaRef=\"#/components/schemas/AgenticSystemTurnResponseEvent\" />"
"name": "AgentTurnResponseEvent",
"description": "Streamed agent execution response.\n\n<SchemaDefinition schemaRef=\"#/components/schemas/AgentTurnResponseEvent\" />"
},
{
"name": "AgenticSystemTurnResponseStepCompletePayload",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgenticSystemTurnResponseStepCompletePayload\" />"
"name": "AgentTurnResponseStepCompletePayload",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgentTurnResponseStepCompletePayload\" />"
},
{
"name": "AgenticSystemTurnResponseStepProgressPayload",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgenticSystemTurnResponseStepProgressPayload\" />"
"name": "AgentTurnResponseStepProgressPayload",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgentTurnResponseStepProgressPayload\" />"
},
{
"name": "AgenticSystemTurnResponseStepStartPayload",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgenticSystemTurnResponseStepStartPayload\" />"
"name": "AgentTurnResponseStepStartPayload",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgentTurnResponseStepStartPayload\" />"
},
{
"name": "AgenticSystemTurnResponseStreamChunk",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgenticSystemTurnResponseStreamChunk\" />"
"name": "AgentTurnResponseStreamChunk",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgentTurnResponseStreamChunk\" />"
},
{
"name": "AgenticSystemTurnResponseTurnCompletePayload",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgenticSystemTurnResponseTurnCompletePayload\" />"
"name": "AgentTurnResponseTurnCompletePayload",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgentTurnResponseTurnCompletePayload\" />"
},
{
"name": "AgenticSystemTurnResponseTurnStartPayload",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgenticSystemTurnResponseTurnStartPayload\" />"
"name": "AgentTurnResponseTurnStartPayload",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgentTurnResponseTurnStartPayload\" />"
},
{
"name": "InferenceStep",
@ -5443,12 +5442,12 @@
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/MemoryBank\" />"
},
{
"name": "DeleteAgenticSystemRequest",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/DeleteAgenticSystemRequest\" />"
"name": "DeleteAgentsRequest",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/DeleteAgentsRequest\" />"
},
{
"name": "DeleteAgenticSystemSessionRequest",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/DeleteAgenticSystemSessionRequest\" />"
"name": "DeleteAgentsSessionRequest",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/DeleteAgentsSessionRequest\" />"
},
{
"name": "DeleteDatasetRequest",
@ -5487,16 +5486,16 @@
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/EvaluateTextGenerationRequest\" />"
},
{
"name": "GetAgenticSystemSessionRequest",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/GetAgenticSystemSessionRequest\" />"
"name": "GetAgentsSessionRequest",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/GetAgentsSessionRequest\" />"
},
{
"name": "Session",
"description": "A single session of an interaction with an Agentic System.\n\n<SchemaDefinition schemaRef=\"#/components/schemas/Session\" />"
},
{
"name": "AgenticSystemStepResponse",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgenticSystemStepResponse\" />"
"name": "AgentStepResponse",
"description": "<SchemaDefinition schemaRef=\"#/components/schemas/AgentStepResponse\" />"
},
{
"name": "GetDocumentsRequest",
@ -5675,7 +5674,7 @@
{
"name": "Operations",
"tags": [
"AgenticSystem",
"Agents",
"BatchInference",
"Datasets",
"Evaluations",
@ -5692,16 +5691,16 @@
"name": "Types",
"tags": [
"AgentConfig",
"AgenticSystemCreateResponse",
"AgenticSystemSessionCreateResponse",
"AgenticSystemStepResponse",
"AgenticSystemTurnResponseEvent",
"AgenticSystemTurnResponseStepCompletePayload",
"AgenticSystemTurnResponseStepProgressPayload",
"AgenticSystemTurnResponseStepStartPayload",
"AgenticSystemTurnResponseStreamChunk",
"AgenticSystemTurnResponseTurnCompletePayload",
"AgenticSystemTurnResponseTurnStartPayload",
"AgentCreateResponse",
"AgentSessionCreateResponse",
"AgentStepResponse",
"AgentTurnResponseEvent",
"AgentTurnResponseStepCompletePayload",
"AgentTurnResponseStepProgressPayload",
"AgentTurnResponseStepStartPayload",
"AgentTurnResponseStreamChunk",
"AgentTurnResponseTurnCompletePayload",
"AgentTurnResponseTurnStartPayload",
"Attachment",
"BatchChatCompletionRequest",
"BatchChatCompletionResponse",
@ -5722,14 +5721,14 @@
"CompletionRequest",
"CompletionResponse",
"CompletionResponseStreamChunk",
"CreateAgenticSystemRequest",
"CreateAgenticSystemSessionRequest",
"CreateAgenticSystemTurnRequest",
"CreateAgentRequest",
"CreateAgentSessionRequest",
"CreateAgentTurnRequest",
"CreateDatasetRequest",
"CreateMemoryBankRequest",
"DPOAlignmentConfig",
"DeleteAgenticSystemRequest",
"DeleteAgenticSystemSessionRequest",
"DeleteAgentsRequest",
"DeleteAgentsSessionRequest",
"DeleteDatasetRequest",
"DeleteDocumentsRequest",
"DialogGenerations",
@ -5746,7 +5745,7 @@
"EvaluationJobStatusResponse",
"FinetuningAlgorithm",
"FunctionCallToolDefinition",
"GetAgenticSystemSessionRequest",
"GetAgentsSessionRequest",
"GetDocumentsRequest",
"InferenceStep",
"InsertDocumentsRequest",

214
rfcs/RFC-0001-llama-stack-assets/llama-stack-spec.yaml
View file

@ -152,7 +152,7 @@ components:
- model
- instructions
type: object
AgenticSystemCreateResponse:
AgentCreateResponse:
additionalProperties: false
properties:
agent_id:
@ -160,7 +160,7 @@ components:
required:
- agent_id
type: object
AgenticSystemSessionCreateResponse:
AgentSessionCreateResponse:
additionalProperties: false
properties:
session_id:
@ -168,7 +168,7 @@ components:
required:
- session_id
type: object
AgenticSystemStepResponse:
AgentStepResponse:
additionalProperties: false
properties:
step:
@ -180,21 +180,21 @@ components:
required:
- step
type: object
AgenticSystemTurnResponseEvent:
AgentTurnResponseEvent:
additionalProperties: false
properties:
payload:
oneOf:
- $ref: '#/components/schemas/AgenticSystemTurnResponseStepStartPayload'
- $ref: '#/components/schemas/AgenticSystemTurnResponseStepProgressPayload'
- $ref: '#/components/schemas/AgenticSystemTurnResponseStepCompletePayload'
- $ref: '#/components/schemas/AgenticSystemTurnResponseTurnStartPayload'
- $ref: '#/components/schemas/AgenticSystemTurnResponseTurnCompletePayload'
- $ref: '#/components/schemas/AgentTurnResponseStepStartPayload'
- $ref: '#/components/schemas/AgentTurnResponseStepProgressPayload'
- $ref: '#/components/schemas/AgentTurnResponseStepCompletePayload'
- $ref: '#/components/schemas/AgentTurnResponseTurnStartPayload'
- $ref: '#/components/schemas/AgentTurnResponseTurnCompletePayload'
required:
- payload
title: Streamed agent execution response.
type: object
AgenticSystemTurnResponseStepCompletePayload:
AgentTurnResponseStepCompletePayload:
additionalProperties: false
properties:
event_type:
@ -218,7 +218,7 @@ components:
- step_type
- step_details
type: object
AgenticSystemTurnResponseStepProgressPayload:
AgentTurnResponseStepProgressPayload:
additionalProperties: false
properties:
event_type:
@ -244,7 +244,7 @@ components:
- step_type
- step_id
type: object
AgenticSystemTurnResponseStepStartPayload:
AgentTurnResponseStepStartPayload:
additionalProperties: false
properties:
event_type:
@ -274,15 +274,15 @@ components:
- step_type
- step_id
type: object
AgenticSystemTurnResponseStreamChunk:
AgentTurnResponseStreamChunk:
additionalProperties: false
properties:
event:
$ref: '#/components/schemas/AgenticSystemTurnResponseEvent'
$ref: '#/components/schemas/AgentTurnResponseEvent'
required:
- event
type: object
AgenticSystemTurnResponseTurnCompletePayload:
AgentTurnResponseTurnCompletePayload:
additionalProperties: false
properties:
event_type:
@ -294,7 +294,7 @@ components:
- event_type
- turn
type: object
AgenticSystemTurnResponseTurnStartPayload:
AgentTurnResponseTurnStartPayload:
additionalProperties: false
properties:
event_type:
@ -617,7 +617,7 @@ components:
- delta
title: streamed completion response.
type: object
CreateAgenticSystemRequest:
CreateAgentRequest:
additionalProperties: false
properties:
agent_config:
@ -625,7 +625,7 @@ components:
required:
- agent_config
type: object
CreateAgenticSystemSessionRequest:
CreateAgentSessionRequest:
additionalProperties: false
properties:
agent_id:
@ -636,7 +636,7 @@ components:
- agent_id
- session_name
type: object
CreateAgenticSystemTurnRequest:
CreateAgentTurnRequest:
additionalProperties: false
properties:
agent_id:
@ -741,7 +741,7 @@ components:
- epsilon
- gamma
type: object
DeleteAgenticSystemRequest:
DeleteAgentsRequest:
additionalProperties: false
properties:
agent_id:
@ -749,7 +749,7 @@ components:
required:
- agent_id
type: object
DeleteAgenticSystemSessionRequest:
DeleteAgentsSessionRequest:
additionalProperties: false
properties:
agent_id:
@ -973,7 +973,7 @@ components:
- description
- parameters
type: object
GetAgenticSystemSessionRequest:
GetAgentsSessionRequest:
additionalProperties: false
properties:
turn_ids:
@ -1155,7 +1155,6 @@ components:
required:
- document_id
- content
- mime_type
- metadata
type: object
MemoryRetrievalStep:
@ -2357,77 +2356,77 @@ info:
description: "This is the specification of the llama stack that provides\n \
\ a set of endpoints and their corresponding interfaces that are tailored\
\ to\n best leverage Llama Models. The specification is still in\
\ draft and subject to change.\n Generated at 2024-09-11 16:05:23.016090"
\ draft and subject to change.\n Generated at 2024-09-17 12:55:45.538053"
title: '[DRAFT] Llama Stack Specification'
version: 0.0.1
jsonSchemaDialect: https://json-schema.org/draft/2020-12/schema
openapi: 3.1.0
paths:
/agentic_system/create:
/agents/create:
post:
parameters: []
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/CreateAgenticSystemRequest'
$ref: '#/components/schemas/CreateAgentRequest'
required: true
responses:
'200':
content:
application/json:
schema:
$ref: '#/components/schemas/AgenticSystemCreateResponse'
$ref: '#/components/schemas/AgentCreateResponse'
description: OK
tags:
- AgenticSystem
/agentic_system/delete:
- Agents
/agents/delete:
post:
parameters: []
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/DeleteAgenticSystemRequest'
$ref: '#/components/schemas/DeleteAgentsRequest'
required: true
responses:
'200':
description: OK
tags:
- AgenticSystem
/agentic_system/session/create:
- Agents
/agents/session/create:
post:
parameters: []
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/CreateAgenticSystemSessionRequest'
$ref: '#/components/schemas/CreateAgentSessionRequest'
required: true
responses:
'200':
content:
application/json:
schema:
$ref: '#/components/schemas/AgenticSystemSessionCreateResponse'
$ref: '#/components/schemas/AgentSessionCreateResponse'
description: OK
tags:
- AgenticSystem
/agentic_system/session/delete:
- Agents
/agents/session/delete:
post:
parameters: []
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/DeleteAgenticSystemSessionRequest'
$ref: '#/components/schemas/DeleteAgentsSessionRequest'
required: true
responses:
'200':
description: OK
tags:
- AgenticSystem
/agentic_system/session/get:
- Agents
/agents/session/get:
post:
parameters:
- in: query
@ -2444,7 +2443,7 @@ paths:
content:
application/json:
schema:
$ref: '#/components/schemas/GetAgenticSystemSessionRequest'
$ref: '#/components/schemas/GetAgentsSessionRequest'
required: true
responses:
'200':
@ -2454,8 +2453,8 @@ paths:
$ref: '#/components/schemas/Session'
description: OK
tags:
- AgenticSystem
/agentic_system/step/get:
- Agents
/agents/step/get:
get:
parameters:
- in: query
@ -2478,29 +2477,29 @@ paths:
content:
application/json:
schema:
$ref: '#/components/schemas/AgenticSystemStepResponse'
$ref: '#/components/schemas/AgentStepResponse'
description: OK
tags:
- AgenticSystem
/agentic_system/turn/create:
- Agents
/agents/turn/create:
post:
parameters: []
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/CreateAgenticSystemTurnRequest'
$ref: '#/components/schemas/CreateAgentTurnRequest'
required: true
responses:
'200':
content:
text/event-stream:
application/json:
schema:
$ref: '#/components/schemas/AgenticSystemTurnResponseStreamChunk'
$ref: '#/components/schemas/AgentTurnResponseStreamChunk'
description: OK
tags:
- AgenticSystem
/agentic_system/turn/get:
- Agents
/agents/turn/get:
get:
parameters:
- in: query
@ -2521,7 +2520,7 @@ paths:
$ref: '#/components/schemas/Turn'
description: OK
tags:
- AgenticSystem
- Agents
/batch_inference/chat_completion:
post:
parameters: []
@ -3145,17 +3144,17 @@ security:
servers:
- url: http://any-hosted-llama-stack.com
tags:
- name: SyntheticDataGeneration
- name: Datasets
- name: Evaluations
- name: Agents
- name: Safety
- name: Inference
- name: SyntheticDataGeneration
- name: Telemetry
- name: PostTraining
- name: Memory
- name: Datasets
- name: RewardScoring
- name: Evaluations
- name: PostTraining
- name: Inference
- name: BatchInference
- name: AgenticSystem
- name: Memory
- description: <SchemaDefinition schemaRef="#/components/schemas/BuiltinTool" />
name: BuiltinTool
- description: <SchemaDefinition schemaRef="#/components/schemas/CompletionMessage"
@ -3289,47 +3288,46 @@ tags:
- description: <SchemaDefinition schemaRef="#/components/schemas/WolframAlphaToolDefinition"
/>
name: WolframAlphaToolDefinition
- description: <SchemaDefinition schemaRef="#/components/schemas/CreateAgenticSystemRequest"
- description: <SchemaDefinition schemaRef="#/components/schemas/CreateAgentRequest"
/>
name: CreateAgenticSystemRequest
- description: <SchemaDefinition schemaRef="#/components/schemas/AgenticSystemCreateResponse"
name: CreateAgentRequest
- description: <SchemaDefinition schemaRef="#/components/schemas/AgentCreateResponse"
/>
name: AgenticSystemCreateResponse
- description: <SchemaDefinition schemaRef="#/components/schemas/CreateAgenticSystemSessionRequest"
name: AgentCreateResponse
- description: <SchemaDefinition schemaRef="#/components/schemas/CreateAgentSessionRequest"
/>
name: CreateAgenticSystemSessionRequest
- description: <SchemaDefinition schemaRef="#/components/schemas/AgenticSystemSessionCreateResponse"
name: CreateAgentSessionRequest
- description: <SchemaDefinition schemaRef="#/components/schemas/AgentSessionCreateResponse"
/>
name: AgenticSystemSessionCreateResponse
name: AgentSessionCreateResponse
- description: <SchemaDefinition schemaRef="#/components/schemas/Attachment" />
name: Attachment
- description: <SchemaDefinition schemaRef="#/components/schemas/CreateAgenticSystemTurnRequest"
- description: <SchemaDefinition schemaRef="#/components/schemas/CreateAgentTurnRequest"
/>
name: CreateAgenticSystemTurnRequest
name: CreateAgentTurnRequest
- description: 'Streamed agent execution response.
<SchemaDefinition schemaRef="#/components/schemas/AgenticSystemTurnResponseEvent"
/>'
name: AgenticSystemTurnResponseEvent
- description: <SchemaDefinition schemaRef="#/components/schemas/AgenticSystemTurnResponseStepCompletePayload"
<SchemaDefinition schemaRef="#/components/schemas/AgentTurnResponseEvent" />'
name: AgentTurnResponseEvent
- description: <SchemaDefinition schemaRef="#/components/schemas/AgentTurnResponseStepCompletePayload"
/>
name: AgenticSystemTurnResponseStepCompletePayload
- description: <SchemaDefinition schemaRef="#/components/schemas/AgenticSystemTurnResponseStepProgressPayload"
name: AgentTurnResponseStepCompletePayload
- description: <SchemaDefinition schemaRef="#/components/schemas/AgentTurnResponseStepProgressPayload"
/>
name: AgenticSystemTurnResponseStepProgressPayload
- description: <SchemaDefinition schemaRef="#/components/schemas/AgenticSystemTurnResponseStepStartPayload"
name: AgentTurnResponseStepProgressPayload
- description: <SchemaDefinition schemaRef="#/components/schemas/AgentTurnResponseStepStartPayload"
/>
name: AgenticSystemTurnResponseStepStartPayload
- description: <SchemaDefinition schemaRef="#/components/schemas/AgenticSystemTurnResponseStreamChunk"
name: AgentTurnResponseStepStartPayload
- description: <SchemaDefinition schemaRef="#/components/schemas/AgentTurnResponseStreamChunk"
/>
name: AgenticSystemTurnResponseStreamChunk
- description: <SchemaDefinition schemaRef="#/components/schemas/AgenticSystemTurnResponseTurnCompletePayload"
name: AgentTurnResponseStreamChunk
- description: <SchemaDefinition schemaRef="#/components/schemas/AgentTurnResponseTurnCompletePayload"
/>
name: AgenticSystemTurnResponseTurnCompletePayload
- description: <SchemaDefinition schemaRef="#/components/schemas/AgenticSystemTurnResponseTurnStartPayload"
name: AgentTurnResponseTurnCompletePayload
- description: <SchemaDefinition schemaRef="#/components/schemas/AgentTurnResponseTurnStartPayload"
/>
name: AgenticSystemTurnResponseTurnStartPayload
name: AgentTurnResponseTurnStartPayload
- description: <SchemaDefinition schemaRef="#/components/schemas/InferenceStep" />
name: InferenceStep
- description: <SchemaDefinition schemaRef="#/components/schemas/MemoryRetrievalStep"
@ -3365,12 +3363,12 @@ tags:
name: CreateMemoryBankRequest
- description: <SchemaDefinition schemaRef="#/components/schemas/MemoryBank" />
name: MemoryBank
- description: <SchemaDefinition schemaRef="#/components/schemas/DeleteAgenticSystemRequest"
- description: <SchemaDefinition schemaRef="#/components/schemas/DeleteAgentsRequest"
/>
name: DeleteAgenticSystemRequest
- description: <SchemaDefinition schemaRef="#/components/schemas/DeleteAgenticSystemSessionRequest"
name: DeleteAgentsRequest
- description: <SchemaDefinition schemaRef="#/components/schemas/DeleteAgentsSessionRequest"
/>
name: DeleteAgenticSystemSessionRequest
name: DeleteAgentsSessionRequest
- description: <SchemaDefinition schemaRef="#/components/schemas/DeleteDatasetRequest"
/>
name: DeleteDatasetRequest
@ -3397,17 +3395,17 @@ tags:
- description: <SchemaDefinition schemaRef="#/components/schemas/EvaluateTextGenerationRequest"
/>
name: EvaluateTextGenerationRequest
- description: <SchemaDefinition schemaRef="#/components/schemas/GetAgenticSystemSessionRequest"
- description: <SchemaDefinition schemaRef="#/components/schemas/GetAgentsSessionRequest"
/>
name: GetAgenticSystemSessionRequest
name: GetAgentsSessionRequest
- description: 'A single session of an interaction with an Agentic System.
<SchemaDefinition schemaRef="#/components/schemas/Session" />'
name: Session
- description: <SchemaDefinition schemaRef="#/components/schemas/AgenticSystemStepResponse"
- description: <SchemaDefinition schemaRef="#/components/schemas/AgentStepResponse"
/>
name: AgenticSystemStepResponse
name: AgentStepResponse
- description: <SchemaDefinition schemaRef="#/components/schemas/GetDocumentsRequest"
/>
name: GetDocumentsRequest
@ -3552,7 +3550,7 @@ tags:
x-tagGroups:
- name: Operations
tags:
- AgenticSystem
- Agents
- BatchInference
- Datasets
- Evaluations
@ -3566,16 +3564,16 @@ x-tagGroups:
- name: Types
tags:
- AgentConfig
- AgenticSystemCreateResponse
- AgenticSystemSessionCreateResponse
- AgenticSystemStepResponse
- AgenticSystemTurnResponseEvent
- AgenticSystemTurnResponseStepCompletePayload
- AgenticSystemTurnResponseStepProgressPayload
- AgenticSystemTurnResponseStepStartPayload
- AgenticSystemTurnResponseStreamChunk
- AgenticSystemTurnResponseTurnCompletePayload
- AgenticSystemTurnResponseTurnStartPayload
- AgentCreateResponse
- AgentSessionCreateResponse
- AgentStepResponse
- AgentTurnResponseEvent
- AgentTurnResponseStepCompletePayload
- AgentTurnResponseStepProgressPayload
- AgentTurnResponseStepStartPayload
- AgentTurnResponseStreamChunk
- AgentTurnResponseTurnCompletePayload
- AgentTurnResponseTurnStartPayload
- Attachment
- BatchChatCompletionRequest
- BatchChatCompletionResponse
@ -3596,14 +3594,14 @@ x-tagGroups:
- CompletionRequest
- CompletionResponse
- CompletionResponseStreamChunk
- CreateAgenticSystemRequest
- CreateAgenticSystemSessionRequest
- CreateAgenticSystemTurnRequest
- CreateAgentRequest
- CreateAgentSessionRequest
- CreateAgentTurnRequest
- CreateDatasetRequest
- CreateMemoryBankRequest
- DPOAlignmentConfig
- DeleteAgenticSystemRequest
- DeleteAgenticSystemSessionRequest
- DeleteAgentsRequest
- DeleteAgentsSessionRequest
- DeleteDatasetRequest
- DeleteDocumentsRequest
- DialogGenerations
@ -3620,7 +3618,7 @@ x-tagGroups:
- EvaluationJobStatusResponse
- FinetuningAlgorithm
- FunctionCallToolDefinition
- GetAgenticSystemSessionRequest
- GetAgentsSessionRequest
- GetDocumentsRequest
- InferenceStep
- InsertDocumentsRequest

20
rfcs/RFC-0001-llama-stack.md
View file

@ -1,19 +1,19 @@
# The Llama Stack API
**Authors:**
* Meta: @raghotham, @ashwinb, @hjshah, @jspisak
* Meta: @raghotham, @ashwinb, @hjshah, @jspisak
## Summary
As part of the Llama 3.1 release, Meta is releasing an RFC for ‘Llama Stack’, a comprehensive set of interfaces / API for ML developers building on top of Llama foundation models. We are looking for feedback on where the API can be improved, any corner cases we may have missed and your general thoughts on how useful this will be. Ultimately, our hope is to create a standard for working with Llama models in order to simplify the developer experience and foster innovation across the Llama ecosystem.
## Motivation
Llama models were always intended to work as part of an overall system that can orchestrate several components, including calling external tools. Our vision is to go beyond the foundation models and give developers access to a broader system that gives them the flexibility to design and create custom offerings that align with their vision. This thinking started last year when we first introduced a system-level safety model. Meta has continued to release new components for orchestration at the system level and, most recently in Llama 3.1, we’ve introduced the Llama Guard 3 safety model that is multilingual, a prompt injection filter, Prompt Guard and refreshed v3 of our CyberSec Evals. We are also releasing a reference implementation of an agentic system to demonstrate how all the pieces fit together.
Llama models were always intended to work as part of an overall system that can orchestrate several components, including calling external tools. Our vision is to go beyond the foundation models and give developers access to a broader system that gives them the flexibility to design and create custom offerings that align with their vision. This thinking started last year when we first introduced a system-level safety model. Meta has continued to release new components for orchestration at the system level and, most recently in Llama 3.1, we’ve introduced the Llama Guard 3 safety model that is multilingual, a prompt injection filter, Prompt Guard and refreshed v3 of our CyberSec Evals. We are also releasing a reference implementation of an agentic system to demonstrate how all the pieces fit together.
While building the reference implementation, we realized that having a clean and consistent way to interface between components could be valuable not only for us but for anyone leveraging Llama models and other components as part of their system. We’ve also heard from the community as they face a similar challenge as components exist with overlapping functionality and there are incompatible interfaces and yet don't cover the end-to-end model life cycle.
While building the reference implementation, we realized that having a clean and consistent way to interface between components could be valuable not only for us but for anyone leveraging Llama models and other components as part of their system. We’ve also heard from the community as they face a similar challenge as components exist with overlapping functionality and there are incompatible interfaces and yet don't cover the end-to-end model life cycle.
With these motivations, we engaged folks in industry, startups, and the broader developer community to help better define the interfaces of these components. We’re releasing this Llama Stack RFC as a set of standardized and opinionated interfaces for how to surface canonical toolchain components (like inference, fine-tuning, evals, synthetic data generation) and agentic applications to ML developers. Our hope is to have these become well adopted across the ecosystem, which should help with easier interoperability. We would like for builders of multiple components to provide implementations to these standard APIs so that there can be vertically integrated “distributions” of the Llama Stack that can work out of the box easily.
We welcome feedback and ways to improve the proposal. We’re excited to grow the ecosystem around Llama and lower barriers for both developers and platform providers.
We welcome feedback and ways to improve the proposal. We’re excited to grow the ecosystem around Llama and lower barriers for both developers and platform providers.
## Design decisions
Meta releases weights of both the pretrained and instruction fine-tuned Llama models to support several use cases. These weights can be improved - fine tuned and aligned - with curated datasets to then be deployed for inference to support specific applications. The curated datasets can be produced manually by humans or synthetically by other models or by leveraging human feedback by collecting usage data of the application itself. This results in a continuous improvement cycle where the model gets better over time. This is the model life cycle.
@ -42,8 +42,8 @@ Note that as of today, in the OSS world, such a “loop” is often coded explic
**Let's consider an example:**
1. The user asks the system "Who played the NBA finals last year?"
1. The model "understands" that this question needs to be answered using web search. It answers this abstractly with a message of the form "Please call the search tool for me with the query: 'List finalist teams for NBA in the last year' ". Note that the model by itself does not call the tool (of course!)
1. The executor consults the set of tool implementations which have been configured by the developer to find an implementation for the "search tool". If it does not find it, it returns an error to the model. Otherwise, it executes this tool and returns the result of this tool back to the model.
1. The model "understands" that this question needs to be answered using web search. It answers this abstractly with a message of the form "Please call the search tool for me with the query: 'List finalist teams for NBA in the last year' ". Note that the model by itself does not call the tool (of course!)
1. The executor consults the set of tool implementations which have been configured by the developer to find an implementation for the "search tool". If it does not find it, it returns an error to the model. Otherwise, it executes this tool and returns the result of this tool back to the model.
1. The model reasons once again (using all the messages above) and decides to send a final response "In 2023, Denver Nuggets played against the Miami Heat in the NBA finals." to the executor
1. The executor returns the response directly to the user (since there is no tool call to be executed.)
@ -65,7 +65,7 @@ We define the Llama Stack as a layer cake shown below.
The API is defined in the [YAML](RFC-0001-llama-stack-assets/llama-stack-spec.yaml) and [HTML](RFC-0001-llama-stack-assets/llama-stack-spec.html) files. These files were generated using the Pydantic definitions in (api/datatypes.py and api/endpoints.py) files that are in the llama-models, llama-toolchain, and llama-agentic-system repositories.
The API is defined in the [YAML](RFC-0001-llama-stack-assets/llama-stack-spec.yaml) and [HTML](RFC-0001-llama-stack-assets/llama-stack-spec.html) files. These files were generated using the Pydantic definitions in (api/datatypes.py and api/endpoints.py) files that are in the llama-models, llama-stack, and llama-agentic-system repositories.
@ -73,14 +73,14 @@ The API is defined in the [YAML](RFC-0001-llama-stack-assets/llama-stack-spec.ya
## Sample implementations
To prove out the API, we implemented a handful of use cases to make things more concrete. The [llama-agentic-system](https://github.com/meta-llama/llama-agentic-system) repository contains [6 different examples](https://github.com/meta-llama/llama-agentic-system/tree/main/examples/scripts) ranging from very basic to a multi turn agent.
To prove out the API, we implemented a handful of use cases to make things more concrete. The [llama-agentic-system](https://github.com/meta-llama/llama-agentic-system) repository contains [6 different examples](https://github.com/meta-llama/llama-agentic-system/tree/main/examples/scripts) ranging from very basic to a multi turn agent.
There is also a sample inference endpoint implementation in the [llama-toolchain](https://github.com/meta-llama/llama-toolchain/blob/main/llama_toolchain/inference/server.py) repository.
There is also a sample inference endpoint implementation in the [llama-stack](https://github.com/meta-llama/llama-stack/blob/main/llama_stack/inference/server.py) repository.
## Limitations
The reference implementation for Llama Stack APIs to date only includes sample implementations using the inference API. We are planning to flesh out the design of Llama Stack Distributions (distros) by combining capabilities from different providers into a single vertically integrated stack. We plan to implement other APIs and, of course, we’d love contributions!!
Thank you in advance for your feedback, support and contributions to make this a better API.
Thank you in advance for your feedback, support and contributions to make this a better API.
Cheers!

2
rfcs/openapi_generator/README.md
View file

@ -1,4 +1,4 @@
The RFC Specification (OpenAPI format) is generated from the set of API endpoints located in `llama_toolchain/[<subdir>]/api/endpoints.py` using the `generate.py` utility.
The RFC Specification (OpenAPI format) is generated from the set of API endpoints located in `llama_stack/[<subdir>]/api/endpoints.py` using the `generate.py` utility.
Please install the following packages before running the script:

2
rfcs/openapi_generator/generate.py
View file

@ -31,7 +31,7 @@ from .pyopenapi.utility import Specification
schema_utils.json_schema_type = json_schema_type
from llama_toolchain.stack import LlamaStack
from llama_stack.apis.stack import LlamaStack
# TODO: this should be fixed in the generator itself so it reads appropriate annotations