phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-04 02:03:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge remote-tracking branch 'upstream/main' into feat-3410-bedrock-openai-compatible-provider

Browse source
This commit is contained in:
skamenan7 2025-11-06 14:17:05 -05:00
commit 10961017e0
108 changed files with 9553 additions and 618 deletions
Download patch file Download diff file Expand all files Collapse all files

41
docs/docs/distributions/self_hosted_distro/starter.md
View file

@ -163,7 +163,41 @@ docker run \
--port $LLAMA_STACK_PORT
```
### Via venv
The container will run the distribution with a SQLite store by default. This store is used for the following components:
- Metadata store: store metadata about the models, providers, etc.
- Inference store: collect of responses from the inference provider
- Agents store: store agent configurations (sessions, turns, etc.)
- Agents Responses store: store responses from the agents
However, you can use PostgreSQL instead by running the `starter::run-with-postgres-store.yaml` configuration:
```bash
docker run \
-it \
--pull always \
-p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \
-e OPENAI_API_KEY=your_openai_key \
-e FIREWORKS_API_KEY=your_fireworks_key \
-e TOGETHER_API_KEY=your_together_key \
-e POSTGRES_HOST=your_postgres_host \
-e POSTGRES_PORT=your_postgres_port \
-e POSTGRES_DB=your_postgres_db \
-e POSTGRES_USER=your_postgres_user \
-e POSTGRES_PASSWORD=your_postgres_password \
llamastack/distribution-starter \
starter::run-with-postgres-store.yaml
```
Postgres environment variables:
- `POSTGRES_HOST`: Postgres host (default: `localhost`)
- `POSTGRES_PORT`: Postgres port (default: `5432`)
- `POSTGRES_DB`: Postgres database name (default: `llamastack`)
- `POSTGRES_USER`: Postgres username (default: `llamastack`)
- `POSTGRES_PASSWORD`: Postgres password (default: `llamastack`)
### Via Conda or venv
Ensure you have configured the starter distribution using the environment variables explained above.
@ -171,8 +205,11 @@ Ensure you have configured the starter distribution using the environment variab
# Install dependencies for the starter distribution
uv run --with llama-stack llama stack list-deps starter | xargs -L1 uv pip install
# Run the server
# Run the server (with SQLite - default)
uv run --with llama-stack llama stack run starter
# Or run with PostgreSQL
uv run --with llama-stack llama stack run starter::run-with-postgres-store.yaml
```
## Example Usage

2
docs/docs/providers/inference/remote_passthrough.mdx
View file

@ -16,7 +16,7 @@ Passthrough inference provider for connecting to any external inference service
|-------|------|----------|---------|-------------|
| `allowed_models` | `list[str \| None` | No | | List of models that should be registered with the model registry. If None, all models are allowed. |
| `refresh_models` | `<class 'bool'>` | No | False | Whether to refresh models periodically from the provider |
| `api_key` | `pydantic.types.SecretStr \| None` | No | | API Key for the passthrouth endpoint |
| `api_key` | `pydantic.types.SecretStr \| None` | No | | Authentication credential for the provider |
| `url` | `<class 'str'>` | No | | The URL for the passthrough endpoint |
## Sample Configuration

142
docs/static/llama-stack-spec.yaml vendored
View file

@ -9260,6 +9260,70 @@ components:
- metadata
title: VectorStoreObject
description: OpenAI Vector Store object.
VectorStoreChunkingStrategy:
oneOf:
- $ref: '#/components/schemas/VectorStoreChunkingStrategyAuto'
- $ref: '#/components/schemas/VectorStoreChunkingStrategyStatic'
discriminator:
propertyName: type
mapping:
auto: '#/components/schemas/VectorStoreChunkingStrategyAuto'
static: '#/components/schemas/VectorStoreChunkingStrategyStatic'
VectorStoreChunkingStrategyAuto:
type: object
properties:
type:
type: string
const: auto
default: auto
description: >-
Strategy type, always "auto" for automatic chunking
additionalProperties: false
required:
- type
title: VectorStoreChunkingStrategyAuto
description: >-
Automatic chunking strategy for vector store files.
VectorStoreChunkingStrategyStatic:
type: object
properties:
type:
type: string
const: static
default: static
description: >-
Strategy type, always "static" for static chunking
static:
$ref: '#/components/schemas/VectorStoreChunkingStrategyStaticConfig'
description: >-
Configuration parameters for the static chunking strategy
additionalProperties: false
required:
- type
- static
title: VectorStoreChunkingStrategyStatic
description: >-
Static chunking strategy with configurable parameters.
VectorStoreChunkingStrategyStaticConfig:
type: object
properties:
chunk_overlap_tokens:
type: integer
default: 400
description: >-
Number of tokens to overlap between adjacent chunks
max_chunk_size_tokens:
type: integer
default: 800
description: >-
Maximum number of tokens per chunk, must be between 100 and 4096
additionalProperties: false
required:
- chunk_overlap_tokens
- max_chunk_size_tokens
title: VectorStoreChunkingStrategyStaticConfig
description: >-
Configuration for static chunking strategy.
"OpenAICreateVectorStoreRequestWithExtraBody":
type: object
properties:
@ -9285,15 +9349,7 @@ components:
description: >-
(Optional) Expiration policy for the vector store
chunking_strategy:
type: object
additionalProperties:
oneOf:
- type: 'null'
- type: boolean
- type: number
- type: string
- type: array
- type: object
$ref: '#/components/schemas/VectorStoreChunkingStrategy'
description: >-
(Optional) Strategy for splitting files into chunks
metadata:
@ -9369,70 +9425,6 @@ components:
- deleted
title: VectorStoreDeleteResponse
description: Response from deleting a vector store.
VectorStoreChunkingStrategy:
oneOf:
- $ref: '#/components/schemas/VectorStoreChunkingStrategyAuto'
- $ref: '#/components/schemas/VectorStoreChunkingStrategyStatic'
discriminator:
propertyName: type
mapping:
auto: '#/components/schemas/VectorStoreChunkingStrategyAuto'
static: '#/components/schemas/VectorStoreChunkingStrategyStatic'
VectorStoreChunkingStrategyAuto:
type: object
properties:
type:
type: string
const: auto
default: auto
description: >-
Strategy type, always "auto" for automatic chunking
additionalProperties: false
required:
- type
title: VectorStoreChunkingStrategyAuto
description: >-
Automatic chunking strategy for vector store files.
VectorStoreChunkingStrategyStatic:
type: object
properties:
type:
type: string
const: static
default: static
description: >-
Strategy type, always "static" for static chunking
static:
$ref: '#/components/schemas/VectorStoreChunkingStrategyStaticConfig'
description: >-
Configuration parameters for the static chunking strategy
additionalProperties: false
required:
- type
- static
title: VectorStoreChunkingStrategyStatic
description: >-
Static chunking strategy with configurable parameters.
VectorStoreChunkingStrategyStaticConfig:
type: object
properties:
chunk_overlap_tokens:
type: integer
default: 400
description: >-
Number of tokens to overlap between adjacent chunks
max_chunk_size_tokens:
type: integer
default: 800
description: >-
Maximum number of tokens per chunk, must be between 100 and 4096
additionalProperties: false
required:
- chunk_overlap_tokens
- max_chunk_size_tokens
title: VectorStoreChunkingStrategyStaticConfig
description: >-
Configuration for static chunking strategy.
"OpenAICreateVectorStoreFileBatchRequestWithExtraBody":
type: object
properties:
@ -9890,7 +9882,9 @@ components:
description: >-
Object type identifier for the search results page
search_query:
type: string
type: array
items:
type: string
description: >-
The original search query that was executed
data:

142
docs/static/stainless-llama-stack-spec.yaml vendored
View file

@ -9976,6 +9976,70 @@ components:
- metadata
title: VectorStoreObject
description: OpenAI Vector Store object.
VectorStoreChunkingStrategy:
oneOf:
- $ref: '#/components/schemas/VectorStoreChunkingStrategyAuto'
- $ref: '#/components/schemas/VectorStoreChunkingStrategyStatic'
discriminator:
propertyName: type
mapping:
auto: '#/components/schemas/VectorStoreChunkingStrategyAuto'
static: '#/components/schemas/VectorStoreChunkingStrategyStatic'
VectorStoreChunkingStrategyAuto:
type: object
properties:
type:
type: string
const: auto
default: auto
description: >-
Strategy type, always "auto" for automatic chunking
additionalProperties: false
required:
- type
title: VectorStoreChunkingStrategyAuto
description: >-
Automatic chunking strategy for vector store files.
VectorStoreChunkingStrategyStatic:
type: object
properties:
type:
type: string
const: static
default: static
description: >-
Strategy type, always "static" for static chunking
static:
$ref: '#/components/schemas/VectorStoreChunkingStrategyStaticConfig'
description: >-
Configuration parameters for the static chunking strategy
additionalProperties: false
required:
- type
- static
title: VectorStoreChunkingStrategyStatic
description: >-
Static chunking strategy with configurable parameters.
VectorStoreChunkingStrategyStaticConfig:
type: object
properties:
chunk_overlap_tokens:
type: integer
default: 400
description: >-
Number of tokens to overlap between adjacent chunks
max_chunk_size_tokens:
type: integer
default: 800
description: >-
Maximum number of tokens per chunk, must be between 100 and 4096
additionalProperties: false
required:
- chunk_overlap_tokens
- max_chunk_size_tokens
title: VectorStoreChunkingStrategyStaticConfig
description: >-
Configuration for static chunking strategy.
"OpenAICreateVectorStoreRequestWithExtraBody":
type: object
properties:
@ -10001,15 +10065,7 @@ components:
description: >-
(Optional) Expiration policy for the vector store
chunking_strategy:
type: object
additionalProperties:
oneOf:
- type: 'null'
- type: boolean
- type: number
- type: string
- type: array
- type: object
$ref: '#/components/schemas/VectorStoreChunkingStrategy'
description: >-
(Optional) Strategy for splitting files into chunks
metadata:
@ -10085,70 +10141,6 @@ components:
- deleted
title: VectorStoreDeleteResponse
description: Response from deleting a vector store.
VectorStoreChunkingStrategy:
oneOf:
- $ref: '#/components/schemas/VectorStoreChunkingStrategyAuto'
- $ref: '#/components/schemas/VectorStoreChunkingStrategyStatic'
discriminator:
propertyName: type
mapping:
auto: '#/components/schemas/VectorStoreChunkingStrategyAuto'
static: '#/components/schemas/VectorStoreChunkingStrategyStatic'
VectorStoreChunkingStrategyAuto:
type: object
properties:
type:
type: string
const: auto
default: auto
description: >-
Strategy type, always "auto" for automatic chunking
additionalProperties: false
required:
- type
title: VectorStoreChunkingStrategyAuto
description: >-
Automatic chunking strategy for vector store files.
VectorStoreChunkingStrategyStatic:
type: object
properties:
type:
type: string
const: static
default: static
description: >-
Strategy type, always "static" for static chunking
static:
$ref: '#/components/schemas/VectorStoreChunkingStrategyStaticConfig'
description: >-
Configuration parameters for the static chunking strategy
additionalProperties: false
required:
- type
- static
title: VectorStoreChunkingStrategyStatic
description: >-
Static chunking strategy with configurable parameters.
VectorStoreChunkingStrategyStaticConfig:
type: object
properties:
chunk_overlap_tokens:
type: integer
default: 400
description: >-
Number of tokens to overlap between adjacent chunks
max_chunk_size_tokens:
type: integer
default: 800
description: >-
Maximum number of tokens per chunk, must be between 100 and 4096
additionalProperties: false
required:
- chunk_overlap_tokens
- max_chunk_size_tokens
title: VectorStoreChunkingStrategyStaticConfig
description: >-
Configuration for static chunking strategy.
"OpenAICreateVectorStoreFileBatchRequestWithExtraBody":
type: object
properties:
@ -10606,7 +10598,9 @@ components:
description: >-
Object type identifier for the search results page
search_query:
type: string
type: array
items:
type: string
description: >-
The original search query that was executed
data: