phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-08-15 06:00:48 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs: update configuration documentation for global default embedding model

Browse source 
- Clarified the optional nature of the default_embedding_dimension in the YAML configuration, specifying that it defaults to 384 if omitted.
- Added a note in the VectorStoreConfig class to indicate that the router will fall back to 384 as the default dimension if not set.
This commit is contained in:
skamenan7 2025-07-30 13:20:59 -04:00 • committed by Sumanth Kamenani
parent 32e8e4045a
commit aec1df5a39
2 changed files with 4 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

6
docs/source/distributions/configuration.md
View file

@ -690,14 +690,14 @@ shields:
### Global Vector-Store Defaults ### Global Vector-Store Defaults
Starting with Llama-Stack v2, you can provide a *stack-level* default embedding model that will be used whenever a new vector-store is created and the caller does **not** specify an `embedding_model` parameter. You can provide a *stack-level* default embedding model that will be used whenever a new vector-store is created and the caller does **not** specify an `embedding_model` parameter.
Add a top-level block next to `models:` and `vector_io:` in your build/run YAML: Add a top-level block next to `models:` and `vector_io:` in your build/run YAML:
```yaml ```yaml
vector_store_config: vector_store_config:
default_embedding_model: ${env.LLAMA_STACK_DEFAULT_EMBEDDING_MODEL:=all-MiniLM-L6-v2} default_embedding_model: ${env.LLAMA_STACK_DEFAULT_EMBEDDING_MODEL:=all-MiniLM-L6-v2}
# optional but recommended # optional - if omitted, defaults to 384
default_embedding_dimension: ${env.LLAMA_STACK_DEFAULT_EMBEDDING_DIMENSION:=384} default_embedding_dimension: ${env.LLAMA_STACK_DEFAULT_EMBEDDING_DIMENSION:=384}
``` ```
@ -712,7 +712,7 @@ Precedence rules at runtime:
| Variable | Purpose | Example | | Variable | Purpose | Example |
|----------|---------|---------| |----------|---------|---------|
| `LLAMA_STACK_DEFAULT_EMBEDDING_MODEL` | Global default embedding model id | `all-MiniLM-L6-v2` | | `LLAMA_STACK_DEFAULT_EMBEDDING_MODEL` | Global default embedding model id | `all-MiniLM-L6-v2` |
| `LLAMA_STACK_DEFAULT_EMBEDDING_DIMENSION` | Dimension for embeddings (optional) | `384` | | `LLAMA_STACK_DEFAULT_EMBEDDING_DIMENSION` | Dimension for embeddings (optional, defaults to 384) | `384` |
If you include the `${env.…}` placeholder in `vector_store_config`, deployments can override the default without editing YAML: If you include the `${env.…}` placeholder in `vector_store_config`, deployments can override the default without editing YAML:

1
llama_stack/apis/common/vector_store_config.py
View file

@ -41,5 +41,6 @@ class VectorStoreConfig(BaseModel):
default_embedding_dimension: int | None = Field( default_embedding_dimension: int | None = Field(
default_factory=lambda: int(os.getenv("LLAMA_STACK_DEFAULT_EMBEDDING_DIMENSION", 0)) or None, ge=1 default_factory=lambda: int(os.getenv("LLAMA_STACK_DEFAULT_EMBEDDING_DIMENSION", 0)) or None, ge=1
) )
# Note: If not set, the router will fall back to 384 as the default dimension
model_config = ConfigDict(frozen=True) model_config = ConfigDict(frozen=True)