mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-23 19:03:58 +00:00

skamenan7 474b50b422 Add configurable embedding models for vector IO providers

This change lets users configure default embedding models at the provider level instead of always relying on system defaults. Each vector store provider can now specify an embedding_model and optional embedding_dimension in their config.

Key features:
- Auto-dimension lookup for standard models from the registry
- Support for Matryoshka embeddings with custom dimensions
- Three-tier priority: explicit params > provider config > system fallback
- Full backward compatibility - existing setups work unchanged
- Comprehensive test coverage with 20 test cases

Updated all vector IO providers (FAISS, Chroma, Milvus, Qdrant, etc.) with the new config fields and added detailed documentation with examples.

Fixes #2729

2025-07-15 16:46:40 -04:00

1.6 KiB

Raw Blame History

inline::faiss

Description

Faiss is an inline vector database provider for Llama Stack. It allows you to store and query vectors directly in memory. That means you'll get fast and efficient vector retrieval.

Features

Lightweight and easy to use
Fully integrated with Llama Stack
GPU support

Usage

To use Faiss in your Llama Stack project, follow these steps:

Install the necessary dependencies.
Configure your Llama Stack project to use Faiss.
Start storing and querying vectors.

Installation

You can install Faiss using pip:

pip install faiss-cpu

Documentation

See Faiss' documentation or the Faiss Wiki for more details about Faiss in general.

Configuration

Field	Type	Required	Default	Description
`kvstore`	`utils.kvstore.config.RedisKVStoreConfig \| utils.kvstore.config.SqliteKVStoreConfig \| utils.kvstore.config.PostgresKVStoreConfig \| utils.kvstore.config.MongoDBKVStoreConfig`	No	sqlite
`embedding_model`	`str \| None`	No		Optional default embedding model for this provider. If not specified, will use system default.
`embedding_dimension`	`int \| None`	No		Optional embedding dimension override. Only needed for models with variable dimensions (e.g., Matryoshka embeddings). If not specified, will auto-lookup from model registry.

Sample Configuration

kvstore:
  type: sqlite
  db_path: ${env.SQLITE_STORE_DIR:=~/.llama/dummy}/faiss_store.db

1.6 KiB Raw Blame History