mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-12-23 04:22:25 +00:00
474b50b422
This change lets users configure default embedding models at the provider level instead of always relying on system defaults. Each vector store provider can now specify an embedding_model and optional embedding_dimension in their config. Key features: - Auto-dimension lookup for standard models from the registry - Support for Matryoshka embeddings with custom dimensions - Three-tier priority: explicit params > provider config > system fallback - Full backward compatibility - existing setups work unchanged - Comprehensive test coverage with 20 test cases Updated all vector IO providers (FAISS, Chroma, Milvus, Qdrant, etc.) with the new config fields and added detailed documentation with examples. Fixes #2729
2.2 KiB
2.2 KiB
inline::qdrant
Description
Qdrant is an inline and remote vector database provider for Llama Stack. It allows you to store and query vectors directly in memory. That means you'll get fast and efficient vector retrieval.
By default, Qdrant stores vectors in RAM, delivering incredibly fast access for datasets that fit comfortably in memory. But when your dataset exceeds RAM capacity, Qdrant offers Memmap as an alternative.
[An Introduction to Vector Databases](https://qdrant.tech/articles/what-is-a-vector-database/)
Features
- Lightweight and easy to use
- Fully integrated with Llama Stack
- Apache 2.0 license terms
- Store embeddings and their metadata
- Supports search by Keyword and Hybrid search
- Multilingual and Multimodal retrieval
- Medatata filtering
- GPU support
Usage
To use Qdrant in your Llama Stack project, follow these steps:
- Install the necessary dependencies.
- Configure your Llama Stack project to use Qdrant.
- Start storing and querying vectors.
Installation
You can install Qdrant using docker:
docker pull qdrant/qdrant
Documentation
See the Qdrant documentation for more details about Qdrant in general.
Configuration
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
path |
<class 'str'> |
No | PydanticUndefined | |
embedding_model |
str | None |
No | Optional default embedding model for this provider. If not specified, will use system default. | |
embedding_dimension |
int | None |
No | Optional embedding dimension override. Only needed for models with variable dimensions (e.g., Matryoshka embeddings). If not specified, will auto-lookup from model registry. |
Sample Configuration
path: ${env.QDRANT_PATH:=~/.llama/~/.llama/dummy}/qdrant.db