diff --git a/docs/source/concepts/index.md b/docs/source/concepts/index.md index df46e0134..27eb74f00 100644 --- a/docs/source/concepts/index.md +++ b/docs/source/concepts/index.md @@ -33,7 +33,7 @@ Providers come in two flavors: - **Remote**: the provider runs as a separate service external to the Llama Stack codebase. Llama Stack contains a small amount of adapter code. - **Inline**: the provider is fully specified and implemented within the Llama Stack codebase. It may be a simple wrapper around an existing library, or a full fledged implementation within Llama Stack. -Most importantly, Llama Stack always strives to provide at least one fully "local" provider for each API so you can iterate on a fully featured environment locally. +Most importantly, Llama Stack always strives to provide at least one fully inline provider for each API so you can iterate on a fully featured environment locally. ## Resources Some of these APIs are associated with a set of **Resources**. Here is the mapping of APIs to resources: diff --git a/docs/source/conf.py b/docs/source/conf.py index a876333db..fd105a6cf 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -15,7 +15,7 @@ from docutils import nodes project = "llama-stack" -copyright = "2024, Meta" +copyright = "2025, Meta" author = "Meta" # -- General configuration --------------------------------------------------- diff --git a/docs/source/index.md b/docs/source/index.md index cb2355bfd..b6fd314b7 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -67,6 +67,7 @@ A number of "adapters" are available for some popular Inference and Vector Store | **Provider** | **Environments** | | :----: | :----: | | FAISS | Single Node | +| SQLite-Vec| Single Node | | Chroma | Hosted and Single Node | | Postgres (PGVector) | Hosted and Single Node | | Weaviate | Hosted | @@ -88,6 +89,7 @@ self introduction/index getting_started/index concepts/index +providers/index distributions/index distributions/selection building_applications/index diff --git a/docs/source/providers/index.md b/docs/source/providers/index.md new file mode 100644 index 000000000..cc654823e --- /dev/null +++ b/docs/source/providers/index.md @@ -0,0 +1,59 @@ +# Providers Overview + +The goal of Llama Stack is to build an ecosystem where users can easily swap out different implementations for the same API. Examples for these include: +- LLM inference providers (e.g., Fireworks, Together, AWS Bedrock, Groq, Cerebras, SambaNova, etc.), +- Vector databases (e.g., ChromaDB, Weaviate, Qdrant, FAISS, PGVector, etc.), +- Safety providers (e.g., Meta's Llama Guard, AWS Bedrock Guardrails, etc.) + +Providers come in two flavors: +- **Remote**: the provider runs as a separate service external to the Llama Stack codebase. Llama Stack contains a small amount of adapter code. +- **Inline**: the provider is fully specified and implemented within the Llama Stack codebase. It may be a simple wrapper around an existing library, or a full fledged implementation within Llama Stack. + +Importantly, Llama Stack always strives to provide at least one fully inline provider for each API so you can iterate on a fully featured environment locally. + +## Agents +Run multi-step agentic workflows with LLMs with tool usage, memory (RAG), etc. + +## DatasetIO +Interfaces with datasets and data loaders. + +## Eval +Generates outputs (via Inference or Agents) and perform scoring. + +## Inference +Runs inference with an LLM. + +## Post Training +Fine-tunes a model. + +## Safety +Applies safety policies to the output at a Systems (not only model) level. + +## Scoring +Evaluates the outputs of the system. + +## Telemetry +Collects telemetry data from the system. + +## Tool Runtime +Is associated with the ToolGroup resouces. + +## Vector IO + +Vector IO refers to operations on vector databases, such as adding documents, searching, and deleting documents. +Vector IO plays a crucial role in [Retreival Augmented Generation (RAG)](../..//building_applications/rag), where the vector +io and database are used to store and retrieve documents for retrieval. + +#### Vector IO Providers +The following providers (i.e., databases) are available for Vector IO: + +```{toctree} +:maxdepth: 1 + +vector_io/faiss +vector_io/sqlite-vec +vector_io/chromadb +vector_io/pgvector +vector_io/qdrant +vector_io/weaviate +``` diff --git a/docs/source/providers/vector_io/chromadb.md b/docs/source/providers/vector_io/chromadb.md new file mode 100644 index 000000000..4a7caf2e1 --- /dev/null +++ b/docs/source/providers/vector_io/chromadb.md @@ -0,0 +1,36 @@ +--- +orphan: true +--- +# Chroma + +[Chroma](https://www.trychroma.com/) is an inline and remote vector +database provider for Llama Stack. It allows you to store and query vectors directly within a Chroma database. +That means you're not limited to storing vectors in memory or in a separate service. + +## Features +Chroma supports: +- Store embeddings and their metadata +- Vector search +- Full-text search +- Document storage +- Metadata filtering +- Multi-modal retrieval + +## Usage + +To use Chrome in your Llama Stack project, follow these steps: + +1. Install the necessary dependencies. +2. Configure your Llama Stack project to use chroma. +3. Start storing and querying vectors. + +## Installation + +You can install chroma using pip: + +```bash +pip install chromadb +``` + +## Documentation +See [Chroma's documentation](https://docs.trychroma.com/docs/overview/introduction) for more details about Chroma in general. diff --git a/docs/source/providers/vector_io/faiss.md b/docs/source/providers/vector_io/faiss.md new file mode 100644 index 000000000..f894190eb --- /dev/null +++ b/docs/source/providers/vector_io/faiss.md @@ -0,0 +1,33 @@ +--- +orphan: true +--- +# Faiss + +[Faiss](https://github.com/facebookresearch/faiss) is an inline vector database provider for Llama Stack. It +allows you to store and query vectors directly in memory. +That means you'll get fast and efficient vector retrieval. + +## Features + +- Lightweight and easy to use +- Fully integrated with Llama Stack +- GPU support + +## Usage + +To use Faiss in your Llama Stack project, follow these steps: + +1. Install the necessary dependencies. +2. Configure your Llama Stack project to use Faiss. +3. Start storing and querying vectors. + +## Installation + +You can install Faiss using pip: + +```bash +pip install faiss-cpu +``` +## Documentation +See [Faiss' documentation](https://faiss.ai/) or the [Faiss Wiki](https://github.com/facebookresearch/faiss/wiki) for +more details about Faiss in general. diff --git a/docs/source/providers/vector_io/pgvector.md b/docs/source/providers/vector_io/pgvector.md new file mode 100644 index 000000000..919eb88d8 --- /dev/null +++ b/docs/source/providers/vector_io/pgvector.md @@ -0,0 +1,31 @@ +--- +orphan: true +--- +# Postgres PGVector + +[PGVector](https://github.com/pgvector/pgvector) is a remote vector database provider for Llama Stack. It +allows you to store and query vectors directly in memory. +That means you'll get fast and efficient vector retrieval. + +## Features + +- Easy to use +- Fully integrated with Llama Stack + +## Usage + +To use PGVector in your Llama Stack project, follow these steps: + +1. Install the necessary dependencies. +2. Configure your Llama Stack project to use Faiss. +3. Start storing and querying vectors. + +## Installation + +You can install PGVector using docker: + +```bash +docker pull pgvector/pgvector:pg17 +``` +## Documentation +See [PGVector's documentation](https://github.com/pgvector/pgvector) for more details about PGVector in general. diff --git a/docs/source/providers/vector_io/qdrant.md b/docs/source/providers/vector_io/qdrant.md new file mode 100644 index 000000000..c374ade98 --- /dev/null +++ b/docs/source/providers/vector_io/qdrant.md @@ -0,0 +1,31 @@ +--- +orphan: true +--- +# Qdrant + +[Qdrant](https://qdrant.tech/documentation/) is a remote vector database provider for Llama Stack. It +allows you to store and query vectors directly in memory. +That means you'll get fast and efficient vector retrieval. + +## Features + +- Easy to use +- Fully integrated with Llama Stack + +## Usage + +To use Qdrant in your Llama Stack project, follow these steps: + +1. Install the necessary dependencies. +2. Configure your Llama Stack project to use Faiss. +3. Start storing and querying vectors. + +## Installation + +You can install Qdrant using docker: + +```bash +docker pull qdrant/qdrant +``` +## Documentation +See the [Qdrant documentation](https://qdrant.tech/documentation/) for more details about Qdrant in general. diff --git a/docs/source/providers/vector_io/sqlite-vec.md b/docs/source/providers/vector_io/sqlite-vec.md new file mode 100644 index 000000000..f5ce4c003 --- /dev/null +++ b/docs/source/providers/vector_io/sqlite-vec.md @@ -0,0 +1,33 @@ +--- +orphan: true +--- +# SQLite-Vec + +[SQLite-Vec](https://github.com/asg017/sqlite-vec) is an inline vector database provider for Llama Stack. It +allows you to store and query vectors directly within an SQLite database. +That means you're not limited to storing vectors in memory or in a separate service. + +## Features + +- Lightweight and easy to use +- Fully integrated with Llama Stack + +## Usage + +To use SQLite-Vec in your Llama Stack project, follow these steps: + +1. Install the necessary dependencies. +2. Configure your Llama Stack project to use SQLite-Vec. +3. Start storing and querying vectors. + +## Installation + +You can install SQLite-Vec using pip: + +```bash +pip install sqlite-vec +``` + +## Documentation + +See [sqlite-vec's GitHub repo](https://github.com/asg017/sqlite-vec/tree/main) for more details about sqlite-vec in general. diff --git a/docs/source/providers/vector_io/weaviate.md b/docs/source/providers/vector_io/weaviate.md new file mode 100644 index 000000000..47321781c --- /dev/null +++ b/docs/source/providers/vector_io/weaviate.md @@ -0,0 +1,33 @@ +--- +orphan: true +--- +# Weaviate + +[Weaviate](https://weaviate.io/) is a vector database provider for Llama Stack. +It allows you to store and query vectors directly within a Weaviate database. +That means you're not limited to storing vectors in memory or in a separate service. + +## Features +Weaviate supports: +- Store embeddings and their metadata +- Vector search +- Full-text search +- Hybrid search +- Document storage +- Metadata filtering +- Multi-modal retrieval + +## Usage + +To use Weaviate in your Llama Stack project, follow these steps: + +1. Install the necessary dependencies. +2. Configure your Llama Stack project to use chroma. +3. Start storing and querying vectors. + +## Installation + +To install Weaviate see the [Weaviate quickstart documentation](https://weaviate.io/developers/weaviate/quickstart). + +## Documentation +See [Weaviate's documentation](https://weaviate.io/developers/weaviate) for more details about Weaviate in general.