mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-10-23 08:33:09 +00:00
94 lines
4.4 KiB
Text
94 lines
4.4 KiB
Text
---
|
||
title: Building Custom Distributions
|
||
description: Building a Llama Stack distribution from scratch
|
||
sidebar_label: Build your own Distribution
|
||
sidebar_position: 3
|
||
---
|
||
|
||
This guide walks you through inspecting existing distributions, customising their configuration, and building runnable artefacts for your own deployment.
|
||
|
||
### Explore existing distributions
|
||
|
||
All first-party distributions live under `llama_stack/distributions/`. Each directory contains:
|
||
|
||
- `build.yaml` – the distribution specification (providers, additional dependencies, optional external provider directories).
|
||
- Optional run-config examples such as `<name>-run.yaml`.
|
||
- Documentation fragments that power this site.
|
||
|
||
Browse that folder to understand available providers and copy a distribution to use as a starting point. When creating a new stack, duplicate an existing directory, rename it, and adjust the `build.yaml` file to match your requirements.
|
||
|
||
### Install dependencies locally (optional)
|
||
|
||
If you want to run Llama Stack outside a container, install the package and distribution dependencies in your virtual environment:
|
||
|
||
```bash
|
||
uv pip install llama-stack
|
||
llama stack list-deps starter | xargs -L1 uv pip install
|
||
```
|
||
|
||
Replace `starter` with the distribution you are targeting. This mirrors what the Docker build process does and ensures parity between local and container environments.
|
||
|
||
### Build a container image
|
||
|
||
The recommended approach is to use the Dockerfile at `llama_stack/core/Dockerfile`, which installs `llama-stack`, resolves distribution dependencies via `llama stack list-deps`, and sets the entrypoint to `llama stack run`.
|
||
|
||
```bash
|
||
docker build \
|
||
-f llama_stack/core/Dockerfile \
|
||
--build-arg DISTRO_NAME=starter \
|
||
--tag llama-stack:starter .
|
||
```
|
||
|
||
Handy build arguments:
|
||
|
||
- `DISTRO_NAME` – distribution directory name (defaults to `starter`).
|
||
- `RUN_CONFIG_PATH` – absolute path inside the build context for a run config that should be baked into the image (e.g. `/workspace/run.yaml`).
|
||
- `INSTALL_MODE=editable` – install the repository copied into `/workspace` with `uv pip install -e`. Pair it with `--build-arg LLAMA_STACK_DIR=/workspace`.
|
||
- `LLAMA_STACK_CLIENT_DIR` – optional editable install of the Python client.
|
||
- `PYPI_VERSION` / `TEST_PYPI_VERSION` – pin specific releases when not using editable installs.
|
||
- `KEEP_WORKSPACE=1` – retain `/workspace` in the final image if you need to access additional files (such as sample configs or provider bundles).
|
||
|
||
Make sure any custom `build.yaml`, run configs, or provider directories you reference are included in the Docker build context so the Dockerfile can read them.
|
||
|
||
### Working with external providers
|
||
|
||
External providers live outside the main repository but can be loaded by configuring `external_providers_dir` in your `build.yaml`. To bundle them into your image:
|
||
|
||
1. Copy the providers into the Docker build context (for example, under `providers.d/`).
|
||
2. Set `external_providers_dir` in `build.yaml` to the on-disk path you copied.
|
||
3. If the run config references that directory, update it to match the in-container location (e.g. `/.llama/providers.d`). Provide the run config to the Docker build via `--build-arg RUN_CONFIG_PATH=/workspace/run.yaml`.
|
||
|
||
During the build, `llama stack list-deps` will install the provider’s declared dependencies.
|
||
|
||
For more detail on crafting external providers, see the [External Providers documentation](../providers/external/).
|
||
|
||
### Run your stack server
|
||
|
||
After building the image, launch it directly with Docker or Podman—the entrypoint calls `llama stack run` using the baked distribution or the bundled run config:
|
||
|
||
```bash
|
||
docker run -d \
|
||
-p 8321:8321 \
|
||
-e OPENAI_API_KEY=... \
|
||
llama-stack:starter
|
||
```
|
||
|
||
To run a different distribution at runtime, pass it as an extra argument:
|
||
|
||
```bash
|
||
docker run llama-stack:starter meta-reference-gpu
|
||
```
|
||
|
||
If you prepared a custom run config, mount it into the container and reference it explicitly:
|
||
|
||
```bash
|
||
docker run \
|
||
-p 8321:8321 \
|
||
-v $(pwd)/run.yaml:/app/run.yaml \
|
||
llama-stack:starter \
|
||
/app/run.yaml
|
||
```
|
||
|
||
### Listing distributions
|
||
|
||
The repository is the source of truth for available stacks. Use your editor or `ls llama_stack/distributions` to inspect what ships with the project. The CLI command `llama stack list` still enumerates installed stacks on your machine, but when authoring new ones you should work directly with the files in `llama_stack/distributions`.
|