phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-07-30 23:51:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs

Browse source
This commit is contained in:
Xi Yan 2024-11-05 13:23:27 -08:00
parent cf4e41609d
commit 5161a8ef8c
1 changed files with 13 additions and 24 deletions
Download patch file Download diff file Expand all files Collapse all files

37
docs/source/distribution_dev/building_distro.md
View file

@ -5,22 +5,10 @@ This guide will walk you through the steps to get started with building a Llama
## Step 1. Build ## Step 1. Build
### Llama Stack Build Options
``` ```
llama stack build -h llama stack build -h
usage: llama stack build [-h] [--config CONFIG] [--template TEMPLATE] [--list-templates | --no-list-templates] [--image-type {conda,docker}]
Build a Llama stack container
options:
-h, --help show this help message and exit
--config CONFIG Path to a config file to use for the build. You can find example configs in llama_stack/distribution/example_configs. If this argument is not provided, you will be prompted to enter information interactively
--template TEMPLATE Name of the example template config to use for build. You may use `llama stack build --list-templates` to check out the available templates
--list-templates, --no-list-templates
Show the available templates for building a Llama Stack distribution
--image-type {conda,docker}
Image Type to use for the build. This can be either conda or docker. If not specified, will use the image type from the template config.
``` ```
We will start build our distribution (in the form of a Conda environment, or Docker image). In this step, we will specify: We will start build our distribution (in the form of a Conda environment, or Docker image). In this step, we will specify:
- `name`: the name for our distribution (e.g. `my-stack`) - `name`: the name for our distribution (e.g. `my-stack`)
@ -32,14 +20,9 @@ We will start build our distribution (in the form of a Conda environment, or Doc
After this step is complete, a file named `<name>-build.yaml` and template file `<name>-run.yaml` will be generated and saved at the output file path specified at the end of the command. After this step is complete, a file named `<name>-build.yaml` and template file `<name>-run.yaml` will be generated and saved at the output file path specified at the end of the command.
::::{tab-set}
:::{tab-item} Building from Scratch
You have 3 options for building your distribution:
1.1 Building from scratch
1.2. Building from a template
1.3. Building from a pre-existing build config file
### 1.1 Building from scratch
- For a new user, we could start off with running `llama stack build` which will allow you to a interactively enter wizard where you will be prompted to enter build configurations. - For a new user, we could start off with running `llama stack build` which will allow you to a interactively enter wizard where you will be prompted to enter build configurations.
``` ```
llama stack build llama stack build
@ -65,8 +48,9 @@ Tip: use <TAB> to see options for the providers.
You can now edit ~/.llama/distributions/llamastack-my-local-stack/my-local-stack-run.yaml and run `llama stack run ~/.llama/distributions/llamastack-my-local-stack/my-local-stack-run.yaml` You can now edit ~/.llama/distributions/llamastack-my-local-stack/my-local-stack-run.yaml and run `llama stack run ~/.llama/distributions/llamastack-my-local-stack/my-local-stack-run.yaml`
``` ```
:::
### 1.2 Building from a template :::{tab-item} Building from a template
- To build from alternative API providers, we provide distribution templates for users to get started building a distribution backed by different providers. - To build from alternative API providers, we provide distribution templates for users to get started building a distribution backed by different providers.
The following command will allow you to see the available templates and their corresponding providers. The following command will allow you to see the available templates and their corresponding providers.
@ -204,8 +188,9 @@ $ llama stack build --template tgi
... ...
You can now edit ~/.llama/distributions/llamastack-tgi/tgi-run.yaml and run `llama stack run ~/.llama/distributions/llamastack-tgi/tgi-run.yaml` You can now edit ~/.llama/distributions/llamastack-tgi/tgi-run.yaml and run `llama stack run ~/.llama/distributions/llamastack-tgi/tgi-run.yaml`
``` ```
:::
### 1.3 Building from a pre-existing build config file :::{tab-item} Building from a pre-existing build config file
- In addition to templates, you may customize the build to your liking through editing config files and build from config files with the following command. - In addition to templates, you may customize the build to your liking through editing config files and build from config files with the following command.
- The config file will be of contents like the ones in `llama_stack/templates/*build.yaml`. - The config file will be of contents like the ones in `llama_stack/templates/*build.yaml`.
@ -228,8 +213,9 @@ image_type: conda
``` ```
llama stack build --config llama_stack/templates/ollama/build.yaml llama stack build --config llama_stack/templates/ollama/build.yaml
``` ```
:::
### How to build distribution with Docker image :::{tab-item} Building Docker
> [!TIP] > [!TIP]
> Podman is supported as an alternative to Docker. Set `DOCKER_BINARY` to `podman` in your environment to use Podman. > Podman is supported as an alternative to Docker. Set `DOCKER_BINARY` to `podman` in your environment to use Podman.
@ -249,6 +235,9 @@ You can now edit ~/meta-llama/llama-stack/tmp/configs/ollama-run.yaml and run `l
``` ```
After this step is successful, you should be able to find the built docker image and test it with `llama stack run <path/to/run.yaml>`. After this step is successful, you should be able to find the built docker image and test it with `llama stack run <path/to/run.yaml>`.
:::
::::
## Step 2. Run ## Step 2. Run