phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-07-29 15:23:51 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

comments, get started guide

Browse source
This commit is contained in:
Xi Yan 2024-09-18 11:31:46 -07:00
parent e9fbe85dc5
commit 0fa7f1d1aa
3 changed files with 41 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

52
docs/getting_started.md
View file

@ -8,43 +8,61 @@ This guides allows you to quickly get started with building and running a Llama
- Quick 3 line command to build and start a LlamaStack server using our Meta Reference implementation for all API endpoints with `conda` as build type.
**`llama stack build`**
- You'll be prompted to enter build information interactively.
```
llama stack build ./llama_stack/distribution/example_configs/conda/local-conda-example-build.yaml --name my-local-llama-stack
...
...
Build spec configuration saved at ~/.llama/distributions/conda/my-local-llama-stack-build.yaml
llama stack build
> Enter an unique name for identifying your Llama Stack build distribution (e.g. my-local-stack): my-local-llama-stack
> Enter the image type you want your distribution to be built with (docker or conda): conda
Llama Stack is composed of several APIs working together. Let's configure the providers (implementations) you want to use for these APIs.
> Enter the API provider for the inference API: (default=meta-reference): meta-reference
> Enter the API provider for the safety API: (default=meta-reference): meta-reference
> Enter the API provider for the agents API: (default=meta-reference): meta-reference
> Enter the API provider for the memory API: (default=meta-reference): meta-reference
> Enter the API provider for the telemetry API: (default=meta-reference): meta-reference
> (Optional) Enter a short description for your Llama Stack distribution:
Build spec configuration saved at ~/.conda/envs/llamastack-my-local-llama-stack/my-local-llama-stack-build.yaml
```
**`llama stack configure`**
```
llama stack configure ~/.llama/distributions/conda/my-local-llama-stack-build.yaml
llama stack configure my-local-llama-stack
Configuring API: inference (meta-reference)
Configuring APIs to serve...
Enter comma-separated list of APIs to serve:
Configuring API `inference`...
Configuring provider `meta-reference`...
Enter value for model (default: Meta-Llama3.1-8B-Instruct) (required):
Enter value for quantization (optional):
Do you want to configure quantization? (y/n): n
Enter value for torch_seed (optional):
Enter value for max_seq_len (required): 4096
Enter value for max_batch_size (default: 1) (required):
Configuring API `safety`...
Configuring API: memory (meta-reference-faiss)
Configuring API: safety (meta-reference)
Configuring provider `meta-reference`...
Do you want to configure llama_guard_shield? (y/n): n
Do you want to configure prompt_guard_shield? (y/n): n
Configuring API `agents`...
Configuring API: agentic_system (meta-reference)
Enter value for brave_search_api_key (optional):
Enter value for bing_search_api_key (optional):
Enter value for wolfram_api_key (optional):
Configuring provider `meta-reference`...
Configuring API `memory`...
Configuring API: telemetry (console)
Configuring provider `meta-reference`...
Configuring API `telemetry`...
YAML configuration has been written to ~/.llama/builds/conda/my-local-llama-stack-run.yaml
Configuring provider `meta-reference`...
> YAML configuration has been written to /home/xiyan/.llama/builds/conda/my-local-llama-stack-run.yaml.
You can now run `llama stack run my-local-llama-stack --port PORT` or `llama stack run /home/xiyan/.llama/builds/conda/my-local-llama-stack-run.yaml --port PORT
```
**`llama stack run`**
```
llama stack run ~/.llama/builds/conda/my-local-llama-stack-run.yaml
llama stack run my-local-llama-stack
...
> initializing model parallel with size 1

10
llama_stack/cli/stack/build.py
View file

@ -11,8 +11,6 @@ from llama_stack.distribution.datatypes import * # noqa: F403
from pathlib import Path
import yaml
from prompt_toolkit.validation import Validator
from termcolor import cprint
class StackBuild(Subcommand):
@ -84,16 +82,18 @@ class StackBuild(Subcommand):
from llama_stack.distribution.distribution import Api, api_providers
from llama_stack.distribution.utils.dynamic import instantiate_class_type
from prompt_toolkit import prompt
from prompt_toolkit.validation import Validator
from termcolor import cprint
if not args.config:
name = prompt(
"> Enter an unique name for identifying your Llama Stack build distribution (e.g. my-local-stack): "
"> Enter a unique name for identifying your Llama Stack build distribution (e.g. my-local-stack): "
)
image_type = prompt(
"> Enter the image type you want your distribution to be built with (docker or conda): ",
"> Enter the image type you want your distribution to be built as (docker or conda): ",
validator=Validator.from_callable(
lambda x: x in ["docker", "conda"],
error_message="Invalid image type, please enter (conda|docker)",
error_message="Invalid image type, please enter conda or docker",
),
default="conda",
)

2
llama_stack/cli/stack/configure.py
View file

@ -14,7 +14,6 @@ from llama_stack.cli.subcommand import Subcommand
from llama_stack.distribution.utils.config_dirs import BUILDS_BASE_DIR
from llama_stack.distribution.utils.exec import run_with_pty
from termcolor import cprint
from llama_stack.distribution.datatypes import * # noqa: F403
import os
@ -50,6 +49,7 @@ class StackConfigure(Subcommand):
import pkg_resources
from llama_stack.distribution.build import ImageType
from termcolor import cprint
docker_image = None
conda_dir = Path(os.getenv("CONDA_PREFIX")).parent / f"llamastack-{args.config}"