phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-07-29 15:23:51 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update documentation again and add error messages to llama stack start

Browse source
This commit is contained in:
Ashwin Bharambe 2024-09-02 21:36:32 -07:00
parent 279565499b
commit fab6bd1728
2 changed files with 13 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

9
docs/cli_reference.md
View file

@ -328,7 +328,7 @@ Successfully setup conda environment. Configuring build...
... ...
... ...
YAML configuration has been written to ~/.llama/builds/stack/env-local-llama-8b.yaml YAML configuration has been written to ~/.llama/builds/local/conda/llama-8b.yaml
``` ```
You can re-configure this distribution by running: You can re-configure this distribution by running:
@ -358,7 +358,7 @@ Entering sub-configuration for prompt_guard_shield:
Enter value for model (required): Prompt-Guard-86M Enter value for model (required): Prompt-Guard-86M
... ...
... ...
YAML configuration has been written to ~/.llama/builds/conda/local/llama-8b.yaml YAML configuration has been written to ~/.llama/builds/local/conda/llama-8b.yaml
``` ```
As you can see, we did basic configuration above and configured: As you can see, we did basic configuration above and configured:
@ -378,9 +378,6 @@ You need the YAML configuration file which was written out at the end by the `ll
``` ```
llama stack start local --name llama-8b --port 5000 llama stack start local --name llama-8b --port 5000
# or you can give the full path of the YAML file
llama stack start ~/.llama/builds/conda/local/llama-8b.yaml --port 5000
``` ```
You should see the Stack server start and print the APIs that it is supporting, You should see the Stack server start and print the APIs that it is supporting,
@ -417,7 +414,7 @@ INFO: Uvicorn running on http://[::]:5000 (Press CTRL+C to quit)
> [!NOTE] > [!NOTE]
> Configuration is in `~/.llama/builds/conda/local/llama-8b.yaml`. Feel free to increase `max_seq_len`. > Configuration is in `~/.llama/builds/local/conda/llama-8b.yaml`. Feel free to increase `max_seq_len`.
> [!IMPORTANT] > [!IMPORTANT]
> The "local" distribution inference server currently only supports CUDA. It will not work on Apple Silicon machines. > The "local" distribution inference server currently only supports CUDA. It will not work on Apple Silicon machines.

22
llama_toolchain/cli/stack/start.py
View file

@ -65,25 +65,23 @@ class StackStart(Subcommand):
from llama_toolchain.common.exec import run_with_pty from llama_toolchain.common.exec import run_with_pty
from llama_toolchain.core.package import BuildType from llama_toolchain.core.package import BuildType
if args.name.endswith(".yaml"): build_type = BuildType(args.type)
path = args.name build_dir = BUILDS_BASE_DIR / args.distribution / build_type.descriptor()
else: path = build_dir / f"{args.name}.yaml"
build_type = BuildType(args.type)
build_dir = BUILDS_BASE_DIR / args.distribution / build_type.descriptor()
path = build_dir / f"{args.name}.yaml"
config_file = Path(path) config_file = Path(path)
if not config_file.exists():
self.parser.error(
f"File {str(config_file)} does not exist. Did you run `llama stack build`?"
)
return
with open(config_file, "r") as f: with open(config_file, "r") as f:
config = PackageConfig(**yaml.safe_load(f)) config = PackageConfig(**yaml.safe_load(f))
if not config.distribution_id: if not config.distribution_id:
# this is technically not necessary. everything else continues to work, raise ValueError("Build config appears to be corrupt.")
# but maybe we want to be very clear for the users
self.parser.error(
"No distribution_id found. Did you want to start a provider?"
)
return
if config.docker_image: if config.docker_image:
script = pkg_resources.resource_filename( script = pkg_resources.resource_filename(