Add config file based CLI (#60)

* config file for build

* fix build command

* configure script with config

* fix configure script to work with config file

* update build.sh

* update readme

* distribution_type -> distribution

* fix run-config/config-file to config

* move import to inline

* only consume config as argument

* update configure to only consume config

* update readme

* update readme
This commit is contained in:
Xi Yan 2024-09-11 11:39:46 -07:00 committed by GitHub
parent 58def874a9
commit 89300df5dc
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 155 additions and 111 deletions

View file

@ -295,13 +295,18 @@ As you can see above, each “distribution” details the “providers” it is
Let's imagine you are working with a 8B-Instruct model. The following command will build a package (in the form of a Conda environment) _and_ configure it. As part of the configuration, you will be asked for some inputs (model_id, max_seq_len, etc.) Since we are working with a 8B model, we will name our build `8b-instruct` to help us remember the config.
```
llama stack build local --name 8b-instruct
llama stack build
```
Once it runs successfully , you should see some outputs in the form:
Once it runs, you will be prompted to enter build name and optional arguments, and should see some outputs in the form:
```
$ llama stack build local --name 8b-instruct
$ llama stack build
Enter value for name (required): 8b-instruct
Enter value for distribution (default: local) (required): local
Enter value for api_providers (optional):
Enter value for image_type (default: conda) (required):
....
....
Successfully installed cfgv-3.4.0 distlib-0.3.8 identify-2.6.0 libcst-1.4.0 llama_toolchain-0.0.2 moreorless-0.4.0 nodeenv-1.9.1 pre-commit-3.8.0 stdlibs-2024.5.15 toml-0.10.2 tomlkit-0.13.0 trailrunner-1.4.0 ufmt-2.7.0 usort-1.0.8 virtualenv-20.26.3
@ -312,17 +317,41 @@ Successfully setup conda environment. Configuring build...
...
YAML configuration has been written to ~/.llama/builds/local/conda/8b-instruct.yaml
Target `8b-test` built with configuration at /home/xiyan/.llama/builds/local/conda/8b-test.yaml
Build spec configuration saved at /home/xiyan/.llama/distributions/local/conda/8b-test-build.yaml
```
You can re-build package based on build config
```
$ cat ~/.llama/distributions/local/conda/8b-instruct-build.yaml
name: 8b-instruct
distribution: local
api_providers: null
image_type: conda
$ llama stack build --config ~/.llama/distributions/local/conda/8b-instruct-build.yaml
Successfully setup conda environment. Configuring build...
...
...
YAML configuration has been written to ~/.llama/builds/local/conda/8b-instruct.yaml
Target `8b-instruct` built with configuration at ~/.llama/builds/local/conda/8b-instruct.yaml
Build spec configuration saved at ~/.llama/distributions/local/conda/8b-instruct-build.yaml
```
### Step 3.3: Configure a distribution
You can re-configure this distribution by running:
```
llama stack configure local --name 8b-instruct
llama stack configure ~/.llama/builds/local/conda/8b-instruct.yaml
```
Here is an example run of how the CLI will guide you to fill the configuration
```
$ llama stack configure local --name 8b-instruct
$ llama stack configure ~/.llama/builds/local/conda/8b-instruct.yaml
Configuring API: inference (meta-reference)
Enter value for model (required): Meta-Llama3.1-8B-Instruct
@ -363,12 +392,12 @@ Now lets start Llama Stack Distribution Server.
You need the YAML configuration file which was written out at the end by the `llama stack build` step.
```
llama stack run local --name 8b-instruct --port 5000
llama stack run ~/.llama/builds/local/conda/8b-instruct.yaml --port 5000
```
You should see the Stack server start and print the APIs that it is supporting,
```
$ llama stack run local --name 8b-instruct --port 5000
$ llama stack run ~/.llama/builds/local/conda/8b-instruct.yaml --port 5000
> initializing model parallel with size 1
> initializing ddp with size 1