phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-08-01 16:24:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs: recheck conda env and fix accordingly

Browse source
This commit is contained in:
seyeong-han 2024-11-30 12:00:43 -06:00
parent 41b267d1c9
commit c769ba7de3
1 changed files with 15 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

38
llama_stack/templates/sambanova/doc_template.md
View file

@ -41,37 +41,29 @@ Make sure you have access to a Sambanova API Key. You can get one by visiting [S
## Running Llama Stack with Sambanova
You can do this via Conda (build code) or Docker which has a pre-built image.
You can do this via Conda (build code).
### Available INFERENCE_MODEL
- Meta-Llama-3.1-8B-Instruct
- Meta-Llama-3.1-70B-Instruct
- Meta-Llama-3.1-405B-Instruct
- Meta-Llama-3.2-1B-Instruct
- Meta-Llama-3.2-3B-Instruct
### Via Docker
This method allows you to get started quickly without having to build the distribution code.
```bash
LLAMA_STACK_PORT=5001
docker run \
-it \
-p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \
llamastack/distribution-{{ name }} \
--port $LLAMA_STACK_PORT \
--env SAMBANOVA_API_KEY=$SAMBANOVA_API_KEY \
--env INFERENCE_MODEL=$INFERENCE_MODEL
```
- meta-llama/Llama-3.1-8B-Instruct
- meta-llama/Llama-3.1-70B-Instruct
- meta-llama/Llama-3.1-405B-Instruct
- meta-llama/Llama-3.2-1B-Instruct
- meta-llama/Llama-3.2-3B-Instruct
### Via Conda
```bash
llama stack build --template sambanova --image-type conda
llama stack run ./run.yaml \
conda activate llamastack-sambanova
export SAMBANOVA_API_KEY={YOUR_API_KEY}
export INFERENCE_MODEL={CHOOSE_AND_FIND_AVAILABLE_MODEL_ABOVE}
llama stack run \
--port $LLAMA_STACK_PORT \
--env SAMBANOVA_API_KEY=$SAMBANOVA_API_KEY \
--env INFERENCE_MODEL=$INFERENCE_MODEL
--env INFERENCE_MODEL=$INFERENCE_MODEL \
sambanova
```