phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-08-02 08:44:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat: add INFERENCE_MODEL env

Browse source
This commit is contained in:
seyeong-han 2024-11-24 01:41:55 -06:00
parent 1d3cc0b138
commit 839f4a4779
1 changed files with 12 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

14
llama_stack/templates/ssambanova/doc_template.md
View file

@ -43,6 +43,14 @@ Make sure you have access to a Ssambanova API Key. You can get one by visiting [
You can do this via Conda (build code) or Docker which has a pre-built image.
### Available INFERENCE_MODEL
- Meta-Llama-3.1-8B-Instruct
- Meta-Llama-3.1-70B-Instruct
- Meta-Llama-3.1-405B-Instruct
- Meta-Llama-3.2-1B-Instruct
- Meta-Llama-3.2-3B-Instruct
### Via Docker
This method allows you to get started quickly without having to build the distribution code.
@ -54,7 +62,8 @@ docker run \
-p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \
llamastack/distribution-{{ name }} \
--port $LLAMA_STACK_PORT \
--env SSAMBANOVA_API_KEY=$SSAMBANOVA_API_KEY
--env SSAMBANOVA_API_KEY=$SSAMBANOVA_API_KEY \
--env INFERENCE_MODEL=$INFERENCE_MODEL
```
### Via Conda
@ -63,5 +72,6 @@ docker run \
llama stack build --template ssambanova --image-type conda
llama stack run ./run.yaml \
--port $LLAMA_STACK_PORT \
--env SSAMBANOVA_API_KEY=$SSAMBANOVA_API_KEY
--env SSAMBANOVA_API_KEY=$SSAMBANOVA_API_KEY \
--env INFERENCE_MODEL=$INFERENCE_MODEL
```