phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-18 09:39:48 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

refine

Browse source
This commit is contained in:
Botao Chen 2024-12-18 16:14:51 -08:00
parent cd1fc4fd17
commit 78b6518b2c
5 changed files with 26 additions and 26 deletions
Download patch file Download diff file Expand all files Collapse all files

12
llama_stack/templates/meta-reference-gpu/doc_template.md
View file

@ -53,7 +53,7 @@ docker run \
-v ~/.llama:/root/.llama \
llamastack/distribution-{{ name }} \
--port $LLAMA_STACK_PORT \
--env INFERENCE_MODEL=Llama3.2-3B-Instruct
--env INFERENCE_MODEL=meta-llama/Llama-3.2-3B-Instruct
```
If you are using Llama Stack Safety / Shield APIs, use:
@ -65,8 +65,8 @@ docker run \
-v ~/.llama:/root/.llama \
llamastack/distribution-{{ name }} \
--port $LLAMA_STACK_PORT \
--env INFERENCE_MODEL=Llama3.2-3B-Instruct \
--env SAFETY_MODEL=Llama-Guard-3-1B
--env INFERENCE_MODEL=meta-llama/Llama-3.2-3B-Instruct \
--env SAFETY_MODEL=meta-llama/Llama-Guard-3-1B
```
### Via Conda
@ -77,7 +77,7 @@ Make sure you have done `pip install llama-stack` and have the Llama Stack CLI a
llama stack build --template {{ name }} --image-type conda
llama stack run distributions/{{ name }}/run.yaml \
--port 5001 \
--env INFERENCE_MODEL=Llama3.2-3B-Instruct
--env INFERENCE_MODEL=meta-llama/Llama-3.2-3B-Instruct
```
If you are using Llama Stack Safety / Shield APIs, use:
@ -85,6 +85,6 @@ If you are using Llama Stack Safety / Shield APIs, use:
```bash
llama stack run distributions/{{ name }}/run-with-safety.yaml \
--port 5001 \
--env INFERENCE_MODEL=Llama3.2-3B-Instruct \
--env SAFETY_MODEL=Llama-Guard-3-1B
--env INFERENCE_MODEL=meta-llama/Llama-3.2-3B-Instruct \
--env SAFETY_MODEL=meta-llama/Llama-Guard-3-1B
```

12
llama_stack/templates/meta-reference-quantized-gpu/doc_template.md
View file

@ -55,7 +55,7 @@ docker run \
-v ~/.llama:/root/.llama \
llamastack/distribution-{{ name }} \
--port $LLAMA_STACK_PORT \
--env INFERENCE_MODEL=Llama3.2-3B-Instruct
--env INFERENCE_MODEL=meta-llama/Llama-3.2-3B-Instruct
```
If you are using Llama Stack Safety / Shield APIs, use:
@ -67,8 +67,8 @@ docker run \
-v ~/.llama:/root/.llama \
llamastack/distribution-{{ name }} \
--port $LLAMA_STACK_PORT \
--env INFERENCE_MODEL=Llama3.2-3B-Instruct \
--env SAFETY_MODEL=Llama-Guard-3-1B
--env INFERENCE_MODEL=meta-llama/Llama-3.2-3B-Instruct \
--env SAFETY_MODEL=meta-llama/Llama-Guard-3-1B
```
### Via Conda
@ -79,7 +79,7 @@ Make sure you have done `pip install llama-stack` and have the Llama Stack CLI a
llama stack build --template {{ name }} --image-type conda
llama stack run distributions/{{ name }}/run.yaml \
--port $LLAMA_STACK_PORT \
--env INFERENCE_MODEL=Llama3.2-3B-Instruct
--env INFERENCE_MODEL=meta-llama/Llama-3.2-3B-Instruct
```
If you are using Llama Stack Safety / Shield APIs, use:
@ -87,6 +87,6 @@ If you are using Llama Stack Safety / Shield APIs, use:
```bash
llama stack run distributions/{{ name }}/run-with-safety.yaml \
--port $LLAMA_STACK_PORT \
--env INFERENCE_MODEL=Llama3.2-3B-Instruct \
--env SAFETY_MODEL=Llama-Guard-3-1B
--env INFERENCE_MODEL=meta-llama/Llama-3.2-3B-Instruct \
--env SAFETY_MODEL=meta-llama/Llama-Guard-3-1B
```

2
llama_stack/templates/meta-reference-quantized-gpu/meta_reference.py
View file

@ -84,7 +84,7 @@ def get_distribution_template() -> DistributionTemplate:
"Port for the Llama Stack distribution server",
),
"INFERENCE_MODEL": (
"Llama3.2-3B-Instruct",
"meta-llama/Llama-3.2-3B-Instruct",
"Inference model loaded into the Meta Reference server",
),
"INFERENCE_CHECKPOINT_DIR": (