phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-08-02 08:44:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

refine

Browse source
This commit is contained in:
Botao Chen 2024-12-18 16:08:17 -08:00
parent 7b0deee899
commit cd1fc4fd17
2 changed files with 4 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

4
docs/source/distributions/self_hosted_distro/meta-reference-gpu.md
View file

@ -31,9 +31,9 @@ Note that you need access to nvidia GPUs to run this distribution. This distribu
The following environment variables can be configured:
- `LLAMASTACK_PORT`: Port for the Llama Stack distribution server (default: `5001`)
- `INFERENCE_MODEL`: Inference model loaded into the Meta Reference server (default: `Llama3.2-3B-Instruct`)
- `INFERENCE_MODEL`: Inference model loaded into the Meta Reference server (default: `meta-llama/Llama-3.2-3B-Instruct`)
- `INFERENCE_CHECKPOINT_DIR`: Directory containing the Meta Reference model checkpoint (default: `null`)
- `SAFETY_MODEL`: Name of the safety (Llama-Guard) model to use (default: `Llama-Guard-3-1B`)
- `SAFETY_MODEL`: Name of the safety (Llama-Guard) model to use (default: `meta-llama/Llama-Guard-3-1B`)
- `SAFETY_CHECKPOINT_DIR`: Directory containing the Llama-Guard model checkpoint (default: `null`)

4
llama_stack/templates/meta-reference-gpu/meta_reference.py
View file

@ -112,7 +112,7 @@ def get_distribution_template() -> DistributionTemplate:
"Port for the Llama Stack distribution server",
),
"INFERENCE_MODEL": (
"Llama3.2-3B-Instruct",
"meta-llama/Llama-3.2-3B-Instruct",
"Inference model loaded into the Meta Reference server",
),
"INFERENCE_CHECKPOINT_DIR": (
@ -120,7 +120,7 @@ def get_distribution_template() -> DistributionTemplate:
"Directory containing the Meta Reference model checkpoint",
),
"SAFETY_MODEL": (
"Llama-Guard-3-1B",
"meta-llama/Llama-Guard-3-1B",
"Name of the safety (Llama-Guard) model to use",
),
"SAFETY_CHECKPOINT_DIR": (