phoenix-oss/llama-stack
1
0
Fork 0
forked from phoenix-oss/llama-stack-mirror
Code Pull requests Releases Packages 2 Activity Actions

Add a pre-commit for distro_codegen but it does not work yet

Browse source
This commit is contained in:
Ashwin Bharambe 2024-11-18 15:20:49 -08:00
parent 57a9b4d57f
commit 3aedde2ab4
8 changed files with 66 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

9
docs/source/getting_started/distributions/self_hosted_distro/meta-reference-gpu.md
View file

@ -13,6 +13,15 @@ The `llamastack/distribution-meta-reference-gpu` distribution consists of the fo
Note that you need access to nvidia GPUs to run this distribution. This distribution is not compatible with CPU-only machines or machines with AMD GPUs.
### Environment Variables
The following environment variables can be configured:
- `LLAMASTACK_PORT`: Port for the Llama Stack distribution server (default: `5001`)
- `INFERENCE_MODEL`: Inference model loaded into the Meta Reference server (default: `meta-llama/Llama-3.2-3B-Instruct`)
- `INFERENCE_CHECKPOINT_DIR`: Directory containing the Meta Reference model checkpoint (default: `null`)
- `SAFETY_MODEL`: Name of the safety (Llama-Guard) model to use (default: `meta-llama/Llama-Guard-3-1B`)
- `SAFETY_CHECKPOINT_DIR`: Directory containing the Llama-Guard model checkpoint (default: `null`)
## Prerequisite: Downloading Models