phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-08-03 01:03:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs: Fix missing --gpu all flag in Docker run commands

Browse source 
dding the --gpu all flag to Docker run commands
for meta-reference-gpu distributions ensures models are
loaded into GPU instead of CPU.

Fixes: #1798

Signed-off-by: Derek Higgins <derekh@redhat.com>
This commit is contained in:
Derek Higgins 2025-04-25 12:06:08 +01:00
parent 7397534497
commit ec4c04fa2d
2 changed files with 4 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/source/distributions/self_hosted_distro/meta-reference-gpu.md
View file

@ -81,6 +81,7 @@ LLAMA_STACK_PORT=8321
docker run \ docker run \
-it \ -it \
--pull always \ --pull always \
--gpu all \
-p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \ -p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \
-v ~/.llama:/root/.llama \ -v ~/.llama:/root/.llama \
llamastack/distribution-meta-reference-gpu \ llamastack/distribution-meta-reference-gpu \
@ -94,6 +95,7 @@ If you are using Llama Stack Safety / Shield APIs, use:
docker run \ docker run \
-it \ -it \
--pull always \ --pull always \
--gpu all \
-p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \ -p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \
-v ~/.llama:/root/.llama \ -v ~/.llama:/root/.llama \
llamastack/distribution-meta-reference-gpu \ llamastack/distribution-meta-reference-gpu \

2
llama_stack/templates/meta-reference-gpu/doc_template.md
View file

@ -69,6 +69,7 @@ LLAMA_STACK_PORT=8321
docker run \ docker run \
-it \ -it \
--pull always \ --pull always \
--gpu all \
-p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \ -p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \
-v ~/.llama:/root/.llama \ -v ~/.llama:/root/.llama \
llamastack/distribution-{{ name }} \ llamastack/distribution-{{ name }} \
@ -82,6 +83,7 @@ If you are using Llama Stack Safety / Shield APIs, use:
docker run \ docker run \
-it \ -it \
--pull always \ --pull always \
--gpu all \
-p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \ -p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \
-v ~/.llama:/root/.llama \ -v ~/.llama:/root/.llama \
llamastack/distribution-{{ name }} \ llamastack/distribution-{{ name }} \