phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-07-16 01:53:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update to docs

Browse source
This commit is contained in:
Ashwin Bharambe 2024-11-18 16:52:48 -08:00
parent 91f3009c67
commit e40404625b
2 changed files with 8 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

8
docs/source/getting_started/distributions/self_hosted_distro/remote-vllm.md
View file

@ -107,7 +107,7 @@ docker run \
--env INFERENCE_MODEL=$INFERENCE_MODEL \
--env VLLM_URL=http://host.docker.internal:$INFERENCE_PORT/v1 \
--env SAFETY_MODEL=$SAFETY_MODEL \
--env VLLM_SAFETY_URL=http://host.docker.internal:$SAFETY_PORT/v1
--env SAFETY_VLLM_URL=http://host.docker.internal:$SAFETY_PORT/v1
```
@ -126,7 +126,7 @@ llama stack build --template remote-vllm --image-type conda
llama stack run ./run.yaml \
--port $LLAMA_STACK_PORT \
--env INFERENCE_MODEL=$INFERENCE_MODEL \
--env VLLM_URL=http://127.0.0.1:$INFERENCE_PORT/v1
--env VLLM_URL=http://localhost:$INFERENCE_PORT/v1
```
If you are using Llama Stack Safety / Shield APIs, use:
@ -138,7 +138,7 @@ export SAFETY_MODEL=meta-llama/Llama-Guard-3-1B
llama stack run ./run-with-safety.yaml \
--port $LLAMA_STACK_PORT \
--env INFERENCE_MODEL=$INFERENCE_MODEL \
--env VLLM_URL=http://127.0.0.1:$INFERENCE_PORT/v1 \
--env VLLM_URL=http://localhost:$INFERENCE_PORT/v1 \
--env SAFETY_MODEL=$SAFETY_MODEL \
--env VLLM_SAFETY_URL=http://127.0.0.1:$SAFETY_PORT/v1
--env SAFETY_VLLM_URL=http://localhost:$SAFETY_PORT/v1
```