phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-03 09:53:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Fix incorrect entrypoint for broken llama stack run (#706)

Browse source 
This fixes the issue when using `llama stack run` by correctly
specifying entrypoint:

```
LLAMA_STACK_DIR=. llama stack run /home/yutang/.llama/distributions/llamastack-vllm/vllm-run.yaml
Using config file: /home/yutang/.llama/distributions/llamastack-vllm/vllm-run.yaml
+ command -v selinuxenabled
+ selinuxenabled
+ DOCKER_OPTS=' --security-opt label=disable'
+ mounts=
+ '[' -n . ']'
++ readlink -f .
+ mounts=' -v /home/yutang/repos/llama-stack:/app/llama-stack-source'
+ '[' -n '' ']'
+ version_tag=latest
+ '[' -n '' ']'
+ '[' -n . ']'
+ version_tag=dev
+ podman run --security-opt label=disable -it -p 5000:5000 -v /home/yutang/.llama/distributions/llamastack-vllm/vllm-run.yaml:/app/config.yaml -v /home/yutang/repos/llama-stack:/app/llama-stack-source localhost/distribution-vllm:dev python -m llama_stack.distribution.server.server --yaml-config /app/config.yaml --port 5000
usage: server.py
       [-h]
       [--yaml-config YAML_CONFIG]
       [--template TEMPLATE]
       [--port PORT]
       [--disable-ipv6]
       [--env ENV]
server.py: error: unrecognized arguments: python -m llama_stack.distribution.server.server
++ error_handler 88
++ echo 'Error occurred in script at line: 88'
Error occurred in script at line: 88
++ exit 1

```

---------

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
This commit is contained in:
Yuan Tang 2025-01-03 11:47:10 -06:00 • committed by GitHub
parent 21357a6dee
commit 96d8375663
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 9 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

7
llama_stack/distribution/server/server.py
View file

@ -239,7 +239,12 @@ def main():
"--template", "--template",
help="One of the template names in llama_stack/templates (e.g., tgi, fireworks, remote-vllm, etc.)", help="One of the template names in llama_stack/templates (e.g., tgi, fireworks, remote-vllm, etc.)",
) )
parser.add_argument("--port", type=int, default=5000, help="Port to listen on") parser.add_argument(
"--port",
type=int,
default=int(os.getenv("LLAMASTACK_PORT", 5000)),
help="Port to listen on",
)
parser.add_argument( parser.add_argument(
"--disable-ipv6", action="store_true", help="Whether to disable IPv6 support" "--disable-ipv6", action="store_true", help="Whether to disable IPv6 support"
) )

7
llama_stack/distribution/start_container.sh
View file

@ -90,7 +90,6 @@ $DOCKER_BINARY run $DOCKER_OPTS -it \
$env_vars \ $env_vars \
-v "$yaml_config:/app/config.yaml" \ -v "$yaml_config:/app/config.yaml" \
$mounts \ $mounts \
$docker_image:$version_tag \ --env LLAMASTACK_PORT=$port \
python -m llama_stack.distribution.server.server \ --entrypoint='["python", "-m", "llama_stack.distribution.server.server", "--yaml-config", "/app/config.yaml"]' \
--yaml-config /app/config.yaml \ $docker_image:$version_tag
--port "$port"