phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-08-06 10:42:39 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

env var and fix logs

Browse source 
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
This commit is contained in:
Yuan Tang 2025-01-29 21:39:29 -05:00
parent ddba43fada
commit 3bcc778e16
No known key found for this signature in database
1 changed files with 5 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

10
docs/source/distributions/kubernetes_deployment.md
View file

@ -8,7 +8,7 @@ First, create a local Kubernetes cluster via Kind:
kind create cluster --image kindest/node:v1.32.0 --name llama-stack-test
```
Start vLLM server as a Kubernetes Pod and Service (remember to replace `<YOUR-HF-TOKEN>` with your actual token and `<VLLM-IMAGE>` to meet your local system architecture):
Start vLLM server as a Kubernetes Pod and Service:
```bash
cat <<EOF |kubectl apply -f -
@ -30,7 +30,7 @@ metadata:
name: hf-token-secret
type: Opaque
data:
token: "<YOUR-HF-TOKEN>"
token: $(HF_TOKEN)
---
apiVersion: apps/v1
kind: Deployment
@ -48,7 +48,7 @@ spec:
spec:
containers:
- name: llama-stack
image: <VLLM-IMAGE>
image: $(VLLM_IMAGE)
command:
- bash
- -c
@ -92,7 +92,7 @@ EOF
We can verify that the vLLM server has started successfully via the logs (this might take a couple of minutes to download the model):
```bash
$ kubectl logs vllm-server
$ kubectl logs -l app.kubernetes.io/name=vllm
...
INFO: Started server process [1]
INFO: Waiting for application startup.
@ -190,7 +190,7 @@ EOF
We can check that the LlamaStack server has started:
```bash
$ kubectl logs vllm-server
$ kubectl logs -l app.kubernetes.io/name=llama-stack
...
INFO: Started server process [1]
INFO: Waiting for application startup.