mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-10-07 12:47:37 +00:00
c2a9c65fff
inference recordings from Qwen3-0.6B and vLLM 0.8.3 -
```
docker run --gpus all -v ~/.cache/huggingface:/root/.cache/huggingface -p 8000:8000 --ipc=host \
vllm/vllm-openai:latest \
--model Qwen/Qwen3-0.6B --enable-auto-tool-choice --tool-call-parser hermes
```
test with -
```
./scripts/integration-tests.sh --stack-config server:ci-tests --setup vllm --subdirs inference
```
|
||
|---|---|---|
| .. | ||
| bedrock | ||
| common | ||
| datasetio | ||
| inference | ||
| kvstore | ||
| memory | ||
| responses | ||
| scoring | ||
| sqlstore | ||
| telemetry | ||
| tools | ||
| vector_io | ||
| __init__.py | ||
| pagination.py | ||
| scheduler.py | ||