llama-stack-mirror/tests/integration/recordings/responses/c42baf6a3700.json
Matthew Farrellee c2a9c65fff chore: update the vLLM inference impl to use OpenAIMixin for openai-compat functions
inference recordings from Qwen3-0.6B and vLLM 0.8.3 -
```
docker run --gpus all -v ~/.cache/huggingface:/root/.cache/huggingface -p 8000:8000 --ipc=host \
    vllm/vllm-openai:latest \
    --model Qwen/Qwen3-0.6B --enable-auto-tool-choice --tool-call-parser hermes
```

test with -

```
./scripts/integration-tests.sh --stack-config server:ci-tests --setup vllm --subdirs inference
```
2025-09-10 10:10:10 -04:00

66 lines
2.2 KiB
JSON

{
"request": {
"method": "POST",
"url": "http://localhost:8000/v1/v1/chat/completions",
"headers": {},
"body": {
"model": "Qwen/Qwen3-0.6B",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Which planet do humans live on?"
}
]
}
],
"stream": false,
"temperature": 0.0,
"max_tokens": 4096
},
"endpoint": "/v1/chat/completions",
"model": "Qwen/Qwen3-0.6B"
},
"response": {
"body": {
"__type__": "openai.types.chat.chat_completion.ChatCompletion",
"__data__": {
"id": "chatcmpl-f7cda312aefe4a4ca4c34e7342f23802",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"message": {
"content": "<think>\nOkay, the user is asking which planet humans live on. Let me think. I know that humans are on Earth. But maybe they're confused because sometimes people think of other planets. I should clarify that Earth is the only planet where humans have been found. Also, maybe mention that Earth is a unique planet with all the necessary conditions for life. I should keep it simple and direct.\n</think>\n\nHumans live on **Earth**. This planet is unique in having the conditions necessary for life, such as liquid water, breathable air, and suitable temperatures.",
"refusal": null,
"role": "assistant",
"annotations": null,
"audio": null,
"function_call": null,
"tool_calls": [],
"reasoning_content": null
},
"stop_reason": null
}
],
"created": 1757524135,
"model": "Qwen/Qwen3-0.6B",
"object": "chat.completion",
"service_tier": null,
"system_fingerprint": null,
"usage": {
"completion_tokens": 114,
"prompt_tokens": 15,
"total_tokens": 129,
"completion_tokens_details": null,
"prompt_tokens_details": null
},
"prompt_logprobs": null
}
},
"is_streaming": false
}
}