phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-03 09:53:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

ci: Add vLLM support to integration testing infrastructure (with qwen) (#3545)
Some checks failed
SqlStore Integration Tests / test-postgres (3.12) (push) Failing after 1s
Details
Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 2s
Details
SqlStore Integration Tests / test-postgres (3.13) (push) Failing after 1s
Details
Integration Tests (Replay) / generate-matrix (push) Successful in 4s
Details
Python Package Build Test / build (3.13) (push) Failing after 2s
Details
Test External Providers Installed via Module / test-external-providers-from-module (venv) (push) Has been skipped
Details
Vector IO Integration Tests / test-matrix (push) Failing after 6s
Details
Pre-commit / pre-commit (push) Failing after 6s
Details
Test External API and Providers / test-external (venv) (push) Failing after 5s
Details
API Conformance Tests / check-schema-compatibility (push) Successful in 14s
Details
Integration Tests (Replay) / Integration Tests (, , , client=, ) (push) Failing after 5s
Details
Python Package Build Test / build (3.12) (push) Failing after 22s
Details
UI Tests / ui-tests (22) (push) Successful in 57s
Details

Browse source 
o Introduces vLLM provider support to the record/replay testing
framework
o Enabling both recording and replay of vLLM API interactions alongside
existing Ollama support.

The changes enable testing of vLLM functionality. vLLM tests focus on
inference capabilities, while Ollama continues to exercise the full API
surface
including vision features.

--
This is an alternative to #3128 , using qwen3 instead of llama 3.2 1B
appears to be more capable at structure output and tool calls.

---------

Signed-off-by: Derek Higgins <derekh@redhat.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This commit is contained in:
Derek Higgins 2025-11-06 09:36:40 +00:00 • committed by GitHub
parent bef1b044bd
commit c62a09ab76
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
67 changed files with 8261 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

5
scripts/integration-tests.sh
View file

@ -405,11 +405,6 @@ fi
echo "=== Running Integration Tests ==="
EXCLUDE_TESTS="builtin_tool or safety_with_image or code_interpreter or test_rag"
# Additional exclusions for vllm setup
if [[ "$TEST_SETUP" == "vllm" ]]; then
EXCLUDE_TESTS="${EXCLUDE_TESTS} or test_inference_store_tool_calls"
fi
PYTEST_PATTERN="not( $EXCLUDE_TESTS )"
if [[ -n "$TEST_PATTERN" ]]; then
PYTEST_PATTERN="${PYTEST_PATTERN} and $TEST_PATTERN"