phoenix-oss/llama-stack-mirror

mirror of https://github.com/meta-llama/llama-stack.git synced 2025-06-27 18:50:41 +00:00

History

Sébastien Han 6039d922c0 Some checks failed Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 8s Details Integration Tests / test-matrix (http, 3.11, datasets) (push) Failing after 8s Details Integration Tests / test-matrix (http, 3.11, providers) (push) Failing after 6s Details Integration Tests / test-matrix (http, 3.11, agents) (push) Failing after 9s Details Integration Tests / test-matrix (http, 3.11, post_training) (push) Failing after 9s Details Integration Tests / test-matrix (http, 3.11, vector_io) (push) Failing after 7s Details Integration Tests / test-matrix (http, 3.12, inspect) (push) Failing after 6s Details Integration Tests / test-matrix (http, 3.12, tool_runtime) (push) Failing after 6s Details Integration Tests / test-matrix (http, 3.11, inspect) (push) Failing after 13s Details Integration Tests / test-matrix (library, 3.11, datasets) (push) Failing after 5s Details Integration Tests / test-matrix (http, 3.11, tool_runtime) (push) Failing after 13s Details Integration Tests / test-matrix (http, 3.12, inference) (push) Failing after 11s Details Integration Tests / test-matrix (http, 3.12, scoring) (push) Failing after 12s Details Integration Tests / test-matrix (library, 3.11, agents) (push) Failing after 10s Details Integration Tests / test-matrix (library, 3.11, scoring) (push) Failing after 5s Details Integration Tests / test-matrix (library, 3.11, vector_io) (push) Failing after 5s Details Integration Tests / test-matrix (library, 3.12, datasets) (push) Failing after 5s Details Integration Tests / test-matrix (library, 3.11, inspect) (push) Failing after 13s Details Integration Tests / test-matrix (library, 3.11, providers) (push) Failing after 11s Details Integration Tests / test-matrix (library, 3.11, tool_runtime) (push) Failing after 10s Details Integration Tests / test-matrix (library, 3.12, inference) (push) Failing after 6s Details Integration Tests / test-matrix (library, 3.12, agents) (push) Failing after 8s Details Integration Tests / test-matrix (library, 3.12, inspect) (push) Failing after 6s Details Integration Tests / test-matrix (library, 3.12, post_training) (push) Failing after 5s Details Integration Tests / test-matrix (http, 3.11, scoring) (push) Failing after 28s Details Integration Tests / test-matrix (http, 3.12, providers) (push) Failing after 24s Details Integration Tests / test-matrix (http, 3.12, datasets) (push) Failing after 26s Details Integration Tests / test-matrix (http, 3.11, inference) (push) Failing after 30s Details Integration Tests / test-matrix (http, 3.12, agents) (push) Failing after 28s Details Integration Tests / test-matrix (http, 3.12, post_training) (push) Failing after 26s Details Integration Tests / test-matrix (http, 3.12, vector_io) (push) Failing after 23s Details Test Llama Stack Build / generate-matrix (push) Successful in 5s Details Integration Tests / test-matrix (library, 3.12, scoring) (push) Failing after 5s Details Test Llama Stack Build / build-custom-container-distribution (push) Failing after 5s Details Test External Providers / test-external-providers (venv) (push) Failing after 5s Details Integration Tests / test-matrix (library, 3.11, post_training) (push) Failing after 20s Details Integration Tests / test-matrix (library, 3.12, providers) (push) Failing after 7s Details Unit Tests / unit-tests (3.11) (push) Failing after 7s Details Update ReadTheDocs / update-readthedocs (push) Failing after 6s Details Integration Tests / test-matrix (library, 3.12, vector_io) (push) Failing after 8s Details Integration Tests / test-matrix (library, 3.11, inference) (push) Failing after 22s Details Test Llama Stack Build / build (push) Failing after 17s Details Unit Tests / unit-tests (3.13) (push) Failing after 37s Details Integration Tests / test-matrix (library, 3.12, tool_runtime) (push) Failing after 1m7s Details Test Llama Stack Build / build-single-provider (push) Failing after 1m15s Details Test Llama Stack Build / build-ubi9-container-distribution (push) Failing after 1m17s Details Unit Tests / unit-tests (3.12) (push) Failing after 1m32s Details Pre-commit / pre-commit (push) Failing after 2m14s Details fix: allow running vector tests with embedding dimension (#2467 ) # What does this PR do? Do not force 384 for the embedding dimension, use the one provided by the test run. ## Test Plan ``` pytest -s -vvv tests/integration/vector_io/test_vector_io.py --stack-config=http://localhost:8321 \ -k "not(builtin_tool or safety_with_image or code_interpreter or test_rag)" \ --text-model="meta-llama/Llama-3.2-3B-Instruct" \ --embedding-model=granite-embedding-125m --embedding-dimension=768 Uninstalled 1 package in 16ms Installed 1 package in 11ms INFO 2025-06-18 10:52:03,314 tests.integration.conftest:59 tests: Setting DISABLE_CODE_SANDBOX=1 for macOS /Users/leseb/Documents/AI/llama-stack/.venv/lib/python3.10/site-packages/pytest_asyncio/plugin.py:207: PytestDeprecationWarning: The configuration option "asyncio_default_fixture_loop_scope" is unset. The event loop scope for asynchronous fixtures will default to the fixture caching scope. Future versions of pytest-asyncio will default the loop scope for asynchronous fixtures to function scope. Set the default fixture loop scope explicitly in order to avoid unexpected behavior in the future. Valid fixture loop scopes are: "function", "class", "module", "package", "session" warnings.warn(PytestDeprecationWarning(_DEFAULT_FIXTURE_LOOP_SCOPE_UNSET)) ================================================= test session starts ================================================= platform darwin -- Python 3.10.16, pytest-8.3.4, pluggy-1.5.0 -- /Users/leseb/Documents/AI/llama-stack/.venv/bin/python cachedir: .pytest_cache metadata: {'Python': '3.10.16', 'Platform': 'macOS-15.5-arm64-arm-64bit', 'Packages': {'pytest': '8.3.4', 'pluggy': '1.5.0'}, 'Plugins': {'cov': '6.0.0', 'html': '4.1.1', 'json-report': '1.5.0', 'timeout': '2.4.0', 'metadata': '3.1.1', 'asyncio': '0.25.3', 'anyio': '4.8.0', 'nbval': '0.11.0'}} rootdir: /Users/leseb/Documents/AI/llama-stack configfile: pyproject.toml plugins: cov-6.0.0, html-4.1.1, json-report-1.5.0, timeout-2.4.0, metadata-3.1.1, asyncio-0.25.3, anyio-4.8.0, nbval-0.11.0 asyncio: mode=strict, asyncio_default_fixture_loop_scope=None collected 8 items tests/integration/vector_io/test_vector_io.py::test_vector_db_retrieve[emb=granite-embedding-125m:dim=768] PASSED tests/integration/vector_io/test_vector_io.py::test_vector_db_register[emb=granite-embedding-125m:dim=768] PASSED tests/integration/vector_io/test_vector_io.py::test_insert_chunks[emb=granite-embedding-125m:dim=768-test_case0] PASSED tests/integration/vector_io/test_vector_io.py::test_insert_chunks[emb=granite-embedding-125m:dim=768-test_case1] PASSED tests/integration/vector_io/test_vector_io.py::test_insert_chunks[emb=granite-embedding-125m:dim=768-test_case2] PASSED tests/integration/vector_io/test_vector_io.py::test_insert_chunks[emb=granite-embedding-125m:dim=768-test_case3] PASSED tests/integration/vector_io/test_vector_io.py::test_insert_chunks[emb=granite-embedding-125m:dim=768-test_case4] PASSED tests/integration/vector_io/test_vector_io.py::test_insert_chunks_with_precomputed_embeddings[emb=granite-embedding-125m:dim=768] PASSED ================================================== 8 passed in 5.50s ================================================== ``` Signed-off-by: Sébastien Han <seb@redhat.com>		2025-06-19 13:29:04 +05:30
..
client-sdk/post_training	feat: Add nemo customizer (#1448 )	2025-03-25 11:01:10 -07:00
common	feat(responses): implement full multi-turn support (#2295 )	2025-06-02 15:35:49 -07:00
external-provider/llama-stack-provider-ollama	feat: drop python 3.10 support (#2469 )	2025-06-19 12:07:14 +05:30
integration	fix: allow running vector tests with embedding dimension (#2467 )	2025-06-19 13:29:04 +05:30
unit	feat: drop python 3.10 support (#2469 )	2025-06-19 12:07:14 +05:30
verifications	feat: support filters in file search (#2472 )	2025-06-18 21:50:55 -07:00
__init__.py	refactor(test): introduce --stack-config and simplify options (#1404 )	2025-03-05 17:02:02 -08:00
Containerfile	ci: use ollama container image with loaded models (#2410 )	2025-06-06 12:08:20 +02:00
README.md	docs: revamp testing documentation (#2155 )	2025-05-13 11:28:29 -07:00

README.md

Llama Stack Tests

Llama Stack has multiple layers of testing done to ensure continuous functionality and prevent regressions to the codebase.

Testing Type	Details
Unit	unit/README.md
Integration	integration/README.md
Verification	verifications/README.md