llama-stack-mirror/.github/workflows
Ashwin Bharambe a8aa815b6a
feat(tests): migrate to global "setups" system for test configuration (#3390)
This PR refactors the integration test system to use global "setups"
which provides better separation of concerns:

**suites = what to test, setups = how to configure.**

NOTE: if you naming suggestions, please provide feedback

Changes:
- New `tests/integration/setups.py` with global, reusable configurations
(ollama, vllm, gpt, claude)
- Modified `scripts/integration-tests.sh` options to match with the
underlying pytest options
    - Updated documentation to reflect the new global setup system

The main benefit is that setups can be reused across multiple suites
(e.g., use "gpt" with any suite) even though sometimes they could
specifically tailored for a suite (vision <> ollama-vision). It is now
easier to add new configurations without modifying existing suites.

Usage examples:
    - `pytest tests/integration --suite=responses --setup=gpt`
- `pytest tests/integration --suite=vision` # auto-selects
"ollama-vision" setup
    - `pytest tests/integration --suite=base --setup=vllm`
2025-09-09 15:50:56 -07:00
..
changelog.yml chore(github-deps): bump actions/checkout from 4.2.2 to 5.0.0 (#3178) 2025-08-20 16:51:40 -07:00
conformance.yml chore(github-deps): bump actions/checkout from 4.1.7 to 5.0.0 (#3357) 2025-09-08 10:07:03 +02:00
install-script-ci.yml chore(github-deps): bump actions/checkout from 4.2.2 to 5.0.0 (#3178) 2025-08-20 16:51:40 -07:00
integration-auth-tests.yml fix(ci): make all CI workflows have the correct concurrency defn 2025-08-21 16:05:25 -07:00
integration-sql-store-tests.yml fix(ci): make all CI workflows have the correct concurrency defn 2025-08-21 16:05:25 -07:00
integration-tests.yml feat(tests): migrate to global "setups" system for test configuration (#3390) 2025-09-09 15:50:56 -07:00
integration-vector-io-tests.yml chore(github-deps): bump actions/checkout from 4.2.2 to 5.0.0 (#3178) 2025-08-20 16:51:40 -07:00
pre-commit.yml chore(github-deps): bump actions/setup-python from 5.6.0 to 6.0.0 (#3354) 2025-09-08 10:05:34 +02:00
providers-build.yml fix(ci): small fixes to the provider build workflow 2025-08-21 16:37:11 -07:00
python-build-test.yml chore(github-deps): bump astral-sh/setup-uv from 6.6.0 to 6.6.1 (#3355) 2025-09-08 10:00:41 +02:00
README.md test: introduce api conformance test (#3257) 2025-09-06 12:40:33 -07:00
record-integration-tests.yml feat(tests): migrate to global "setups" system for test configuration (#3390) 2025-09-09 15:50:56 -07:00
semantic-pr.yml chore(github-deps): bump amannn/action-semantic-pull-request from 6.1.0 to 6.1.1 (#3248) 2025-08-25 17:34:17 +02:00
stale_bot.yml chore(github-deps): bump actions/stale from 9.1.0 to 10.0.0 (#3352) 2025-09-08 10:04:41 +02:00
test-external-provider-module.yml chore(github-deps): bump actions/checkout from 4.2.2 to 5.0.0 (#3178) 2025-08-20 16:51:40 -07:00
test-external.yml chore(github-deps): bump actions/checkout from 4.2.2 to 5.0.0 (#3178) 2025-08-20 16:51:40 -07:00
ui-unit-tests.yml chore(github-deps): bump actions/setup-node from 4.4.0 to 5.0.0 (#3353) 2025-09-08 10:05:00 +02:00
unit-tests.yml fix(ci): make all CI workflows have the correct concurrency defn 2025-08-21 16:05:25 -07:00
update-readthedocs.yml fix(ci): make all CI workflows have the correct concurrency defn 2025-08-21 16:05:25 -07:00

Llama Stack CI

Llama Stack uses GitHub Actions for Continuous Integration (CI). Below is a table detailing what CI the project includes and the purpose.

Name File Purpose
Update Changelog changelog.yml Creates PR for updating the CHANGELOG.md
API Conformance Tests conformance.yml Run the API Conformance test suite on the changes.
Installer CI install-script-ci.yml Test the installation script
Integration Auth Tests integration-auth-tests.yml Run the integration test suite with Kubernetes authentication
SqlStore Integration Tests integration-sql-store-tests.yml Run the integration test suite with SqlStore
Integration Tests (Replay) integration-tests.yml Run the integration test suites from tests/integration in replay mode
Vector IO Integration Tests integration-vector-io-tests.yml Run the integration test suite with various VectorIO providers
Pre-commit pre-commit.yml Run pre-commit checks
Test Llama Stack Build providers-build.yml Test llama stack build
Python Package Build Test python-build-test.yml Test building the llama-stack PyPI project
Integration Tests (Record) record-integration-tests.yml Run the integration test suite from tests/integration
Check semantic PR titles semantic-pr.yml Ensure that PR titles follow the conventional commit spec
Close stale issues and PRs stale_bot.yml Run the Stale Bot action
Test External Providers Installed via Module test-external-provider-module.yml Test External Provider installation via Python module
Test External API and Providers test-external.yml Test the External API and Provider mechanisms
UI Tests ui-unit-tests.yml Run the UI test suite
Unit Tests unit-tests.yml Run the unit test suite
Update ReadTheDocs update-readthedocs.yml Update the Llama Stack ReadTheDocs site