mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-12-03 09:53:45 +00:00
e809d21357
Some checks failed
SqlStore Integration Tests / test-postgres (3.12) (push) Failing after 1s
Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 1s
SqlStore Integration Tests / test-postgres (3.13) (push) Failing after 0s
Python Package Build Test / build (3.12) (push) Failing after 1s
Test External Providers Installed via Module / test-external-providers-from-module (venv) (push) Has been skipped
Python Package Build Test / build (3.13) (push) Failing after 1s
Integration Tests (Replay) / Integration Tests (, , , client=, ) (push) Failing after 4s
Unit Tests / unit-tests (3.12) (push) Failing after 4s
Unit Tests / unit-tests (3.13) (push) Failing after 4s
Test External API and Providers / test-external (venv) (push) Failing after 42s
Vector IO Integration Tests / test-matrix (push) Failing after 45s
API Conformance Tests / check-schema-compatibility (push) Successful in 54s
UI Tests / ui-tests (22) (push) Successful in 52s
Pre-commit / pre-commit (push) Successful in 3m28s
This adds automated backward compatibility testing for `run.yaml` files. As we evolve `StackRunConfig`, changes can inadvertently break existing user configurations. This workflow catches those breaks before merge. We test old run.yaml files (from main and the latest release) against the PR's new code. If configs that worked before now fail, the PR is blocked unless explicitly acknowledged as a breaking change. **Two test layers:** - Schema validation: Quick pytest checks that configs parse without errors - Integration tests: Full test suite execution to catch runtime semantic issues (cross-field validations, provider initialization, etc.) **What we test against:** - main branch: Breaking changes here block the PR (this is the gate) - Latest release: Informational only - shows if we've drifted from what users have If tests fail, the PR author must acknowledge the breaking change by adding `!:` to the PR title (e.g., `feat!: change xyz`) or including `BREAKING CHANGE:` in a commit message. Once acknowledged, the check passes with a warning. These jobs are run: 1. `check-main-compatibility` - Schema validation of all distribution run.yaml files from main 2. `test-integration-main` - Full integration test suite using main's ci-tests run.yaml 3. `test-integration-release` - Integration tests with latest release config (informational) 4. `check-schema-release-compatibility` - Schema checks against release (informational) The integration tests catch issues that schema validation alone would miss, like assertion failures in `StackRunConfig.validate_server_stores()` or provider-specific runtime logic. Resolves #3311 Related to #3237
26 lines
2.5 KiB
Markdown
26 lines
2.5 KiB
Markdown
# Llama Stack CI
|
|
|
|
Llama Stack uses GitHub Actions for Continuous Integration (CI). Below is a table detailing what CI the project includes and the purpose.
|
|
|
|
| Name | File | Purpose |
|
|
| ---- | ---- | ------- |
|
|
| Backward Compatibility Check | [backward-compat.yml](backward-compat.yml) | Check backward compatibility for run.yaml configs |
|
|
| Update Changelog | [changelog.yml](changelog.yml) | Creates PR for updating the CHANGELOG.md |
|
|
| API Conformance Tests | [conformance.yml](conformance.yml) | Run the API Conformance test suite on the changes. |
|
|
| Installer CI | [install-script-ci.yml](install-script-ci.yml) | Test the installation script |
|
|
| Integration Auth Tests | [integration-auth-tests.yml](integration-auth-tests.yml) | Run the integration test suite with Kubernetes authentication |
|
|
| SqlStore Integration Tests | [integration-sql-store-tests.yml](integration-sql-store-tests.yml) | Run the integration test suite with SqlStore |
|
|
| Integration Tests (Replay) | [integration-tests.yml](integration-tests.yml) | Run the integration test suites from tests/integration in replay mode |
|
|
| Vector IO Integration Tests | [integration-vector-io-tests.yml](integration-vector-io-tests.yml) | Run the integration test suite with various VectorIO providers |
|
|
| Pre-commit | [pre-commit.yml](pre-commit.yml) | Run pre-commit checks |
|
|
| Pre-commit Bot | [precommit-trigger.yml](precommit-trigger.yml) | Pre-commit bot for PR |
|
|
| Test Llama Stack Build | [providers-build.yml](providers-build.yml) | Test llama stack build |
|
|
| Test llama stack list-deps | [providers-list-deps.yml](providers-list-deps.yml) | Test llama stack list-deps |
|
|
| Python Package Build Test | [python-build-test.yml](python-build-test.yml) | Test building the llama-stack PyPI project |
|
|
| Integration Tests (Record) | [record-integration-tests.yml](record-integration-tests.yml) | Run the integration test suite from tests/integration |
|
|
| Check semantic PR titles | [semantic-pr.yml](semantic-pr.yml) | Ensure that PR titles follow the conventional commit spec |
|
|
| Close stale issues and PRs | [stale_bot.yml](stale_bot.yml) | Run the Stale Bot action |
|
|
| Test External Providers Installed via Module | [test-external-provider-module.yml](test-external-provider-module.yml) | Test External Provider installation via Python module |
|
|
| Test External API and Providers | [test-external.yml](test-external.yml) | Test the External API and Provider mechanisms |
|
|
| UI Tests | [ui-unit-tests.yml](ui-unit-tests.yml) | Run the UI test suite |
|
|
| Unit Tests | [unit-tests.yml](unit-tests.yml) | Run the unit test suite |
|