llama-stack-mirror/.github/workflows/README.md
Charlie Doern ecd9d8dc1a
Some checks failed
Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 1s
SqlStore Integration Tests / test-postgres (3.12) (push) Failing after 0s
SqlStore Integration Tests / test-postgres (3.13) (push) Failing after 0s
Test External Providers Installed via Module / test-external-providers-from-module (venv) (push) Has been skipped
Python Package Build Test / build (3.12) (push) Failing after 1s
Vector IO Integration Tests / test-matrix (push) Failing after 4s
Python Package Build Test / build (3.13) (push) Failing after 3s
Integration Tests (Replay) / Integration Tests (, , , client=, ) (push) Failing after 7s
Test External API and Providers / test-external (venv) (push) Failing after 4s
Unit Tests / unit-tests (3.12) (push) Failing after 4s
Unit Tests / unit-tests (3.13) (push) Failing after 4s
API Conformance Tests / check-schema-compatibility (push) Successful in 12s
UI Tests / ui-tests (22) (push) Successful in 40s
Pre-commit / pre-commit (push) Successful in 1m48s
test: introduce api conformance test (#3257)
# What does this PR do?

this test runs on each PR and uses a new conformance workflow to compare
the base (main) branch openapi spec to the one on this PR if one of our
"stable" APIs change, the test will fail.

this workflow uses `oasdiff` to identify breaking changes for paths we
want to ensure comptability for.

specifically this is using `oasdiff breaking` with `--match-path` which
only checks breaking changes for the specified paths.

As a follow up to this, we can add an optional way to make it so that it
is OK to make these change if properly documented or in a changelog or
something. or by using a label on the PR to override the failing test.

related to #3237


## Test Plan

conformance test should pass given there are no changes

Signed-off-by: Charlie Doern <cdoern@redhat.com>
2025-09-06 12:40:33 -07:00

2.3 KiB

Llama Stack CI

Llama Stack uses GitHub Actions for Continuous Integration (CI). Below is a table detailing what CI the project includes and the purpose.

Name File Purpose
Update Changelog changelog.yml Creates PR for updating the CHANGELOG.md
API Conformance Tests conformance.yml Run the API Conformance test suite on the changes.
Installer CI install-script-ci.yml Test the installation script
Integration Auth Tests integration-auth-tests.yml Run the integration test suite with Kubernetes authentication
SqlStore Integration Tests integration-sql-store-tests.yml Run the integration test suite with SqlStore
Integration Tests (Replay) integration-tests.yml Run the integration test suites from tests/integration in replay mode
Vector IO Integration Tests integration-vector-io-tests.yml Run the integration test suite with various VectorIO providers
Pre-commit pre-commit.yml Run pre-commit checks
Test Llama Stack Build providers-build.yml Test llama stack build
Python Package Build Test python-build-test.yml Test building the llama-stack PyPI project
Integration Tests (Record) record-integration-tests.yml Run the integration test suite from tests/integration
Check semantic PR titles semantic-pr.yml Ensure that PR titles follow the conventional commit spec
Close stale issues and PRs stale_bot.yml Run the Stale Bot action
Test External Providers Installed via Module test-external-provider-module.yml Test External Provider installation via Python module
Test External API and Providers test-external.yml Test the External API and Provider mechanisms
UI Tests ui-unit-tests.yml Run the UI test suite
Unit Tests unit-tests.yml Run the unit test suite
Update ReadTheDocs update-readthedocs.yml Update the Llama Stack ReadTheDocs site