test: introduce api conformance test

this test runs on each PR and uses a new conformance workflow to compare the base (main) branch openapi spec to the one on this PR
if one of our "stable" APIs change, the test will fail.

this workflow uses `oasdiff` to identify breaking changes for paths we want to ensure comptability for.

specifically this is using `oasdiff breaking` with `--match-path` which only checks breaking changes for the specified paths.

As a follow up to this, we can add an optional way to make it so that it is OK to make these change if properly documented or in a changelog or something.

related to #3237

Signed-off-by: Charlie Doern <cdoern@redhat.com>
This commit is contained in:
Charlie Doern 2025-08-25 16:00:07 -04:00
parent 3a7ac4227d
commit e23481e44a
2 changed files with 58 additions and 0 deletions

View file

@ -5,6 +5,7 @@ Llama Stack uses GitHub Actions for Continuous Integration (CI). Below is a tabl
| Name | File | Purpose |
| ---- | ---- | ------- |
| Update Changelog | [changelog.yml](changelog.yml) | Creates PR for updating the CHANGELOG.md |
| API Conformance Tests | [conformance.yml](conformance.yml) | Run the API Conformance test suite on the changes. |
| Installer CI | [install-script-ci.yml](install-script-ci.yml) | Test the installation script |
| Integration Auth Tests | [integration-auth-tests.yml](integration-auth-tests.yml) | Run the integration test suite with Kubernetes authentication |
| SqlStore Integration Tests | [integration-sql-store-tests.yml](integration-sql-store-tests.yml) | Run the integration test suite with SqlStore |