feat(api): oasdiff OpenAI openAPI spec against ours (#3529)

# What does this PR do?

diff the `/v1/` routes that are OpenAI compatible against the OpenAI
openAPI spec. This will of course only trigger on PRs where the spec is
changed.

This will catch errors with new handwritten additions to our openAI
compat routes.

Instead of fetching the OpenAPI spec from a dynamic URL, which could
cause non-deterministic build failures,

this change uses a local copy stored at `docs/static/openai-spec.yml`.

This makes the conformance check fully reproducible and prevents CI
failures caused by uncontrolled upstream changes.

I am marking this test with `continue-on-error: true`, until we get rid
of all of the errors. Nevertheless, this is a nice utility to have so
folks know if their spec changes introduce more breaking changes or fix
breakages when comparing to the OpenAI openapi spec.

## Test Plan

test should pass.

Signed-off-by: Charlie Doern <cdoern@redhat.com>

.github/workflows/conformance.yml

@@ -64,6 +64,7 @@ jobs:
           ref: ${{ github.event.pull_request.base.ref }}
           path: 'base'
 
+
       # Cache oasdiff to avoid checksum failures and speed up builds
       - name: Cache oasdiff
         if: steps.skip-check.outputs.skip != 'true'
@@ -136,6 +137,23 @@ jobs:
         run: |
           oasdiff breaking --fail-on ERR $BASE_SPEC $CURRENT_SPEC --match-path '^/v1/'
 
+      # Run oasdiff to detect breaking changes in the API specification when compared to the OpenAI openAPI spec
+      - name: Run OpenAPI Breaking Change Diff Against OpenAI API
+        if: steps.skip-check.outputs.skip != 'true'
+        continue-on-error: true
+        shell: bash
+        run: |
+          OPENAI_SPEC=docs/static/openai-spec-2.3.0.yml
+          LLAMA_STACK_SPEC=docs/static/llama-stack-spec.yaml
+
+          # Compare Llama Stack spec against OpenAI spec.
+          # This finds breaking changes in our implementation of common endpoints.
+          # By using our spec as the base, we avoid errors for endpoints we don't implement.
+          oasdiff breaking --fail-on ERR \
+            "$LLAMA_STACK_SPEC" \
+            "$OPENAI_SPEC" \
+            --strip-prefix-base "/v1"
+
       # Report when test is skipped
       - name: Report skip reason
         if: steps.skip-check.outputs.skip == 'true'

.pre-commit-config.yaml

@@ -19,6 +19,7 @@ repos:
     -   id: no-commit-to-branch
     -   id: check-yaml
         args: ["--unsafe"]
+        exclude: 'docs/static/openai-spec-2.3.0.yml'
     -   id: detect-private-key
     -   id: mixed-line-ending
         args: [--fix=lf] # Forces to replace line ending by LF (line feed)