chore: use Pydantic to generate OpenAPI schema

Removes the need for the strong_typing and pyopenapi packages and purely
use Pydantic for schema generation.

Our generator now purely relies on Pydantic and FastAPI, it is available
at `scripts/fastapi_generator.py`, you can run it like so:

```
uv run ./scripts/run_openapi_generator.sh
```

The generator will:

* Generate the deprecated, experimental, stable and combined specs
* Validate all the spec it generates against OpenAPI standards

A few changes in the schema required for oasdiff some updates so I've
made the following ignore rules. The new Pydantic-based generator is
likely more correct and follows OpenAPI standards better than the old
pyopenapi generator. Instead of trying to make the new generator match
the old one's quirks, we should focus on what's actually correct
according to OpenAPI standards.

These are non-critical changes:

* response-property-became-nullable: Backward compatible:
  existing non-null values still work, now also accepts null
* response-required-property-removed: oasdiff reports a false
  positive because it doesn't resolve $refs inside anyOf; we could use
  tool like 'redocly' to flatten the schema to a single file.
* response-property-type-changed: properties are still object
  types, but oasdiff doesn't resolve $refs, so it flags the missing
  inline type: object even though the referenced schemas define type:
  object
* request-property-one-of-removed: These are false positives
  caused by schema restructuring (wrapping in anyOf for nullability,
  using -Input variants, or simplifying nested oneOf structures)
  that don't change the actual API contract - the same data types are
  still accepted, just represented differently in the schema.
* request-parameter-enum-value-removed: These are false
  positives caused by oasdiff not resolving $refs - the enum values
  (asc, desc, assistants, batch) are still present in the referenced
  schemas (Order and OpenAIFilePurpose), just represented via schema
  references instead of inline enums.
* request-property-enum-value-removed: this is a false positive caused
    by oasdiff not resolving $refs - the enum values (llm, embedding,
    rerank) are still present in the referenced ModelType schema,
    just represented via schema reference instead of inline enums.
* request-property-type-changed: These are schema quality issues
    where type information is missing (due to Any fallback in dynamic
    model creation), but the API contract remains unchanged -
    properties still exist with correct names and defaults, so the same
    requests will work.
* response-body-type-changed: These are false positives caused
  by schema representation changes (from inferred/empty types to
  explicit $ref schemas, or vice versa) - the actual response types
  an API contract remain unchanged, just how they're represented in the
  OpenAPI spec.
* response-media-type-removed: This is a false positive caused
  by FastAPI's OpenAPI generator not documenting union return types with
  AsyncIterator - the streaming functionality with text/event-stream
  media type still works when stream=True is passed, it's just not
  reflected in the generated OpenAPI spec.
* request-body-type-changed: This is a schema correction - the
  old spec incorrectly represented the request body as an object, but
  the function signature shows chunks: list[Chunk], so the new spec
  correctly shows it as an array, matching the actual API
  implementation.

Signed-off-by: Sébastien Han <seb@redhat.com>
This commit is contained in:
Sébastien Han 2025-10-29 14:38:56 +01:00
parent a078f089d9
commit e3cb8ed74a
No known key found for this signature in database
45 changed files with 46606 additions and 18616 deletions

1591
scripts/fastapi_generator.py Executable file

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,19 @@
#!/usr/bin/env bash
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the terms described in the LICENSE file in
# the root directory of this source tree.
PYTHONPATH=${PYTHONPATH:-}
THIS_DIR="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)"
set -euo pipefail
stack_dir=$(dirname "$THIS_DIR")
PYTHONPATH=$PYTHONPATH:$stack_dir \
python3 -m scripts.fastapi_generator "$stack_dir"/docs/static
cp "$stack_dir"/docs/static/stainless-llama-stack-spec.yaml "$stack_dir"/client-sdks/stainless/openapi.yml

290
scripts/validate_openapi.py Executable file
View file

@ -0,0 +1,290 @@
#!/usr/bin/env python3
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the terms described in the LICENSE file in
# the root directory of this source tree.
"""
OpenAPI Schema Validator for Llama Stack.
This script provides comprehensive validation of OpenAPI specifications
using multiple validation tools and approaches.
"""
import argparse
import json
import sys
from pathlib import Path
from typing import Any
import yaml
from openapi_spec_validator import validate_spec
from openapi_spec_validator.exceptions import OpenAPISpecValidatorError
def validate_openapi_schema(schema: dict[str, Any], schema_name: str = "OpenAPI schema") -> bool:
"""
Validate an OpenAPI schema using openapi-spec-validator.
Args:
schema: The OpenAPI schema dictionary to validate
schema_name: Name of the schema for error reporting
Returns:
True if valid, False otherwise
"""
try:
validate_spec(schema)
print(f"{schema_name} is valid")
return True
except OpenAPISpecValidatorError as e:
print(f"{schema_name} validation failed:")
print(f" {e}")
return False
except Exception as e:
print(f"{schema_name} validation error: {e}")
return False
def validate_schema_file(file_path: Path) -> bool:
"""
Validate an OpenAPI schema file (YAML or JSON).
Args:
file_path: Path to the schema file
Returns:
True if valid, False otherwise
"""
try:
with open(file_path) as f:
if file_path.suffix.lower() in [".yaml", ".yml"]:
schema = yaml.safe_load(f)
elif file_path.suffix.lower() == ".json":
schema = json.load(f)
else:
print(f"❌ Unsupported file format: {file_path.suffix}")
return False
return validate_openapi_schema(schema, str(file_path))
except Exception as e:
print(f"❌ Failed to read {file_path}: {e}")
return False
def validate_directory(directory: Path, pattern: str = "*.yaml") -> bool:
"""
Validate all OpenAPI schema files in a directory.
Args:
directory: Directory containing schema files
pattern: Glob pattern to match schema files
Returns:
True if all files are valid, False otherwise
"""
if not directory.exists():
print(f"❌ Directory not found: {directory}")
return False
schema_files = list(directory.glob(pattern)) + list(directory.glob("*.yml")) + list(directory.glob("*.json"))
if not schema_files:
print(f"❌ No schema files found in {directory}")
return False
print(f"🔍 Found {len(schema_files)} schema files to validate")
all_valid = True
for schema_file in schema_files:
print(f"\n📄 Validating {schema_file.name}...")
is_valid = validate_schema_file(schema_file)
if not is_valid:
all_valid = False
return all_valid
def get_schema_stats(schema: dict[str, Any]) -> dict[str, int]:
"""
Get statistics about an OpenAPI schema.
Args:
schema: The OpenAPI schema dictionary
Returns:
Dictionary with schema statistics
"""
stats = {
"paths": len(schema.get("paths", {})),
"schemas": len(schema.get("components", {}).get("schemas", {})),
"operations": 0,
"parameters": 0,
"responses": 0,
}
# Count operations
for path_info in schema.get("paths", {}).values():
for method in ["get", "post", "put", "delete", "patch", "head", "options"]:
if method in path_info:
stats["operations"] += 1
operation = path_info[method]
if "parameters" in operation:
stats["parameters"] += len(operation["parameters"])
if "responses" in operation:
stats["responses"] += len(operation["responses"])
return stats
def print_schema_stats(schema: dict[str, Any], schema_name: str = "Schema") -> None:
"""
Print statistics about an OpenAPI schema.
Args:
schema: The OpenAPI schema dictionary
schema_name: Name of the schema for display
"""
stats = get_schema_stats(schema)
print(f"\n📊 {schema_name} Statistics:")
print(f" 🛣️ Paths: {stats['paths']}")
print(f" 📋 Schemas: {stats['schemas']}")
print(f" 🔧 Operations: {stats['operations']}")
print(f" 📝 Parameters: {stats['parameters']}")
print(f" 📤 Responses: {stats['responses']}")
def main():
"""Main entry point for the OpenAPI validator."""
parser = argparse.ArgumentParser(
description="Validate OpenAPI specifications",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Validate a specific file
python validate_openapi.py docs/static/llama-stack-spec.yaml
# Validate all YAML files in a directory
python validate_openapi.py docs/static/
# Validate with detailed statistics
python validate_openapi.py docs/static/llama-stack-spec.yaml --stats
# Validate and show only errors
python validate_openapi.py docs/static/ --quiet
""",
)
parser.add_argument("path", help="Path to schema file or directory containing schema files")
parser.add_argument("--stats", action="store_true", help="Show detailed schema statistics")
parser.add_argument("--quiet", action="store_true", help="Only show errors, suppress success messages")
parser.add_argument("--pattern", default="*.yaml", help="Glob pattern for schema files (default: *.yaml)")
args = parser.parse_args()
path = Path(args.path)
if not path.exists():
print(f"❌ Path not found: {path}")
return 1
if path.is_file():
# Validate a single file
if args.quiet:
# Override the validation function to be quiet
def quiet_validate(schema, name):
try:
validate_spec(schema)
return True
except Exception as e:
print(f"{name}: {e}")
return False
try:
with open(path) as f:
if path.suffix.lower() in [".yaml", ".yml"]:
schema = yaml.safe_load(f)
elif path.suffix.lower() == ".json":
schema = json.load(f)
else:
print(f"❌ Unsupported file format: {path.suffix}")
return 1
is_valid = quiet_validate(schema, str(path))
if is_valid and args.stats:
print_schema_stats(schema, path.name)
return 0 if is_valid else 1
except Exception as e:
print(f"❌ Failed to read {path}: {e}")
return 1
else:
is_valid = validate_schema_file(path)
if is_valid and args.stats:
try:
with open(path) as f:
if path.suffix.lower() in [".yaml", ".yml"]:
schema = yaml.safe_load(f)
elif path.suffix.lower() == ".json":
schema = json.load(f)
else:
return 1
print_schema_stats(schema, path.name)
except Exception:
pass
return 0 if is_valid else 1
elif path.is_dir():
# Validate all files in directory
if args.quiet:
all_valid = True
schema_files = list(path.glob(args.pattern)) + list(path.glob("*.yml")) + list(path.glob("*.json"))
for schema_file in schema_files:
try:
with open(schema_file) as f:
if schema_file.suffix.lower() in [".yaml", ".yml"]:
schema = yaml.safe_load(f)
elif schema_file.suffix.lower() == ".json":
schema = json.load(f)
else:
continue
try:
validate_spec(schema)
except Exception as e:
print(f"{schema_file.name}: {e}")
all_valid = False
except Exception as e:
print(f"❌ Failed to read {schema_file.name}: {e}")
all_valid = False
return 0 if all_valid else 1
else:
all_valid = validate_directory(path, args.pattern)
if all_valid and args.stats:
# Show stats for all files
schema_files = list(path.glob(args.pattern)) + list(path.glob("*.yml")) + list(path.glob("*.json"))
for schema_file in schema_files:
try:
with open(schema_file) as f:
if schema_file.suffix.lower() in [".yaml", ".yml"]:
schema = yaml.safe_load(f)
elif schema_file.suffix.lower() == ".json":
schema = json.load(f)
else:
continue
print_schema_stats(schema, schema_file.name)
except Exception:
continue
return 0 if all_valid else 1
else:
print(f"❌ Invalid path type: {path}")
return 1
if __name__ == "__main__":
sys.exit(main())