mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-12-03 09:53:45 +00:00
78a676e231
Integration tests now support replaying TypeScript SDK tests alongside Python tests when running against server-mode stacks. This enables cross-language validation of API contracts and ensures the TypeScript client properly handles recorded responses. The implementation adds a new `RUN_CLIENT_TS_TESTS` environment variable that triggers TypeScript test execution after successful Python runs. A mapping file (`suites.json`) defines which TypeScript test files correspond to each Python test suite/setup combination. The script automatically installs npm dependencies, forwards server configuration (base URL and model defaults from setup definitions), and executes matching TypeScript tests using Jest. CI integration is enabled for server-based test jobs, and the feature can be exercised locally with commands like: \`\`\`bash RUN_CLIENT_TS_TESTS=1 scripts/integration-tests.sh --stack-config server:ci-tests --suite responses --setup gpt \`\`\` The TypeScript tests reuse existing replay fixtures through the forwarded \`TEST_API_BASE_URL\`, avoiding the need for duplicate response recordings. |
||
|---|---|---|
| .. | ||
| backward-compat.yml | ||
| changelog.yml | ||
| conformance.yml | ||
| install-script-ci.yml | ||
| integration-auth-tests.yml | ||
| integration-sql-store-tests.yml | ||
| integration-tests.yml | ||
| integration-vector-io-tests.yml | ||
| pre-commit.yml | ||
| providers-build.yml | ||
| providers-list-deps.yml | ||
| python-build-test.yml | ||
| README.md | ||
| record-integration-tests.yml | ||
| semantic-pr.yml | ||
| stainless-builds.yml | ||
| stale_bot.yml | ||
| test-external-provider-module.yml | ||
| test-external.yml | ||
| ui-unit-tests.yml | ||
| unit-tests.yml | ||
Llama Stack CI
Llama Stack uses GitHub Actions for Continuous Integration (CI). Below is a table detailing what CI the project includes and the purpose.
| Name | File | Purpose |
|---|---|---|
| Backward Compatibility Check | backward-compat.yml | Check backward compatibility for run.yaml configs |
| Update Changelog | changelog.yml | Creates PR for updating the CHANGELOG.md |
| API Conformance Tests | conformance.yml | Run the API Conformance test suite on the changes. |
| Installer CI | install-script-ci.yml | Test the installation script |
| Integration Auth Tests | integration-auth-tests.yml | Run the integration test suite with Kubernetes authentication |
| SqlStore Integration Tests | integration-sql-store-tests.yml | Run the integration test suite with SqlStore |
| Integration Tests (Replay) | integration-tests.yml | Run the integration test suites from tests/integration in replay mode |
| Vector IO Integration Tests | integration-vector-io-tests.yml | Run the integration test suite with various VectorIO providers |
| Pre-commit | pre-commit.yml | Run pre-commit checks |
| Test Llama Stack Build | providers-build.yml | Test llama stack build |
| Test llama stack list-deps | providers-list-deps.yml | Test llama stack list-deps |
| Python Package Build Test | python-build-test.yml | Test building the llama-stack PyPI project |
| Integration Tests (Record) | record-integration-tests.yml | Run the integration test suite from tests/integration |
| Check semantic PR titles | semantic-pr.yml | Ensure that PR titles follow the conventional commit spec |
| Stainless SDK Builds | stainless-builds.yml | Build Stainless SDK from OpenAPI spec changes |
| Close stale issues and PRs | stale_bot.yml | Run the Stale Bot action |
| Test External Providers Installed via Module | test-external-provider-module.yml | Test External Provider installation via Python module |
| Test External API and Providers | test-external.yml | Test the External API and Provider mechanisms |
| UI Tests | ui-unit-tests.yml | Run the UI test suite |
| Unit Tests | unit-tests.yml | Run the unit test suite |