llama-stack-mirror/scripts
Emilio Garcia ba50790a28
feat(tests): metrics tests (#3966)
# What does this PR do?
1. Make telemetry tests as easy as possible for users by expanding the
`SpanStub` data class and creating the `MetricStub` dataclass as a way
to consistently marshal telemetry data in test fixtures and unmarshal
and handle it in tests.
2. Structure server and client tests to always follow the same standards
for consistent testing experience by using the `SpanStub` and
`MetricStub` data class objects.
3. Enable Metrics Testing for completions endpoint
4. Correct token metrics to use histograms instead of counts to capture
tokens per request rather than a cumulative count of tokens over the
lifecycle of the server.

## Test Plan
These are tests
2025-11-05 10:26:15 -08:00
..
github fix: unbound variable PR_HEAD_REPO (#3469) 2025-09-17 10:18:43 +02:00
telemetry chore(telemetry): add grafana dashboards (#3921) 2025-10-27 14:58:27 -07:00
check-init-py.sh chore(package): migrate to src/ layout (#3920) 2025-10-27 12:02:21 -07:00
check-workflows-use-hashes.sh fix: update check-workflows-use-hashes to use github error format (#2875) 2025-07-24 17:41:17 +02:00
cleanup_recordings.py chore(ci): remove unused recordings (#4074) 2025-11-05 09:21:58 -08:00
diagnose_recordings.py feat(ci): add support for docker:distro in tests (#3832) 2025-10-16 19:33:13 -07:00
distro_codegen.py chore: remove leftover llama_stack directory (#3940) 2025-10-28 05:09:08 -07:00
docker.sh fix(ci): use test.pypi as extra index for RC dependencies (#4009) 2025-10-31 12:55:43 -07:00
gen-changelog.py chore: enable ruff for ./scripts too (#1643) 2025-03-18 12:17:21 -07:00
gen-ci-docs.py chore: Remove coverage badge from README.md (#2976) 2025-07-31 09:21:30 -07:00
generate_ci_matrix.py chore(ci): remove unused recordings (#4074) 2025-11-05 09:21:58 -08:00
generate_prompt_format.py chore: standardize model not found error (#2964) 2025-07-30 12:19:53 -07:00
get_setup_env.py fix: pre-commit issues: non executable shebang file and removal of @pytest.mark.asyncio decorator (#3397) 2025-09-10 15:27:35 +02:00
install.sh chore(telemetry): add grafana dashboards (#3921) 2025-10-27 14:58:27 -07:00
integration-tests.sh feat(tests): metrics tests (#3966) 2025-11-05 10:26:15 -08:00
normalize_recordings.py fix: update normalize to search all recordings dirs (#3767) 2025-10-10 07:32:14 -07:00
provider_codegen.py feat: use SecretStr for inference provider auth credentials (#3724) 2025-10-10 07:32:50 -07:00
run-ui-linter.sh chore: move src/llama_stack/ui to src/llama_stack_ui (#4068) 2025-11-04 15:21:49 -08:00
unit-tests.sh chore(package): migrate to src/ layout (#3920) 2025-10-27 12:02:21 -07:00
uv-run-with-index.sh fix(ci): use test.pypi as extra index for RC dependencies (#4009) 2025-10-31 12:55:43 -07:00