chore(test): fix flaky telemetry tests (#2815)
Some checks failed
Installer CI / lint (push) Failing after 2s
Installer CI / smoke-test (push) Has been skipped
Integration Tests / discover-tests (push) Successful in 3s
Coverage Badge / unit-tests (push) Failing after 6s
Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 6s
Python Package Build Test / build (3.12) (push) Failing after 3s
Vector IO Integration Tests / test-matrix (3.12, inline::faiss) (push) Failing after 11s
Test Llama Stack Build / build-ubi9-container-distribution (push) Failing after 5s
Vector IO Integration Tests / test-matrix (3.13, inline::faiss) (push) Failing after 10s
Vector IO Integration Tests / test-matrix (3.12, remote::pgvector) (push) Failing after 9s
Unit Tests / unit-tests (3.12) (push) Failing after 6s
SqlStore Integration Tests / test-postgres (3.12) (push) Failing after 15s
Test Llama Stack Build / generate-matrix (push) Successful in 11s
Vector IO Integration Tests / test-matrix (3.12, remote::chromadb) (push) Failing after 16s
Test Llama Stack Build / build-single-provider (push) Failing after 12s
Update ReadTheDocs / update-readthedocs (push) Failing after 9s
Integration Tests / test-matrix (push) Failing after 9s
Vector IO Integration Tests / test-matrix (3.12, inline::sqlite-vec) (push) Failing after 15s
Test External Providers / test-external-providers (venv) (push) Failing after 11s
Vector IO Integration Tests / test-matrix (3.13, remote::pgvector) (push) Failing after 8s
SqlStore Integration Tests / test-postgres (3.13) (push) Failing after 22s
Vector IO Integration Tests / test-matrix (3.12, inline::milvus) (push) Failing after 16s
Vector IO Integration Tests / test-matrix (3.13, inline::sqlite-vec) (push) Failing after 13s
Vector IO Integration Tests / test-matrix (3.13, remote::chromadb) (push) Failing after 16s
Test Llama Stack Build / build-custom-container-distribution (push) Failing after 13s
Test Llama Stack Build / build (push) Failing after 3s
Python Package Build Test / build (3.13) (push) Failing after 48s
Vector IO Integration Tests / test-matrix (3.13, inline::milvus) (push) Failing after 55s
Unit Tests / unit-tests (3.13) (push) Failing after 52s
Pre-commit / pre-commit (push) Successful in 1m42s

# What does this PR do?
<!-- Provide a short summary of what this PR does and why. Link to
relevant issues if applicable. -->
This PR fixes flaky telemetry tests

<!-- If resolving an issue, uncomment and update the line below -->
<!-- Closes #[issue-number] -->
See https://github.com/meta-llama/llama-stack/pull/2814
## Test Plan
<!-- Describe the tests you ran to verify your changes with result
summaries. *Provide clear instructions so the plan can be easily
re-executed.* -->

Signed-off-by: Mustafa Elbehery <melbeher@redhat.com>
This commit is contained in:
Mustafa Elbehery 2025-07-22 21:30:14 +02:00 committed by GitHub
parent c1a63fcd87
commit 9736f096f6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 9 additions and 4 deletions

View file

@ -47,10 +47,12 @@ def setup_telemetry_data(llama_stack_client, text_model_id):
if len(traces) < 4:
pytest.fail(f"Failed to create sufficient telemetry data after 30s. Got {len(traces)} traces.")
# Wait for 5 seconds to ensure traces has completed logging
time.sleep(5)
yield
@pytest.mark.skip(reason="Skipping telemetry tests for now")
def test_query_traces_basic(llama_stack_client):
"""Test basic trace querying functionality with proper data validation."""
all_traces = llama_stack_client.telemetry.query_traces(limit=5)
@ -106,7 +108,6 @@ def test_query_traces_basic(llama_stack_client):
assert hasattr(trace, "root_span_id") and trace.root_span_id, "Each trace should have non-empty root_span_id"
@pytest.mark.skip(reason="Skipping telemetry tests for now")
def test_query_spans_basic(llama_stack_client):
"""Test basic span querying functionality with proper validation."""
spans = llama_stack_client.telemetry.query_spans(attribute_filters=[], attributes_to_return=[])
@ -155,7 +156,6 @@ def test_query_spans_basic(llama_stack_client):
assert hasattr(span, attr) and getattr(span, attr), f"All spans should have non-empty {attr}"
@pytest.mark.skip(reason="Skipping telemetry tests for now")
def test_telemetry_pagination(llama_stack_client):
"""Test pagination in telemetry queries."""
# Get total count of traces