phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-08-21 09:23:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
llama-stack-mirror/tests/integration/recordings/responses
History
Exact
Ashwin Bharambe eb07a0f86a
Some checks failed
Test External Providers Installed via Module / test-external-providers-from-module (venv) (push) Has been skipped
Details
SqlStore Integration Tests / test-postgres (3.12) (push) Failing after 21s
Details
Test Llama Stack Build / build-single-provider (push) Failing after 23s
Details
SqlStore Integration Tests / test-postgres (3.13) (push) Failing after 28s
Details
Test Llama Stack Build / generate-matrix (push) Successful in 25s
Details
Python Package Build Test / build (3.13) (push) Failing after 25s
Details
Test Llama Stack Build / build-custom-container-distribution (push) Failing after 34s
Details
Integration Tests (Replay) / Integration Tests (, , , client=, vision=) (push) Failing after 37s
Details
Test External API and Providers / test-external (venv) (push) Failing after 33s
Details
Unit Tests / unit-tests (3.13) (push) Failing after 33s
Details
Test Llama Stack Build / build-ubi9-container-distribution (push) Failing after 38s
Details
Python Package Build Test / build (3.12) (push) Failing after 1m0s
Details
Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 1m4s
Details
Unit Tests / unit-tests (3.12) (push) Failing after 59s
Details
Test Llama Stack Build / build (push) Failing after 50s
Details
Vector IO Integration Tests / test-matrix (push) Failing after 1m48s
Details
UI Tests / ui-tests (22) (push) Successful in 2m12s
Details
Pre-commit / pre-commit (push) Successful in 2m41s
Details
fix(ci, tests): ensure uv environments in CI are kosher, record tests (#3193) 
I started this PR trying to unbreak a newly broken test
`test_agent_name`. This test was broken all along but did not show up
because during testing we were pulling the "non-updated" llama stack
client. See this comment:
https://github.com/llamastack/llama-stack/pull/3119#discussion_r2270988205

While fixing this, I encountered a large amount of badness in our CI
workflow definitions.

- We weren't passing `LLAMA_STACK_DIR` or `LLAMA_STACK_CLIENT_DIR`
overrides to `llama stack build` at all in some cases.
- Even when we did, we used `uv run` liberally. The first thing `uv run`
does is "syncs" the project environment. This means, it is going to undo
any mutations we might have done ourselves. But we make many mutations
in our CI runners to these environments. The most important of which is
why `llama stack build` where we install distro dependencies. As a
result, when you tried to run the integration tests, you would see old,
strange versions.


## Test Plan

Re-record using:

```
sh scripts/integration-tests.sh --stack-config ci-tests \
  --provider ollama --test-pattern test_agent_name --inference-mode record
```

Then re-run with `--inference-mode replay`. But: 

Eventually, this test turned out to be quite flaky for telemetry
reasons. I haven't investigated it for now and just disabled it sadly
since we have a release to push out.
2025-08-18 17:02:24 -07:00
..
0b3f2e4754ff.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
0b27fd737699.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
00ba04f74a96.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
0e8f2b001dd9.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
0ff78129bb3a.json feat(ci): use replay mode, setup ollama if specific label exists on PR (#2955) 2025-07-29 16:50:26 -07:00
1adfaa0e062e.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
1b92be674e2a.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
1b8394f90636.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
1b45391880c6.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
1f48f4b2ae33.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
2afe3b38ca01.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
2d187a11704c.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
3c0bf9ba81b2.json feat: add batches API with OpenAI compatibility (with inference replay) (#3162) 2025-08-15 15:34:15 -07:00
3c3f13cb7794.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
3ca695048bee.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
4a3a4447b16b.json fix(ci, tests): ensure uv environments in CI are kosher, record tests (#3193) 2025-08-18 17:02:24 -07:00
5f5d16afadb4.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
6cc063bbd7d3.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
6d35c91287e2.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
6f96090aa955.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
6fbea1abca7c.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
6fe1d4fedf12.json fix(tests): record missing tests for test_responses_store (#3163) 2025-08-15 03:52:45 +00:00
7b4815aba6c5.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
7e6806cba34a.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
8bba71367e87.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
9b812cbcb88d.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
9c4bc9c3e7ac.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
9c007f300365.json fix: telemetry fixes (inference and core telemetry) (#2733) 2025-08-06 13:37:40 -07:00
9c28ec9ac338.json feat(ci): use replay mode, setup ollama if specific label exists on PR (#2955) 2025-07-29 16:50:26 -07:00
9c140a29ae09.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
9e7a83d3d596.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
9fadf5a3d68f.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
10eea8c15ddc.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
011f70e24ce4.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
35db283fef1d.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
40f524d1934a.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
43e106de6736.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
44a1d9de0602.json feat: add batches API with OpenAI compatibility (with inference replay) (#3162) 2025-08-15 15:34:15 -07:00
44fb9cf5875f.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
48d2fb183a2a.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
56ac6a7c6df0.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
61be36ad8ccd.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
70adef2c30c4.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
75d0dd9d0fa3.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
76b89a84cd6f.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
80e4404d8987.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
80f09f27dd61.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
81a91f79c51d.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
83c2ffb72daa.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
84cab42e1f5c.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
90fec951fdb9.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
97d3812bfccb.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
97e259c0d3e5.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
173ecb3aab28.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
177ba8517262.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
198ef7208389.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
211b1562d4e6.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
325a72db5755.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
382c2f22274c.json fix: telemetry fixes (inference and core telemetry) (#2733) 2025-08-06 13:37:40 -07:00
477f8946bf7d.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
545d86510a80.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
554de3cd986f.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
563b994bb7d1.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
830a1fe14938.json fix: telemetry fixes (inference and core telemetry) (#2733) 2025-08-06 13:37:40 -07:00
836f51dfb3c5.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
840fbb380b73.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
3877ecf1bc62.json chore: move recordings one directory upwards 2025-07-29 12:46:19 -07:00
4014dd44c15f.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
6906a6e71988.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
7354ec181984.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
12913f20f6ac.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
17253d7cc667.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
31407e035752.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
50340cd4d253.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
67198cbad48f.json fix: telemetry fixes (inference and core telemetry) (#2733) 2025-08-06 13:37:40 -07:00
85594a69d74a.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
140187e305dc.json fix: telemetry fixes (inference and core telemetry) (#2733) 2025-08-06 13:37:40 -07:00
174458ad71b2.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
178016edef0e.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
197228e26971.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
561746e1c8de.json feat(ci): make recording workflow simpler, more parameterizable (#3169) 2025-08-15 14:47:20 -07:00
731824c54461.json fix(ci, tests): ensure uv environments in CI are kosher, record tests (#3193) 2025-08-18 17:02:24 -07:00
4096743baf8e.json fix: telemetry fixes (inference and core telemetry) (#2733) 2025-08-06 13:37:40 -07:00
04172112ffbb.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
4597743bcd2a.json feat(ci): use replay mode, setup ollama if specific label exists on PR (#2955) 2025-07-29 16:50:26 -07:00
8295382a8e7c.json fix: telemetry fixes (inference and core telemetry) (#2733) 2025-08-06 13:37:40 -07:00
a0c4df33879f.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
a4c8d19bb1eb.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
a59d0d7c1485.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
a410d4840402.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
a5187d9d5057.json fix: telemetry fixes (inference and core telemetry) (#2733) 2025-08-06 13:37:40 -07:00
a6810c23eda8.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
a97477559b10.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
ae1c22f18ecc.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
ae6835cfe70e.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
afb33182f365.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
b4cda53cd04f.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
b14ff438ca99.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
b44cc7a7afc8.json chore(misc): make tests and starter faster (#3042) 2025-08-05 14:55:05 -07:00
b91f1fb4aedb.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
b24590574a85.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
bbd0637dce16.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
bd356b27a085.json feat(ci): use replay mode, setup ollama if specific label exists on PR (#2955) 2025-07-29 16:50:26 -07:00
c9cba6f3ee38.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
c7582fa7c2c4.json feat(ci): use replay mode, setup ollama if specific label exists on PR (#2955) 2025-07-29 16:50:26 -07:00
c9667519ad7c.json fix: telemetry fixes (inference and core telemetry) (#2733) 2025-08-06 13:37:40 -07:00
cb3df2a1dc22.json fix: telemetry fixes (inference and core telemetry) (#2733) 2025-08-06 13:37:40 -07:00
cd094caaf1c0.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
d0ac68cbde69.json fix(ci, tests): ensure uv environments in CI are kosher, record tests (#3193) 2025-08-18 17:02:24 -07:00
d4c86ac355fb.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
d4f56d7d1996.json fix: telemetry fixes (inference and core telemetry) (#2733) 2025-08-06 13:37:40 -07:00
da531c71e64f.json chore(misc): make tests and starter faster (#3042) 2025-08-05 14:55:05 -07:00
dac7a32e5db9.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
dbc41d2417e1.json chore(misc): make tests and starter faster (#3042) 2025-08-05 14:55:05 -07:00
dd9e7d5913e9.json chore: fix: integration tests failures marked as successful (#3039) 2025-08-04 17:06:28 -07:00
dd226d71f844.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
decfd950646c.json fix(tests): record missing tests for test_responses_store (#3163) 2025-08-15 03:52:45 +00:00
e2c9b07709fe.json fix: telemetry fixes (inference and core telemetry) (#2733) 2025-08-06 13:37:40 -07:00
e9c8a0e4f0e0.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
e08848bfcd28.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
e29300494763.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
e96152610712.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
ed9e9b34008d.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
eee6a163b837.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00
eee47930e3ae.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
ef59cbff54d0.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
ef757a75ed08.json feat(ci): use replay mode, setup ollama if specific label exists on PR (#2955) 2025-07-29 16:50:26 -07:00
f1ea938b0b0d.json chore(misc): make tests and starter faster (#3042) 2025-08-05 14:55:05 -07:00
f3c3afbd9b7e.json feat(ci): use replay mode, setup ollama if specific label exists on PR (#2955) 2025-07-29 16:50:26 -07:00
f70f30f54211.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
f477c2fe1332.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
fcdef245da95.json fix(recording): endpoint resolution (#3013) 2025-08-01 16:23:54 -07:00
fe140befeba4.json feat(ci): add support for running vision inference tests (#2972) 2025-07-31 11:50:42 -07:00