phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-10-11 21:48:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat(tests): auto-merge all model list responses and unify recordings

Browse source
This commit is contained in:
Ashwin Bharambe 2025-09-03 10:27:44 -07:00
parent faf891b40c
commit 34f5da6cdc
121 changed files with 17457 additions and 16126 deletions
Download patch file Download diff file Expand all files Collapse all files

19
tests/integration/README.md
View file

@ -98,29 +98,25 @@ pytest -s -v tests/integration/vector_io/ \
The testing system supports three modes controlled by environment variables:
### LIVE Mode (Default)
Tests make real API calls:
### REPLAY Mode (Default)
Uses cached responses instead of making API calls:
```bash
LLAMA_STACK_TEST_INFERENCE_MODE=live pytest tests/integration/
pytest tests/integration/
```
### RECORD Mode
Captures API interactions for later replay:
```bash
LLAMA_STACK_TEST_INFERENCE_MODE=record \
LLAMA_STACK_TEST_RECORDING_DIR=tests/integration/recordings \
pytest tests/integration/inference/test_new_feature.py
```
### REPLAY Mode
Uses cached responses instead of making API calls:
### LIVE Mode
Tests make real API calls (but not recorded):
```bash
LLAMA_STACK_TEST_INFERENCE_MODE=replay \
LLAMA_STACK_TEST_RECORDING_DIR=tests/integration/recordings \
pytest tests/integration/
LLAMA_STACK_TEST_INFERENCE_MODE=live pytest tests/integration/
```
Note that right now you must specify the recording directory. This is because different tests use different recording directories and we don't (yet) have a fool-proof way to map a test to a recording directory. We are working on this.
By default, the recording directory is `tests/integration/recordings`. You can override this by setting the `LLAMA_STACK_TEST_RECORDING_DIR` environment variable.
## Managing Recordings
@ -146,7 +142,6 @@ See the [main testing guide](../README.md#remote-re-recording-recommended) for f
```bash
# Re-record specific tests
LLAMA_STACK_TEST_INFERENCE_MODE=record \
LLAMA_STACK_TEST_RECORDING_DIR=tests/integration/recordings \
pytest -s -v --stack-config=server:starter tests/integration/inference/test_modified.py
```