forked from phoenix-oss/llama-stack-mirror
docs: revamp testing documentation (#2155)
# What does this PR do? reduces duplication and centralizes information to be easier to find for contributors Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
This commit is contained in:
parent
62476a5373
commit
e0d10dd0b1
6 changed files with 36 additions and 27 deletions
9
tests/README.md
Normal file
9
tests/README.md
Normal file
|
@ -0,0 +1,9 @@
|
|||
# Llama Stack Tests
|
||||
|
||||
Llama Stack has multiple layers of testing done to ensure continuous functionality and prevent regressions to the codebase.
|
||||
|
||||
| Testing Type | Details |
|
||||
|--------------|---------|
|
||||
| Unit | [unit/README.md](unit/README.md) |
|
||||
| Integration | [integration/README.md](integration/README.md) |
|
||||
| Verification | [verifications/README.md](verifications/README.md) |
|
|
@ -11,7 +11,7 @@ pytest --help
|
|||
Here are the most important options:
|
||||
- `--stack-config`: specify the stack config to use. You have three ways to point to a stack:
|
||||
- a URL which points to a Llama Stack distribution server
|
||||
- a template (e.g., `fireworks`, `together`) or a path to a run.yaml file
|
||||
- a template (e.g., `fireworks`, `together`) or a path to a `run.yaml` file
|
||||
- a comma-separated list of api=provider pairs, e.g. `inference=fireworks,safety=llama-guard,agents=meta-reference`. This is most useful for testing a single API surface.
|
||||
- `--env`: set environment variables, e.g. --env KEY=value. this is a utility option to set environment variables required by various providers.
|
||||
|
||||
|
|
21
tests/unit/README.md
Normal file
21
tests/unit/README.md
Normal file
|
@ -0,0 +1,21 @@
|
|||
# Llama Stack Unit Tests
|
||||
|
||||
You can run the unit tests by running:
|
||||
|
||||
```bash
|
||||
source .venv/bin/activate
|
||||
./scripts/unit-tests.sh [PYTEST_ARGS]
|
||||
```
|
||||
|
||||
Any additional arguments are passed to pytest. For example, you can specify a test directory, a specific test file, or any pytest flags (e.g., -vvv for verbosity). If no test directory is specified, it defaults to "tests/unit", e.g:
|
||||
|
||||
```bash
|
||||
./scripts/unit-tests.sh tests/unit/registry/test_registry.py -vvv
|
||||
```
|
||||
|
||||
If you'd like to run for a non-default version of Python (currently 3.10), pass `PYTHON_VERSION` variable as follows:
|
||||
|
||||
```
|
||||
source .venv/bin/activate
|
||||
PYTHON_VERSION=3.13 ./scripts/unit-tests.sh
|
||||
```
|
|
@ -4,7 +4,7 @@ Llama Stack Verifications provide standardized test suites to ensure API compati
|
|||
|
||||
## Overview
|
||||
|
||||
This framework allows you to run the same set of verification tests against different LLM providers' OpenAI-compatible endpoints (Fireworks, Together, Groq, Cerebras, etc., and OpenAI itself) to ensure they meet the expected behavior and interface standards.
|
||||
This framework allows you to run the same set of verification tests against different LLM providers' OpenAI-compatible endpoints (Fireworks, Together, Groq, Cerebras, etc., and OpenAI itself) to ensure they meet the expected behavior and interface standards.
|
||||
|
||||
## Features
|
||||
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue