Xi Yan bcb13c492f test: revamp eval related integration tests (#1433) ... # What does this PR do? - revamp and clean up datasets/scoring/eval integration tests - closes https://github.com/meta-llama/llama-stack/issues/1396 [//]: # (If resolving an issue, uncomment and update the line below) [//]: # (Closes #[issue-number]) ## Test Plan **dataset** ``` LLAMA_STACK_BASE_URL=http://localhost:8321 pytest -v tests/integration/datasetio/ ``` <img width="842" alt="image" src="https://github.com/user-attachments/assets/88fc2b6a-b496-47bf-bc0c-8fea48ba36ff" /> **scoring** ``` LLAMA_STACK_CONFIG=fireworks pytest -v tests/integration/scoring --text-model meta-llama/Llama-3.1-8B-Instruct --judge-model meta-llama/Llama-3.1-8B-Instruct ``` <img width="851" alt="image" src="https://github.com/user-attachments/assets/50f46415-b44c-4c37-a6c3-076f2767adb3" /> **eval** ``` LLAMA_STACK_CONFIG=fireworks pytest -v tests/integration/eval --text-model meta-llama/Llama-3.1-8B-Instruct --judge-model meta-llama/Llama-3.1-8B-Instruct ``` <img width="841" alt="image" src="https://github.com/user-attachments/assets/8eb1c65c-3b39-4d66-8ff4-f471ca783e49" /> [//]: # (## Documentation)		2025-03-06 10:51:35 -08:00
..
bedrock	Fix precommit check after moving to ruff (#927)	2025-02-02 06:46:45 -08:00
common	build: format codebase imports using ruff linter (#1028)	2025-02-13 10:06:21 -08:00
datasetio	build: format codebase imports using ruff linter (#1028)	2025-02-13 10:06:21 -08:00
inference	feat: better using get_default_tool_prompt_format (#1360)	2025-03-03 14:50:06 -08:00
kvstore	refactor(test): unify vector_io tests and make them configurable (#1398)	2025-03-04 13:37:45 -08:00
memory	chore: remove dependency on llama_models completely (#1344)	2025-03-01 12:48:08 -08:00
scoring	test: revamp eval related integration tests (#1433)	2025-03-06 10:51:35 -08:00
telemetry	fix: Agent telemetry inputs/outputs should be structured (#1302)	2025-02-27 23:06:37 -08:00
__init__.py	API Updates (#73)	2024-09-17 19:51:35 -07:00