mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-12-22 20:22:27 +00:00
87ec4243ee
# What does this PR do? - This PR adds the ability s.t. users can evaluate on both retrieval + generation separately & as a whole by passing an AgentConfig to the /eval API - The memory_retrieval context will be stored in the "context" column used for scoring functions that can evaluate the retrieved context. ## Test Plan - E2E Test RAG Agent Notebook: https://gist.github.com/yanxi0830/0377594d29958f9b6f9317ab049fa836 <img width="758" alt="image" src="https://github.com/user-attachments/assets/58ed9db7-f07b-400a-931b-923b0d612902" /> <img width="682" alt="image" src="https://github.com/user-attachments/assets/9ebd7fbd-2a6d-4c93-92fa-a9456fae2378" /> ## Sources Please link relevant resources if necessary. ## Before submitting - [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case). - [ ] Ran pre-commit to handle lint / formatting issues. - [ ] Read the [contributor guideline](https://github.com/meta-llama/llama-stack/blob/main/CONTRIBUTING.md), Pull Request section? - [ ] Updated relevant documentation. - [ ] Wrote necessary unit or integration tests. |
||
|---|---|---|
| .. | ||
| agents | ||
| datasetio | ||
| eval | ||
| inference | ||
| ios/inference | ||
| memory | ||
| post_training/torchtune | ||
| safety | ||
| scoring | ||
| telemetry | ||
| tool_runtime/brave_search | ||
| __init__.py | ||