mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-06-28 02:53:30 +00:00
abdf7cddf3
* wip * dataset validation * test_scoring * cleanup * clean up test * comments * error checking * dataset client * test client: * datasetio client * clean up * basic scoring function works * scorer wip * equality scorer * score batch impl * score batch * update scoring test * refactor * validate scorer input * address comments * evals with generation * add all rows scores to ScoringResult * minor typing * bugfix * scoring function def rename * rebase name * refactor * address comments * Update iOS inference instructions for new quantization * Small updates to quantization config * Fix score threshold in faiss * Bump version to 0.0.45 * Handle both ipv6 and ipv4 interfaces together * update manifest for build templates * Update getting_started.md * chatcompletion & completion input type validation * inclusion->subsetof * error checking * scoring_function -> scoring_fn rename, scorer -> scoring_fn rename * address comments * [Evals API][5/n] fixes to generate openapi spec (#323) * generate openapi * typing comment, dataset -> dataset_id * remove custom type * sample eval run.yaml --------- Co-authored-by: Dalton Flanagan <6599399+dltn@users.noreply.github.com> Co-authored-by: Ashwin Bharambe <ashwin.bharambe@gmail.com>
684 B
684 B
| 1 | input_query | generated_answer | expected_answer | chat_completion_input |
|---|---|---|---|---|
| 2 | What is the capital of France? | London | Paris | [{'role': 'user', 'content': 'What is the capital of France?'}] |
| 3 | Who is the CEO of Meta? | Mark Zuckerberg | Mark Zuckerberg | [{'role': 'user', 'content': 'Who is the CEO of Meta?'}] |
| 4 | What is the largest planet in our solar system? | Jupiter | Jupiter | [{'role': 'user', 'content': 'What is the largest planet in our solar system?'}] |
| 5 | What is the smallest country in the world? | China | Vatican City | [{'role': 'user', 'content': 'What is the smallest country in the world?'}] |
| 6 | What is the currency of Japan? | Yen | Yen | [{'role': 'user', 'content': 'What is the currency of Japan?'}] |