llama-stack/llama_stack
Xi Yan 4253cfcd7f
local persistent for hf dataset provider (#451)
# What does this PR do?

- local persistence for HF dataset provider
- follow https://github.com/meta-llama/llama-stack/pull/375

## Test Plan

**e2e**
1. fresh llama stack run w/ yaml
2. kill server
3. restart llama stack run w/ yaml

```yaml
datasets:
  - dataset_id: mmlu
    provider_id: huggingface-0
    url:
      uri: https://huggingface.co/datasets/llamastack/evals
    metadata:
      path: llamastack/evals
      name: evals__mmlu__details
      split: train
    dataset_schema:
      input_query:
        type: string
      expected_answer:
        type: string
```
<img width="686" alt="image"
src="https://github.com/user-attachments/assets/d7737931-6a7d-400a-a17d-fef6cbd97eea">


## Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the
other checks if that's the case).
- [ ] Ran pre-commit to handle lint / formatting issues.
- [ ] Read the [contributor
guideline](https://github.com/meta-llama/llama-stack/blob/main/CONTRIBUTING.md),
      Pull Request section?
- [ ] Updated relevant documentation.
- [ ] Wrote necessary unit or integration tests.
2024-11-14 00:08:37 -05:00
..
apis change schema -> dataset_schema for register_dataset api (#443) 2024-11-13 11:17:46 -05:00
cli Rename all inline providers with an inline:: prefix (#423) 2024-11-11 22:19:16 -08:00
distribution make distribution registry thread safe and other fixes (#449) 2024-11-13 15:12:34 -08:00
providers local persistent for hf dataset provider (#451) 2024-11-14 00:08:37 -05:00
scripts Add a test for CLI, but not fully done so disabled 2024-09-19 13:27:07 -07:00
templates Update provider types and prefix with inline:: 2024-11-12 12:54:44 -08:00
__init__.py API Updates (#73) 2024-09-17 19:51:35 -07:00