mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-12-31 17:40:01 +00:00
b50bffd15c
## Summary
- Clean up datasets API to work with uri & files.
- Example client usage
```python
client.datasets.create(
uri="file://mydata.jsonl",
schema="jsonl_messages",
# optional
dataset_id="my_first_train_data"
)
# ecosystem enablement ( get huggingface dataset easily )
client.datasets.create(
uri="huggingface://tatsu-lab/alpaca",
uri_params={"split": "train"},
schema="jsonl_messages",
metadata={"label": "v/finetuning/test"},
)
# [ in the future with stronger story for authentication ]
client.datasets.create(
uri="s3://bucket/mydata.jsonl", <-- signed s3 url
uri_parms={...},
schema="jsonl_messages",
)
```
## Test Plan:
- Only API spec updates, implementation will be updated in follow PRs.
|
||
|---|---|---|
| .. | ||
| agents | ||
| batch_inference | ||
| benchmarks | ||
| common | ||
| datasetio | ||
| datasets | ||
| eval | ||
| files | ||
| inference | ||
| inspect | ||
| models | ||
| post_training | ||
| safety | ||
| scoring | ||
| scoring_functions | ||
| shields | ||
| synthetic_data_generation | ||
| telemetry | ||
| tools | ||
| vector_dbs | ||
| vector_io | ||
| __init__.py | ||
| datatypes.py | ||
| resource.py | ||
| version.py | ||