Xi Yan 2b7d70ba86 [Evals API][11/n] huggingface dataset provider + mmlu scoring fn (#392) ... * wip * scoring fn api * eval api * eval task * evaluate api update * pre commit * unwrap context -> config * config field doc * typo * naming fix * separate benchmark / app eval * api name * rename * wip tests * wip * datasetio test * delete unused * fixture * scoring resolve * fix scoring register * scoring test pass * score batch * scoring fix * fix eval * test eval works * huggingface provider * datasetdef files * mmlu scoring fn * test wip * remove type ignore * api refactor * add default task_eval_id for routing * add eval_id for jobs * remove type ignore * huggingface provider * wip huggingface register * only keep 1 run_eval * fix optional * register task required * register task required * delete old tests * fix * mmlu loose * refactor * msg * fix tests * move benchmark task def to file * msg * gen openapi * openapi gen * move dataset to hf llamastack repo * remove todo * refactor * add register model to unit test * rename * register to client * delete preregistered dataset/eval task * comments * huggingface -> remote adapter * openapi gen		2024-11-11 14:49:50 -05:00
..
agentic-system.png	moving rfc->docs	2024-09-18 16:54:24 -07:00
list-templates.png	move openapi from rfcs->docs	2024-09-18 16:09:17 -07:00
llama-stack-spec.html	[Evals API][11/n] huggingface dataset provider + mmlu scoring fn (#392)	2024-11-11 14:49:50 -05:00
llama-stack-spec.yaml	[Evals API][11/n] huggingface dataset provider + mmlu scoring fn (#392)	2024-11-11 14:49:50 -05:00
llama-stack.png	add llama-stack.png	2024-09-26 11:17:46 -07:00
model-lifecycle.png	moving rfc->docs	2024-09-18 16:54:24 -07:00
prompt-format.png	Add files via upload	2024-10-08 15:29:19 -07:00