18 commits

Author	SHA1	Message	Date
Xi Yan	e9a9ecb2dc	comments	2024-11-11 11:07:12 -05:00
Xi Yan	9ff903e63b	delete preregistered dataset/eval task	2024-11-11 11:05:47 -05:00
Xi Yan	75ccc05296	rename	2024-11-11 10:48:47 -05:00
Xi Yan	58c6138df1	move dataset to hf llamastack repo	2024-11-08 11:42:16 -08:00
Xi Yan	989f070bc0	move benchmark task def to file	2024-11-07 21:35:02 -08:00
Xi Yan	edeb6dcf04	mmlu loose	2024-11-07 18:36:41 -08:00
Xi Yan	6ee02ca23b	fix	2024-11-07 18:25:39 -08:00
Xi Yan	33b6d9b7b7	merge	2024-11-07 18:15:13 -08:00
Xi Yan	3c17853d79	register task required	2024-11-07 16:42:44 -08:00
Xi Yan	94a56cc3f3	register task required	2024-11-07 16:41:23 -08:00
Xi Yan	fd581c3d88	only keep 1 run_eval	2024-11-07 16:17:49 -08:00
Xi Yan	d1633dc412	huggingface provider	2024-11-07 15:20:22 -08:00
Xi Yan	f05db9a25c	add eval_id for jobs	2024-11-07 14:30:46 -08:00
Xi Yan	ea80f623fb	add default task_eval_id for routing	2024-11-07 14:19:33 -08:00
Xi Yan	51c20f9c29	api refactor	2024-11-07 13:54:26 -08:00
Xi Yan	3f1ac29d57	test eval works	2024-11-06 21:40:38 -08:00
Xi Yan	413a1b6d8b	fix eval	2024-11-06 21:10:54 -08:00
Ashwin Bharambe	994732e2e0	impls -> inline, adapters -> remote (#381)	2024-11-06 14:54:05 -08:00