Commit graph

11 commits

Author SHA1 Message Date
Xi Yan
cccd5be090 move eval_task_config to client 2024-10-15 10:14:35 -07:00
Xi Yan
fcb8dea1ef scorer only api 2024-10-14 17:46:29 -07:00
Xi Yan
a22c31b8a4 processor registry 2024-10-14 16:25:06 -07:00
Xi Yan
95fd53d292 registry refactor 2024-10-14 16:09:55 -07:00
Xi Yan
c50686b6fe scorer registry 2024-10-14 15:41:31 -07:00
Xi Yan
f046899a1c datasets api 2024-10-14 13:16:39 -07:00
Xi Yan
78cb88c3c4 RunEvalTask / InferenceGenerator 2024-10-13 23:48:15 -07:00
Xi Yan
8890de7322 cleanup original BaseTask 2024-10-13 23:30:21 -07:00
Xi Yan
a25aff290e generator + scorer Api for MMLU 2024-10-13 23:27:02 -07:00
Xi Yan
ad18dc94ac add data structure to tasks 2024-10-10 21:33:13 -07:00
Xi Yan
31c046dcdf evals new rebase 2024-10-10 11:35:26 -07:00