17 commits

Author	SHA1	Message	Date
Xi Yan	cccd5be090	move eval_task_config to client	2024-10-15 10:14:35 -07:00
Xi Yan	9cc0a54f0b	rag correctness scorer w/ custom dataset	2024-10-15 00:42:03 -07:00
Xi Yan	ec6c63ba57	dataset accept file uploads	2024-10-14 23:36:15 -07:00
Xi Yan	3c29108b6e	input query optional input for braintrust scorer	2024-10-14 21:17:16 -07:00
Xi Yan	7b5895003a	braintrust scorer	2024-10-14 21:09:59 -07:00
Xi Yan	fcb8dea1ef	scorer only api	2024-10-14 17:46:29 -07:00
Xi Yan	a22c31b8a4	processor registry	2024-10-14 16:25:06 -07:00
Xi Yan	95fd53d292	registry refactor	2024-10-14 16:09:55 -07:00
Xi Yan	c50686b6fe	scorer registry	2024-10-14 15:41:31 -07:00
Xi Yan	f046899a1c	datasets api	2024-10-14 13:16:39 -07:00
Xi Yan	18fe966e96	registry refactor	2024-10-14 00:12:46 -07:00
Xi Yan	78cb88c3c4	RunEvalTask / InferenceGenerator	2024-10-13 23:48:15 -07:00
Xi Yan	8890de7322	cleanup original BaseTask	2024-10-13 23:30:21 -07:00
Xi Yan	a25aff290e	generator + scorer Api for MMLU	2024-10-13 23:27:02 -07:00
Xi Yan	ad18dc94ac	add data structure to tasks	2024-10-10 21:33:13 -07:00
Xi Yan	9816c9aae6	wip add datatypes	2024-10-10 19:56:19 -07:00
Xi Yan	31c046dcdf	evals new rebase	2024-10-10 11:35:26 -07:00