5 commits

Author	SHA1	Message	Date
Dinesh Yeduguru	b95cb5308f	migrate dataset to resource (#420) ... * migrate dataset to resource * remove auto discovery * remove listing of providers's datasets * fix after rebase --------- Co-authored-by: Dinesh Yeduguru <dineshyv@fb.com>	2024-11-11 17:14:41 -08:00
Dinesh Yeduguru	663883cc29	persist registered objects with distribution (#354) ... * persist registered objects with distribution * linter fixes * comment * use annotate and field discriminator * workign tests * donot use global state * precommit failures fixed * add back Any * fix imports * remove unnecessary changes in ollama * precommit failures fixed * make kvstore configurable for dist and rename registry * add comment about registry list return * fix linter errors * use registry to hydrate * remove debug print * linter fixes * remove kvstore.db * rename distribution_registry_store --------- Co-authored-by: Dinesh Yeduguru <dineshyv@fb.com>	2024-11-04 17:25:06 -08:00
Xi Yan	cb84034567	[Evals API][3/n] scoring_functions / scoring meta-reference implementations (#296) ... * wip * dataset validation * test_scoring * cleanup * clean up test * comments * error checking * dataset client * test client: * datasetio client * clean up * basic scoring function works * scorer wip * equality scorer * score batch impl * score batch * update scoring test * refactor * validate scorer input * address comments * add all rows scores to ScoringResult * bugfix * scoring function def rename	2024-10-24 14:52:30 -07:00
Xi Yan	821810657f	[Evals API][2/n] datasets / datasetio meta-reference implementation (#288) ... * skeleton dataset / datasetio * dataset datasetio * config * address comments * delete dataset_utils * address comments * naming fix	2024-10-22 16:12:16 -07:00
Xi Yan	e45f121c77	[Evals API] [1/n] Initial API (#287) ... * type system api * datasets api * fix * datasetio api * kill reward scoring * scoring functions + evals * move jobs, fix errors	2024-10-22 09:31:19 -07:00