Xi Yan
|
6192bf43a4
|
[Evals API][10/n] API updates for EvalTaskDef + new test migration (#379)
* wip
* scoring fn api
* eval api
* eval task
* evaluate api update
* pre commit
* unwrap context -> config
* config field doc
* typo
* naming fix
* separate benchmark / app eval
* api name
* rename
* wip tests
* wip
* datasetio test
* delete unused
* fixture
* scoring resolve
* fix scoring register
* scoring test pass
* score batch
* scoring fix
* fix eval
* test eval works
* remove type ignore
* api refactor
* add default task_eval_id for routing
* add eval_id for jobs
* remove type ignore
* only keep 1 run_eval
* fix optional
* register task required
* register task required
* delete old tests
* delete old tests
* fixture return impl
|
2024-11-07 21:24:12 -08:00 |
|
Ashwin Bharambe
|
694c142b89
|
Add provider deprecation support; change directory structure (#397)
* Add provider deprecation support; change directory structure
* fix a couple dangling imports
* move the meta_reference safety dir also
|
2024-11-07 13:04:53 -08:00 |
|
Xi Yan
|
8fc2d212a2
|
fix safety signature mismatch (#388)
* fix safety sig
* shield_type->identifier
|
2024-11-06 16:30:47 -08:00 |
|
Ashwin Bharambe
|
994732e2e0
|
impls -> inline, adapters -> remote (#381)
|
2024-11-06 14:54:05 -08:00 |
|