phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-06-28 02:53:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

[Evals API][7/n] braintrust scoring provider (#333)

Browse source 
* wip scoring refactor

* llm as judge, move folders

* test full generation + eval

* extract score regex to llm context

* remove prints, cleanup braintrust in this branch

* braintrust skeleton

* datasetio test fix

* braintrust provider

* remove prints

* dependencies

* change json -> class

* json -> class

* remove initialize

* address nits

* check identifier prefix

* braintrust scoring identifier check, rebase

* udpate MANIFEST

* manifest

* remove braintrust scoring_fn

* remove comments

* tests

* imports fix
This commit is contained in:
Xi Yan 2024-10-28 18:59:35 -07:00 • committed by GitHub
parent ae671eaf7a
commit ed833bb758
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
11 changed files with 274 additions and 15 deletions
Download patch file Download diff file Expand all files Collapse all files

3
llama_stack/providers/tests/datasetio/test_datasetio.py
View file

@ -82,7 +82,8 @@ async def register_dataset(
dataset = DatasetDefWithProvider(
identifier=dataset_id,
provider_id=os.environ["PROVIDER_ID"],
provider_id=os.environ.get("DATASETIO_PROVIDER_ID", None)
or os.environ["PROVIDER_ID"],
url=URL(
uri=test_url,
),