Commit graph

  • 4f3b009980 undo some unintentional changes, clean up some stuff Ashwin Bharambe 2024-11-12 19:47:46 -08:00
  • 0121114a5d remote tests are functional! Ashwin Bharambe 2024-11-12 19:17:58 -08:00
  • 8b7be87bec More work towards making remote stacks usable from tests Ashwin Bharambe 2024-11-12 17:09:31 -08:00
  • 8645f8bc9e Refactor safety shield fixtures Ashwin Bharambe 2024-11-12 15:58:58 -08:00
  • 743da9690b Kill the notion of a "remote" / "passthrough" provider Ashwin Bharambe 2024-11-12 15:30:59 -08:00
  • 59a65e34d3
    Update new_api_provider.md Xi Yan 2024-11-13 00:02:13 -05:00
  • fdff24e77a
    Inference to use provider resource id to register and validate (#428) Dinesh Yeduguru 2024-11-12 20:02:00 -08:00
  • 1bb01f9346 remove model lookup class Dinesh Yeduguru 2024-11-12 20:00:48 -08:00
  • 606df220f5 fix evals and scoring Dinesh Yeduguru 2024-11-12 18:14:58 -08:00
  • 55d66ca918 run openapi gen Dinesh Yeduguru 2024-11-12 15:47:41 -08:00
  • e51107e019 Fix compose.yaml Ashwin Bharambe 2024-11-12 15:43:30 -08:00
  • 919d421bcf fixes after rebase Dinesh Yeduguru 2024-11-12 15:37:07 -08:00
  • 948f6ece6e fixes for all providers Dinesh Yeduguru 2024-11-12 14:25:28 -08:00
  • d5874735ea bedrock Dinesh Yeduguru 2024-11-12 14:08:47 -08:00
  • 92ee627e89 vllm Dinesh Yeduguru 2024-11-12 13:59:46 -08:00
  • 71219b4937 ollama Dinesh Yeduguru 2024-11-12 13:23:02 -08:00
  • 5b2282afd4 ollama and databricks Dinesh Yeduguru 2024-11-12 13:17:27 -08:00
  • 8de4cee373 working fireworks and together Dinesh Yeduguru 2024-11-12 13:07:35 -08:00
  • 25d8ab0e14 fix bedrock Dinesh Yeduguru 2024-11-12 10:17:51 -08:00
  • d69f4f8635 fix model provider validation and inference params Dinesh Yeduguru 2024-11-12 10:13:43 -08:00
  • 95b7f57d92 use provider resource id to validate for models Dinesh Yeduguru 2024-11-12 08:21:37 -08:00
  • e4f14eafe2 Use GPUs 0 and 1 Ashwin Bharambe 2024-11-12 14:21:22 -08:00
  • 1245a625ce Update vllm compose and run YAMLs Ashwin Bharambe 2024-11-12 12:46:32 -08:00
  • afe4a53ae8 Check vLLM registration Ashwin Bharambe 2024-11-12 13:14:36 -08:00
  • 1aeac7b9f7 Change order of building the Docker Ashwin Bharambe 2024-11-12 13:09:04 -08:00
  • 998419ffb2 use image tag actually! Ashwin Bharambe 2024-11-12 12:57:08 -08:00
  • 2c294346ae Update provider types and prefix with inline:: Ashwin Bharambe 2024-11-12 12:54:44 -08:00
  • 896b304e62 Use tags for docker images instead of changing image name Ashwin Bharambe 2024-11-12 12:42:11 -08:00
  • 983d6ce2df
    Remove the "ShieldType" concept (#430) Ashwin Bharambe 2024-11-12 12:37:24 -08:00
  • f246b5d98f lint Ashwin Bharambe 2024-11-12 12:35:16 -08:00
  • b94e6c0bd4 Fix Registry so it scopes keys by object types Ashwin Bharambe 2024-11-12 12:25:43 -08:00
  • b1c3a95485 Kill the notion of shield_type Ashwin Bharambe 2024-11-12 11:41:23 -08:00
  • 09269e2a44
    Enable sane naming of registered objects with defaults (#429) Ashwin Bharambe 2024-11-12 11:18:05 -08:00
  • 78d633123a regenerate OpenAPI, some small changes Ashwin Bharambe 2024-11-12 11:17:13 -08:00
  • 04d8660247 fix scoring fixture Ashwin Bharambe 2024-11-12 11:14:38 -08:00
  • d0ad198be9 fix Ashwin Bharambe 2024-11-12 11:11:01 -08:00
  • 593ba80162 fix MemoryBankInput Ashwin Bharambe 2024-11-12 10:28:41 -08:00
  • 48a6e27de9 Enable sane naming of registered objects with defaults Ashwin Bharambe 2024-11-12 10:17:34 -08:00
  • 9e925f43e5 print resources when booting up Ashwin Bharambe 2024-11-11 23:10:12 -08:00
  • 8363764093 Allow specifying resources in StackRunConfig Ashwin Bharambe 2024-11-11 22:08:51 -08:00
  • d9d271a684
    Allow specifying resources in StackRunConfig (#425) Ashwin Bharambe 2024-11-12 10:58:49 -08:00
  • ba8afa9235 print resources when booting up Ashwin Bharambe 2024-11-11 23:10:12 -08:00
  • abe1cc6303 Fix agents tests Ashwin Bharambe 2024-11-11 22:31:09 -08:00
  • 38257a9cbe Allow specifying resources in StackRunConfig Ashwin Bharambe 2024-11-11 22:08:51 -08:00
  • 8035fa1869 versioned persistence key prefixes Dinesh Yeduguru 2024-11-12 10:30:39 -08:00
  • cb77426fb5
    fix fireworks (#427) Xi Yan 2024-11-12 12:15:55 -05:00
  • dc0fa5afa9 fix fireworks Xi Yan 2024-11-12 12:09:23 -05:00
  • ec4fcad5ca
    fix eval task registration (#426) Xi Yan 2024-11-12 11:51:34 -05:00
  • acd0b0f3f6 fix eval tests Xi Yan 2024-11-12 11:49:48 -05:00
  • 6286970ee1 fix eval tasks Xi Yan 2024-11-12 11:43:10 -05:00
  • cb77712510 fix eval tasks Xi Yan 2024-11-12 11:42:49 -05:00
  • 84c6fbbd93
    fix tests after registration migration & rename meta-reference -> basic / llm_as_judge provider (#424) Xi Yan 2024-11-12 10:35:44 -05:00
  • ba1d50c67b naming fix Xi Yan 2024-11-12 10:29:25 -05:00
  • a15a4a4be2 rebase Xi Yan 2024-11-12 10:03:03 -05:00
  • 86fdad3be3
    Merge branch 'main' into restructure_scoring Xi Yan 2024-11-12 09:59:35 -05:00
  • 3d7561e55c
    Rename all inline providers with an inline:: prefix (#423) Ashwin Bharambe 2024-11-11 22:19:16 -08:00
  • f4426f6a43 Fix bug in llama stack build; SERVER_DEPENDENCIES were dropped Ashwin Bharambe 2024-11-11 20:12:13 -08:00
  • fd424e7900 util Xi Yan 2024-11-11 23:02:56 -05:00
  • 87dc116782 rename llm_as_judge, fix test Xi Yan 2024-11-11 22:57:51 -05:00
  • 506b99242a Allow specifying TEST / PYPI VERSION for docker name Ashwin Bharambe 2024-11-11 19:55:23 -08:00
  • e20dc8d133 impl rename Xi Yan 2024-11-11 22:41:54 -05:00
  • ecf1932424 config rename Xi Yan 2024-11-11 22:41:15 -05:00
  • 8dee5faf0d rename meta-reference -> basic Xi Yan 2024-11-11 22:36:43 -05:00
  • 36da9a600e add explicit platform Ashwin Bharambe 2024-11-11 19:30:15 -08:00
  • 218803b7c8 add pypi version to docker tag Ashwin Bharambe 2024-11-11 19:14:06 -08:00
  • 47e7c2dc15 Fix openapi generator and regenerator OpenAPI types Ashwin Bharambe 2024-11-11 18:44:38 -08:00
  • 343458479d Make sure TEST_PYPI_VERSION is used in docker builds Ashwin Bharambe 2024-11-11 18:40:13 -08:00
  • eea517deb5 Rename all inline providers with an inline:: prefix Ashwin Bharambe 2024-11-11 18:04:34 -08:00
  • 285cd26fb2 Replace colon in path so it doesn't cause issue on Windows Ashwin Bharambe 2024-11-11 17:30:36 -08:00
  • 0a3b3d5fb6
    migrate scoring fns to resource (#422) Dinesh Yeduguru 2024-11-11 17:28:48 -08:00
  • 5369422a4d remove print Dinesh Yeduguru 2024-11-11 15:04:06 -08:00
  • 9718989ee1 fix after rebase Dinesh Yeduguru 2024-11-11 17:27:22 -08:00
  • 3802edfc50
    migrate evals to resource (#421) Dinesh Yeduguru 2024-11-11 17:24:03 -08:00
  • fbf988f7d9 linter fix Dinesh Yeduguru 2024-11-11 17:23:05 -08:00
  • cab15c9cfa fix after rebase Dinesh Yeduguru 2024-11-11 17:20:19 -08:00
  • 48dcea18f2 change the order of params in register Dinesh Yeduguru 2024-11-11 12:48:27 -08:00
  • 749d5f6b5e remove listing of providers's evals Dinesh Yeduguru 2024-11-11 12:46:13 -08:00
  • 63b5eb929f migrate evals to resource Dinesh Yeduguru 2024-11-11 12:38:22 -08:00
  • b95cb5308f
    migrate dataset to resource (#420) Dinesh Yeduguru 2024-11-11 17:14:41 -08:00
  • 8c97e2c1c8 fix after rebase Dinesh Yeduguru 2024-11-11 17:13:54 -08:00
  • 600f23703d remove listing of providers's datasets Dinesh Yeduguru 2024-11-11 12:43:51 -08:00
  • f5ea666a6e remove auto discovery Dinesh Yeduguru 2024-11-11 12:40:42 -08:00
  • f8e39ae873 migrate dataset to resource Dinesh Yeduguru 2024-11-11 12:15:41 -08:00
  • 38cce97597
    migrate memory banks to Resource and new registration (#411) Dinesh Yeduguru 2024-11-11 17:10:44 -08:00
  • 0789233dca remove base MemoryBank inheritence Dinesh Yeduguru 2024-11-11 17:09:41 -08:00
  • 8436d386d3 use annotated union to parse Dinesh Yeduguru 2024-11-11 16:22:11 -08:00
  • cf416cb6b8 client fix Dinesh Yeduguru 2024-11-11 15:41:46 -08:00
  • 6b9850e11b run openapi gen Xi Yan 2024-11-11 18:12:24 -05:00
  • 949b82fc6d update client Dinesh Yeduguru 2024-11-11 11:53:51 -08:00
  • 4369c2d0b6 change register signature to make params required Dinesh Yeduguru 2024-11-11 11:49:07 -08:00
  • 4206f07ada remove auto discovery Dinesh Yeduguru 2024-11-11 11:13:33 -08:00
  • f1a2996812 pgvector fix v2 Dinesh Yeduguru 2024-11-11 10:50:27 -08:00
  • b2b49fbdb9 pgvector fix Dinesh Yeduguru 2024-11-11 10:28:23 -08:00
  • 5cdcdbe074 fix tests Dinesh Yeduguru 2024-11-08 17:00:15 -08:00
  • 24b914f0fe address feedback Dinesh Yeduguru 2024-11-08 16:51:05 -08:00
  • bd43d37f4e address feedback Dinesh Yeduguru 2024-11-08 16:45:41 -08:00
  • c82f13bf9e migrate memory banks to Resource and new registration Dinesh Yeduguru 2024-11-08 15:45:26 -08:00
  • b4416b72fd
    Folder restructure for evals/datasets/scoring (#419) Xi Yan 2024-11-11 17:35:40 -05:00
  • 5433873f12 import Xi Yan 2024-11-11 17:02:20 -05:00
  • 258e01ec67 scoring fn braintrust fixture Xi Yan 2024-11-11 16:42:16 -05:00