Commit graph

402 commits

Author SHA1 Message Date
Xi Yan
cdfd584a8f Merge branch 'main' into evals_6 2024-10-24 17:29:22 -07:00
Justin Lee
b6d8246b82
added templates and enhanced readme (#307)
Co-authored-by: Justin Lee <justinai@fb.com>
2024-10-24 17:07:06 -07:00
Xi Yan
2372de70f8 error checking 2024-10-24 16:16:34 -07:00
Xi Yan
f6340a47d1 inclusion->subsetof 2024-10-24 16:13:49 -07:00
Xi Yan
29e48cc5c1 chatcompletion & completion input type validation 2024-10-24 16:11:25 -07:00
Dinesh Yeduguru
3e1c3fdb3f
completion() for tgi (#295) 2024-10-24 16:02:41 -07:00
Xi Yan
e468e23249
Merge branch 'main' into evals_6 2024-10-24 14:59:41 -07:00
Xi Yan
997e3003b9 Update getting_started.md 2024-10-24 14:57:10 -07:00
Xi Yan
dc6393c271 update manifest for build templates 2024-10-24 14:57:10 -07:00
Ashwin Bharambe
2bea4bf801 Handle both ipv6 and ipv4 interfaces together 2024-10-24 14:57:10 -07:00
Ashwin Bharambe
72755da634 Bump version to 0.0.45 2024-10-24 14:57:10 -07:00
Ashwin Bharambe
7b1a45ee0f Fix score threshold in faiss 2024-10-24 14:57:10 -07:00
Ashwin Bharambe
86f1efa680 Small updates to quantization config 2024-10-24 14:57:10 -07:00
Dalton Flanagan
1721d91c95 Update iOS inference instructions for new quantization 2024-10-24 14:57:10 -07:00
Xi Yan
cb84034567
[Evals API][3/n] scoring_functions / scoring meta-reference implementations (#296)
* wip

* dataset validation

* test_scoring

* cleanup

* clean up test

* comments

* error checking

* dataset client

* test client:

* datasetio client

* clean up

* basic scoring function works

* scorer wip

* equality scorer

* score batch impl

* score batch

* update scoring test

* refactor

* validate scorer input

* address comments

* add all rows scores to ScoringResult

* bugfix

* scoring function def rename
2024-10-24 14:52:30 -07:00
Xi Yan
d4887fc746 address comments 2024-10-24 14:49:02 -07:00
Xi Yan
e70420a06e
Update getting_started.md 2024-10-24 14:19:35 -07:00
Xi Yan
8615bc9e08 update manifest for build templates 2024-10-24 14:04:13 -07:00
Xi Yan
ba0186f2c8 refactor 2024-10-24 14:00:41 -07:00
Ashwin Bharambe
94728d6983 Handle both ipv6 and ipv4 interfaces together 2024-10-24 13:59:01 -07:00
Xi Yan
3db1b3fbcd rebase name 2024-10-24 13:53:41 -07:00
Xi Yan
97ca72288c Merge branch 'evals_5' into evals_6 2024-10-24 13:53:00 -07:00
Xi Yan
6053b8dd34 scoring function def rename 2024-10-24 13:51:11 -07:00
Xi Yan
689990b48b Merge branch 'evals_5' into evals_6 2024-10-24 13:06:11 -07:00
Xi Yan
42bac85e1f bugfix 2024-10-24 12:16:28 -07:00
Ashwin Bharambe
0538cc297e Bump version to 0.0.45 2024-10-24 12:14:18 -07:00
Ashwin Bharambe
205bcfdd4e Fix score threshold in faiss 2024-10-24 12:11:58 -07:00
Xi Yan
24dce9cb7a minor typing 2024-10-24 12:08:57 -07:00
Ashwin Bharambe
161aef0aae Small updates to quantization config 2024-10-24 12:08:56 -07:00
Xi Yan
32a496ab0f Merge branch 'evals_5' into evals_6 2024-10-24 12:01:41 -07:00
Xi Yan
a3a8f32541 add all rows scores to ScoringResult 2024-10-24 11:53:15 -07:00
Dalton Flanagan
8eceebec98
Update iOS inference instructions for new quantization 2024-10-24 14:47:27 -04:00
Xi Yan
737fcb795f evals with generation 2024-10-24 11:30:13 -07:00
Xi Yan
071dba8871 Merge branch 'main' into evals_5 2024-10-24 09:18:15 -07:00
Ashwin Bharambe
8aa8847b4a Bump version to 0.0.44 2024-10-24 08:41:39 -07:00
Ashwin Bharambe
7afe51c84d
New quantized models (#301) 2024-10-24 08:38:56 -07:00
Xi Yan
afa0c2b146 address comments 2024-10-23 22:17:38 -07:00
Ashwin Bharambe
05a8d47b98 Add a meta-reference-quantized-gpu distribution 2024-10-23 21:45:50 -07:00
Xi Yan
f5dcc03742 use pytorch/pytorch as base 2024-10-23 20:22:00 -07:00
Xi Yan
59c93548bc validate scorer input 2024-10-23 17:43:41 -07:00
Xi Yan
0ee82571a8 refactor 2024-10-23 17:30:10 -07:00
Xi Yan
7c803cef86 update scoring test 2024-10-23 17:22:48 -07:00
Xi Yan
3c6555c408 score batch 2024-10-23 16:38:00 -07:00
Xi Yan
eb572faf6f score batch impl 2024-10-23 16:19:25 -07:00
Xi Yan
4b1d7da030 equality scorer 2024-10-23 16:07:17 -07:00
Xi Yan
cad8c8710b Merge branch 'main' into evals_5 2024-10-23 15:33:36 -07:00
Xi Yan
caf253e08f Merge branch 'main' into evals_5 2024-10-23 15:33:00 -07:00
Xi Yan
0cec86453b
Fix issue w/ routing_table api getting added when router api is not specified (#298)
* fix issue w/ enforcing api

* cleanup

* inference only yaml
2024-10-23 15:27:22 -07:00
Xi Yan
35981a1a3b scorer wip 2024-10-23 15:02:54 -07:00
Xi Yan
70c08e694d basic scoring function works 2024-10-23 14:42:28 -07:00