phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

(docs) lm eval harness

Browse source
This commit is contained in:
ishaan-jaff 2023-11-04 17:55:32 -07:00
parent 44ee34529b
commit 6e4a575b0f
1 changed files with 14 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

15
docs/my-website/docs/tutorials/lm_evaluation_harness.md
View file

@ -21,11 +21,16 @@ $ litellm --model huggingface/tinyllama --api_base https://k58ory32yinf1ly0.us-e
OpenAI Compatible Endpoint at http://0.0.0.0:8000
**Step 2: Set OpenAI API Base**
**Step 2: Set OpenAI API Base & Key**
```shell
$ export OPENAI_API_BASE=http://0.0.0.0:8000
```
LM Harness requires you to set an OpenAI API key `OPENAI_API_SECRET_KEY` for running benchmarks
```shell
export OPENAI_API_SECRET_KEY=anything
```
**Step 3: Run LM-Eval-Harness**
```shell
@ -34,4 +39,12 @@ python3 -m lm_eval \
--model_args engine=davinci \
--task crows_pairs_english_age
```
## Debugging
### Making a test request to your proxy
This command makes a test Completion, ChatCompletion request to your proxy server
```shell
litellm --test
```