phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

(docs) lm harness tutorial

Browse source
This commit is contained in:
ishaan-jaff 2023-11-04 17:51:16 -07:00
parent 53c2ada9e3
commit 45d681107d
1 changed files with 10 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

10
docs/my-website/docs/tutorials/lm_evaluation_harness.md
View file

@ -1,3 +1,6 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# LM-Evaluation Harness with TGI
Evaluate LLMs 20x faster with TGI via litellm proxy's `/completions` endpoint.
@ -9,6 +12,13 @@ This tutorial assumes you're using the `big-refactor` branch of [lm-evaluation-h
$ litellm --model huggingface/bigcode/starcoder
```
Using a custom api base
```shell
$ export HUGGINGFACE_API_KEY=my-api-key #[OPTIONAL]
$ litellm --model huggingface/tinyllama --api_base https://k58ory32yinf1ly0.us-east-1.aws.endpoints.huggingface.cloud
```
OpenAI Compatible Endpoint at http://0.0.0.0:8000
**Step 2: Set OpenAI API Base**