phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

docs + update benchmark.py

Browse source
This commit is contained in:
ishaan-jaff 2023-09-13 11:25:19 -07:00
parent 721d34bda2
commit 7158d0ba1f
2 changed files with 3 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

3
cookbook/benchmark/benchmark.py
View file

@ -9,13 +9,12 @@ import os
# Define the list of models to benchmark # Define the list of models to benchmark
# select any LLM listed here: https://docs.litellm.ai/docs/providers # select any LLM listed here: https://docs.litellm.ai/docs/providers
models = ['gpt-3.5-turbo', 'togethercomputer/llama-2-70b-chat', 'claude-2'] models = ['gpt-3.5-turbo', 'claude-2']
# Enter LLM API keys # Enter LLM API keys
# https://docs.litellm.ai/docs/providers # https://docs.litellm.ai/docs/providers
os.environ['OPENAI_API_KEY'] = "" os.environ['OPENAI_API_KEY'] = ""
os.environ['ANTHROPIC_API_KEY'] = "" os.environ['ANTHROPIC_API_KEY'] = ""
os.environ['TOGETHERAI_API_KEY'] = ""
# List of questions to benchmark (replace with your questions) # List of questions to benchmark (replace with your questions)
questions = [ questions = [

4
docs/my-website/docs/tutorials/compare_llms.md
View file

@ -13,7 +13,7 @@ Easily benchmark LLMs for a given question by viewing
``` ```
git clone https://github.com/BerriAI/litellm git clone https://github.com/BerriAI/litellm
``` ```
cd to `benchmark` dir cd to `litellm/cookbook/benchmark` dir
Located here: Located here:
https://github.com/BerriAI/litellm/tree/main/cookbook/benchmark https://github.com/BerriAI/litellm/tree/main/cookbook/benchmark
@ -26,7 +26,7 @@ cd litellm/cookbook/benchmark
pip install litellm click tqdm tabulate termcolor pip install litellm click tqdm tabulate termcolor
``` ```
### Configuration ### Configuration - Set LLM API Keys + LLMs in benchmark.py
In `benchmark/benchmark.py` select your LLMs, LLM API Key and questions In `benchmark/benchmark.py` select your LLMs, LLM API Key and questions
Supported LLMs: https://docs.litellm.ai/docs/providers Supported LLMs: https://docs.litellm.ai/docs/providers