phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

Update readme.md

Browse source
This commit is contained in:
Ishaan Jaff 2023-08-30 20:19:03 -07:00 • committed by GitHub
parent fd12287c0d
commit dbb4a0b0c4
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 5 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

7
cookbook/llm-ab-test-server/readme.md
View file

@ -34,8 +34,9 @@
LiteLLM allows you to call 100+ LLMs using completion LiteLLM allows you to call 100+ LLMs using completion
## This template server allows you to define LLMs with their A/B test ratios ## Usage - A/B Test LLMs in Production
### Set your A/B Test Ratios
```python ```python
llm_dict = { llm_dict = {
"gpt-4": 0.2, "gpt-4": 0.2,
@ -45,8 +46,10 @@ llm_dict = {
} }
``` ```
Easily call selected model during `completion` ### Select LLM + Make Completion call
Use weighted selection, and call the model using litellm.completion
```python ```python
from litellm import completion
selected_llm = random.choices(list(llm_dict.keys()), weights=list(llm_dict.values()))[0] selected_llm = random.choices(list(llm_dict.keys()), weights=list(llm_dict.values()))[0]
response = completion(model=selected_model, messages=[{ "content": "Hello, how are you?","role": "user"}]) response = completion(model=selected_model, messages=[{ "content": "Hello, how are you?","role": "user"}])