From dbb4a0b0c4644c3ec69b902c1353367239d14f55 Mon Sep 17 00:00:00 2001 From: Ishaan Jaff Date: Wed, 30 Aug 2023 20:19:03 -0700 Subject: [PATCH] Update readme.md --- cookbook/llm-ab-test-server/readme.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/cookbook/llm-ab-test-server/readme.md b/cookbook/llm-ab-test-server/readme.md index 2512e106b..d7ac6a064 100644 --- a/cookbook/llm-ab-test-server/readme.md +++ b/cookbook/llm-ab-test-server/readme.md @@ -34,8 +34,9 @@ LiteLLM allows you to call 100+ LLMs using completion -## This template server allows you to define LLMs with their A/B test ratios +## Usage - A/B Test LLMs in Production +### Set your A/B Test Ratios ```python llm_dict = { "gpt-4": 0.2, @@ -45,8 +46,10 @@ llm_dict = { } ``` -Easily call selected model during `completion` +### Select LLM + Make Completion call +Use weighted selection, and call the model using litellm.completion ```python +from litellm import completion selected_llm = random.choices(list(llm_dict.keys()), weights=list(llm_dict.values()))[0] response = completion(model=selected_model, messages=[{ "content": "Hello, how are you?","role": "user"}])