phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

Merge pull request #2868 from BerriAI/litellm_add_command_r_on_proxy

Browse source 
Add Azure Command-r-plus on litellm proxy
This commit is contained in:
Ishaan Jaff 2024-04-05 15:13:47 -07:00 • committed by GitHub
commit faa0d38087
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 147 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

5
litellm/router.py
View file

@ -1772,6 +1772,11 @@ class Router:
or "ft:gpt-3.5-turbo" in model_name
or model_name in litellm.open_ai_embedding_models
):
if custom_llm_provider == "azure":
if litellm.utils._is_non_openai_azure_model(model_name):
custom_llm_provider = "openai"
# remove azure prefx from model_name
model_name = model_name.replace("azure/", "")
# glorified / complicated reading of configs
# user can pass vars directly or they can pas os.environ/AZURE_API_KEY, in which case we will read the env
# we do this here because we init clients for Azure, OpenAI and we need to set the right key