phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-25 10:44:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Litellm dev 02 13 2025 p2 (#8525)

Browse source 
* fix(azure/chat/gpt_transformation.py): add 'prediction' as a support azure param

Closes https://github.com/BerriAI/litellm/issues/8500

* build(model_prices_and_context_window.json): add new 'gemini-2.0-pro-exp-02-05' model

* style: cleanup invalid json trailing commma

* feat(utils.py): support passing 'tokenizer_config' to register_prompt_template

enables passing complete tokenizer config of model to litellm

 Allows calling deepseek on bedrock with the correct prompt template

* fix(utils.py): fix register_prompt_template for custom model names

* test(test_prompt_factory.py): fix test

* test(test_completion.py): add e2e test for bedrock invoke deepseek ft model

* feat(base_invoke_transformation.py): support hf_model_name param for bedrock invoke calls

enables proxy admin to set base model for ft bedrock deepseek model

* feat(bedrock/invoke): support deepseek_r1 route for bedrock

makes it easy to apply the right chat template to that call

* feat(constants.py): store deepseek r1 chat template - allow user to get correct response from deepseek r1 without extra work

* test(test_completion.py): add e2e mock test for bedrock deepseek

* docs(bedrock.md): document new deepseek_r1 route for bedrock

allows us to use the right config

* fix(exception_mapping_utils.py): catch read operation timeout
This commit is contained in:
Krish Dholakia 2025-02-13 20:28:42 -08:00 • committed by GitHub
parent 8903bd1c7f
commit 58141df65d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
15 changed files with 444 additions and 39 deletions
Download patch file Download diff file Expand all files Collapse all files

30
litellm/utils.py
View file

@ -5194,9 +5194,10 @@ def _calculate_retry_after(
# custom prompt helper function
def register_prompt_template(
model: str,
roles: dict,
roles: dict = {},
initial_prompt_value: str = "",
final_prompt_value: str = "",
tokenizer_config: dict = {},
):
"""
Register a prompt template to follow your custom format for a given model
@ -5233,12 +5234,27 @@ def register_prompt_template(
)
```
"""
model = get_llm_provider(model=model)[0]
litellm.custom_prompt_dict[model] = {
"roles": roles,
"initial_prompt_value": initial_prompt_value,
"final_prompt_value": final_prompt_value,
}
complete_model = model
potential_models = [complete_model]
try:
model = get_llm_provider(model=model)[0]
potential_models.append(model)
except Exception:
pass
if tokenizer_config:
for m in potential_models:
litellm.known_tokenizer_config[m] = {
"tokenizer": tokenizer_config,
"status": "success",
}
else:
for m in potential_models:
litellm.custom_prompt_dict[m] = {
"roles": roles,
"initial_prompt_value": initial_prompt_value,
"final_prompt_value": final_prompt_value,
}
return litellm.custom_prompt_dict