phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

feat(router.py): enable pre-call checks

Browse source 
filter models outside of context window limits of a given message for a model group

 https://github.com/BerriAI/litellm/issues/872
This commit is contained in:
Krrish Dholakia 2024-03-23 18:03:30 -07:00
parent 2fabff06c0
commit eb3ca85d7e
7 changed files with 3417 additions and 526 deletions
Download patch file Download diff file Expand all files Collapse all files

2
litellm/utils.py
View file

@ -3605,7 +3605,7 @@ def token_counter(
count_response_tokens=count_response_tokens,
)
else:
num_tokens = len(encoding.encode(text)) # type: ignore
num_tokens = len(encoding.encode(text, disallowed_special=())) # type: ignore
return num_tokens