phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-27 19:54:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat(router.py): enable pre-call checks

Browse source 
filter models outside of context window limits of a given message for a model group

 https://github.com/BerriAI/litellm/issues/872
This commit is contained in:
Krrish Dholakia 2024-03-23 18:03:30 -07:00
parent 47b2b0df17
commit 4e70a3e09a
7 changed files with 3417 additions and 526 deletions
Download patch file Download diff file Expand all files Collapse all files

2
litellm/utils.py
View file

@ -3605,7 +3605,7 @@ def token_counter(
count_response_tokens=count_response_tokens,
)
else:
num_tokens = len(encoding.encode(text)) # type: ignore
num_tokens = len(encoding.encode(text, disallowed_special=())) # type: ignore
return num_tokens