phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-26 11:14:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
17160 commits 2484 branches 894 tags 537 MiB
Commit graph

20 commits

Author SHA1 Message Date
Krrish Dholakia
7129e93992 fix(utils.py): fix get_image_dimensions to handle more image types 
Fixes https://github.com/BerriAI/litellm/issues/5205
 2024-08-16 12:00:04 -07:00
Krrish Dholakia
a8644d8a7d fix(utils.py): if openai model, don't check hf tokenizers 2024-08-12 16:28:22 -07:00
Krrish Dholakia
95a0f6839f fix(utils.py): fix token_counter to handle empty tool calls in messages 
Fixes https://github.com/BerriAI/litellm/pull/4749
 2024-07-19 19:39:00 -07:00
Pamela Fox
d43dbc756b Count tokens for tools 2024-07-15 11:07:52 -07:00
Krrish Dholakia
d421486a45 fix(token_counter.py): New `get_modified_max_tokens' helper func 
Fixes https://github.com/BerriAI/litellm/issues/4439
 2024-06-27 15:38:09 -07:00
Krrish Dholakia
b756374a4f feat(utils.py): bump tiktoken dependency to 0.7.0 
adds support for gpt-4o token counting
 2024-06-10 21:21:23 -07:00
Raymond Huang
968f54a65c test 2024-06-06 10:11:27 -07:00
Krrish Dholakia
f43da3597d test: fix test 2024-05-15 08:51:40 -07:00
Krrish Dholakia
900bb9aba8 test(test_token_counter.py): fix load test 2024-05-15 07:12:43 -07:00
Krrish Dholakia
73b6b5e804 test(test_token_counter.py): fix token counting test 2024-05-14 21:35:28 -07:00
Krrish Dholakia
a1dd341ca1 fix(utils.py): default claude-3 to tiktoken (0.8s faster than hf tokenizer) 2024-05-14 18:37:14 -07:00
Christian Privitelli
2d43153efa include methods in init import, add test, fix encode/decode param ordering 2024-05-02 15:49:22 +10:00
ishaan-jaff
ff815e93a8 (fix) test_tokenizers 2024-02-23 10:13:38 -08:00
Krrish Dholakia
0fffcc1579 fix(utils.py): support token counting for gpt-4-vision models 2024-01-02 14:41:42 +05:30
Krrish Dholakia
c4fc28ab0d fix(utils.py): use local tiktoken copy 2023-12-28 11:22:33 +05:30
Krrish Dholakia
4905929de3 refactor: add black formatting 2023-12-25 14:11:20 +05:30
Krrish Dholakia
7b99cfe6b9 test(test_token_counter.py): include testing for function calls 2023-11-30 18:24:21 -08:00
ishaan-jaff
b74b051385 (test) tokenizer worked test 2023-11-11 12:13:57 -08:00
Krrish Dholakia
4eeadd284a feat(utils.py): adding encode and decode functions 2023-10-20 11:59:47 -07:00
Krrish Dholakia
e62b584439 adding support for cohere, anthropic, llama2 tokenizers 2023-09-22 14:03:52 -07:00