Krish Dholakia
aa9f1896c6
anthropic prompt caching cost tracking ( #5453 )
...
* fix(utils.py): support 'drop_params' for embedding requests
Fixes https://github.com/BerriAI/litellm/issues/5444
* feat(anthropic/cost_calculation.py): Support calculating cost for prompt caching on anthropic
* feat(types/utils.py): allows us to migrate to openai's equivalent, once that comes out
* fix: fix linting errors
* test: mark flaky test
2024-08-31 14:50:12 -07:00
Ishaan Jaff
041764a132
retry flaky tests 3 times
2024-08-28 13:10:47 -07:00
Krrish Dholakia
0bc08063e1
fix(dynamic_rate_limiter.py): support setting priority + reserving tpm/rpm
2024-07-01 23:08:54 -07:00
Krrish Dholakia
eadaa51273
test(test_dynamic_rate_limit_handler.py): add unit tests for dynamic rpm limits
2024-07-01 20:20:24 -07:00
Krrish Dholakia
f74490c69b
test(test_dynamic_rate_limit_handler.py): refactor tests for rpm suppprt
2024-07-01 20:16:10 -07:00
Krrish Dholakia
cf52d3fa00
test: skip unstable tests
2024-06-23 00:30:45 -07:00
Krrish Dholakia
acddf355e1
fix(test_dynamic_rate_limit_handler.py): cleanup
2024-06-22 22:43:56 -07:00
Krrish Dholakia
8843b0dc77
feat(dynamic_rate_limiter.py): working e2e
2024-06-22 14:41:22 -07:00
Krrish Dholakia
6a7982fa40
feat(dynamic_rate_limiter.py): passing base case
2024-06-21 22:46:46 -07:00
Krrish Dholakia
0430807178
feat(dynamic_rate_limiter.py): update cache with active project
2024-06-21 20:25:40 -07:00
Krrish Dholakia
89dba82be9
feat(dynamic_rate_limiter.py): initial commit for dynamic rate limiting
...
Closes https://github.com/BerriAI/litellm/issues/4124
2024-06-21 18:41:31 -07:00