Commit graph

9 commits

Author SHA1 Message Date
Krish Dholakia
dec53961f7 LiteLLM Minor Fixes and Improvements (11/09/2024) (#5634)
* fix(caching.py): set ttl for async_increment cache

fixes issue where ttl for redis client was not being set on increment_cache

Fixes https://github.com/BerriAI/litellm/issues/5609

* fix(caching.py): fix increment cache w/ ttl for sync increment cache on redis

Fixes https://github.com/BerriAI/litellm/issues/5609

* fix(router.py): support adding retry policy + allowed fails policy via config.yaml

* fix(router.py): don't cooldown single deployments

No point, as there's no other deployment to loadbalance with.

* fix(user_api_key_auth.py): support setting allowed email domains on jwt tokens

Closes https://github.com/BerriAI/litellm/issues/5605

* docs(token_auth.md): add user upsert + allowed email domain to jwt auth docs

* fix(litellm_pre_call_utils.py): fix dynamic key logging when team id is set

Fixes issue where key logging would not be set if team metadata was not none

* fix(secret_managers/main.py): load environment variables correctly

Fixes issue where os.environ/ was not being loaded correctly

* test(test_router.py): fix test

* feat(spend_tracking_utils.py): support logging additional usage params - e.g. prompt caching values for deepseek

* test: fix tests

* test: fix test

* test: fix test

* test: fix test

* test: fix test
2024-09-11 22:36:06 -07:00
Ishaan Jaff
3e5641ca77 fix test test_router_completion_streaming 2024-06-21 21:02:34 -07:00
Krrish Dholakia
6f80dfc0d7 test(test_tpm_rpm_routing_v2.py): use mock endpoints for call 2024-06-20 14:09:45 -07:00
Krrish Dholakia
f21ec71caf test(test_tpm_rpm_routing_v2.py): fix test - bump number of iteration s 2024-04-30 08:48:55 -07:00
Krrish Dholakia
3afe7ab1a1 fix(lowest_tpm_rpm_v2.py): shuffle deployments with same tpm values 2024-04-29 15:23:47 -07:00
Krrish Dholakia
5da934099f fix(caching.py): dual cache async_batch_get_cache fix + testing
this fixes a bug in usage-based-routing-v2 which was caused b/c of how the result was being returned from dual cache async_batch_get_cache. it also adds unit testing for that function (and it's sync equivalent)
2024-04-19 15:03:25 -07:00
Krrish Dholakia
5bb73dc9c0 fix(router.py): instrument pre-call-checks for all openai endpoints 2024-04-18 21:54:25 -07:00
Krrish Dholakia
376ee4e9d7 fix(test_lowest_tpm_rpm_routing_v2.py): unit testing for usage-based-routing-v2 2024-04-18 21:38:00 -07:00
Krrish Dholakia
72691e05f4 fix(tpm_rpm_routing_v2.py): fix tpm rpm routing 2024-04-18 20:01:22 -07:00