Commit graph

22 commits

Author SHA1 Message Date
Krrish Dholakia
3560f0ef2c refactor: move all testing to top-level of repo
Closes https://github.com/BerriAI/litellm/issues/486
2024-09-28 21:08:14 -07:00
Ishaan Jaff
eb325cce7d
(perf improvement proxy) use one redis set cache to update spend in db (30-40% perf improvement) (#5960)
* use one set op to update spend in db

* fix test_team_cache_update_called
2024-09-28 13:00:31 -07:00
Krish Dholakia
98c34a7e27
LiteLLM Minor Fixes and Improvements (11/09/2024) (#5634)
* fix(caching.py): set ttl for async_increment cache

fixes issue where ttl for redis client was not being set on increment_cache

Fixes https://github.com/BerriAI/litellm/issues/5609

* fix(caching.py): fix increment cache w/ ttl for sync increment cache on redis

Fixes https://github.com/BerriAI/litellm/issues/5609

* fix(router.py): support adding retry policy + allowed fails policy via config.yaml

* fix(router.py): don't cooldown single deployments

No point, as there's no other deployment to loadbalance with.

* fix(user_api_key_auth.py): support setting allowed email domains on jwt tokens

Closes https://github.com/BerriAI/litellm/issues/5605

* docs(token_auth.md): add user upsert + allowed email domain to jwt auth docs

* fix(litellm_pre_call_utils.py): fix dynamic key logging when team id is set

Fixes issue where key logging would not be set if team metadata was not none

* fix(secret_managers/main.py): load environment variables correctly

Fixes issue where os.environ/ was not being loaded correctly

* test(test_router.py): fix test

* feat(spend_tracking_utils.py): support logging additional usage params - e.g. prompt caching values for deepseek

* test: fix tests

* test: fix test

* test: fix test

* test: fix test

* test: fix test
2024-09-11 22:36:06 -07:00
Krrish Dholakia
caa01d20cb build: re-run ci/cd 2024-07-13 11:41:35 -07:00
Krrish Dholakia
25b1463920 style: trigger new build 2024-06-29 19:26:21 -07:00
Krrish Dholakia
6558abf845 fix(proxy_server.py): track team spend for cached team object
fixes issue where team budgets for jwt tokens weren't asserted
2024-06-18 17:10:12 -07:00
Krrish Dholakia
0700aed7d1 fix(test_jwt.py): fix import 2024-06-17 21:08:01 -07:00
Ishaan Jaff
4893fbe27b fix test jwt 2024-06-15 14:24:00 -07:00
Ishaan Jaff
c6986c66cc fix new team test 2024-06-15 13:24:13 -07:00
Ishaan Jaff
b507c54385 fix new team test 2024-06-11 22:03:03 -07:00
Krrish Dholakia
a33b9221da fix(proxy_server.py): fix admin allowed routes 2024-05-22 20:55:21 -07:00
Krrish Dholakia
600b6f7e1d feat(proxy_server.py): support 'user_id_upsert' flag for jwt_auth 2024-05-15 22:19:59 -07:00
Krrish Dholakia
99653d2d3e feat(handle_jwt.py): add support for 'team_id_default
allows admin to set a default team id for spend-tracking + permissions
2024-05-15 21:33:35 -07:00
Krrish Dholakia
f48cd87cf3 feat(proxy_server.py): make team_id optional for jwt token auth (only enforced, if set)
Allows users to use jwt auth for internal chat apps
2024-05-15 21:05:14 -07:00
Sara Ghaemi
86e0dd68c3 updated tests 2024-05-07 13:28:57 -04:00
Sara Ghaemi
7017899d37 updated tests to also check for audience if found 2024-05-07 12:10:47 -04:00
Krrish Dholakia
77097f8e7d test(test_jwt.py): add testing to make sure user api key auth returns the expected values 2024-04-11 20:18:55 -07:00
Krrish Dholakia
5a117490ec fix(proxy_server.py): fix tpm/rpm limiting for jwt auth
fixes tpm/rpm limiting for jwt auth and implements unit tests for jwt auth
2024-03-28 21:19:34 -07:00
Krrish Dholakia
ea8f6672c5 test(test_jwt.py): fix test 2024-03-26 15:22:05 -07:00
Krrish Dholakia
752516df1b fix(handle_jwt.py): support public key caching ttl param 2024-03-26 14:32:55 -07:00
Krrish Dholakia
2e4e97a48f test(test_jwt.py): add unit tests for jwt auth integration 2024-03-25 13:24:39 -07:00
Krrish Dholakia
93959ab5aa fix(handle_jwt.py): allow setting proxy admin role string for jwt auth 2024-03-25 12:20:14 -07:00