Krish Dholakia
fcde3ba213
Merge pull request #3192 from BerriAI/litellm_calculate_max_parallel_requests
...
fix(router.py): Make TPM limits concurrency-safe
2024-04-20 13:24:29 -07:00
Ishaan Jaff
f89f8a4157
Merge pull request #3184 from BerriAI/litellm_ui_non_admins_flow
...
[UI] - non admin flow - only Create + Test Key available
2024-04-20 12:40:43 -07:00
Ishaan Jaff
7b24a74e77
test fix - test_key_delete_ui
2024-04-20 12:30:58 -07:00
Krrish Dholakia
1507b23e30
test(test_openai_endpoints.py): make test stricter
2024-04-20 12:11:54 -07:00
Ishaan Jaff
a34f725db7
fix - test keys
2024-04-20 11:43:43 -07:00
Krrish Dholakia
01a1a8f731
fix(caching.py): dual cache async_batch_get_cache fix + testing
...
this fixes a bug in usage-based-routing-v2 which was caused b/c of how the result was being returned from dual cache async_batch_get_cache. it also adds unit testing for that function (and it's sync equivalent)
2024-04-19 15:03:25 -07:00
Krrish Dholakia
2ffd057042
test(test_models.py): ensure only admin can call /health
2024-04-16 18:13:40 -07:00
Krrish Dholakia
70716b3373
fix(proxy_server.py): fix key create logic + add unit tests
2024-04-16 13:08:34 -07:00
Krrish Dholakia
3f8abe2754
fix(proxy_server.py): secure /team/info endpoint
...
make sure user requesting team info is part of team or admin
2024-04-16 11:40:08 -07:00
Ishaan Jaff
b27e1f5c5c
test - setting up langfuse callback on proxy
2024-04-16 08:45:51 -07:00
Krrish Dholakia
2d4fe072ad
fix(proxy_server.py): fix delete models endpoint
...
https://github.com/BerriAI/litellm/issues/2951
2024-04-15 18:34:58 -07:00
Ishaan Jaff
756966c8be
Merge pull request #3047 from BerriAI/litellm_fix_key_delete_bug
...
[Fix + Test] key delete bug
2024-04-15 17:38:25 -07:00
Krrish Dholakia
f9a9556f6c
test(test_models.py): assert model id in db is the model id passed in
2024-04-15 17:37:45 -07:00
Ishaan Jaff
e20d3f29b0
test - delete keys
2024-04-15 16:37:20 -07:00
Krish Dholakia
a4a21aff41
Merge pull request #3034 from BerriAI/litellm_team_table_fix
...
fix(proxy_server.py): fix /team/update endpoint
2024-04-15 14:40:12 -07:00
Ishaan Jaff
adae555fb1
Merge branch 'main' into litellm_fix_using_wildcard_openai_models_proxy
2024-04-15 14:35:06 -07:00
Ishaan Jaff
6df5337e65
test - wildcard openai models on proxy
2024-04-15 14:05:26 -07:00
Ishaan Jaff
ecc6aa060f
test - team based logging on proxy
2024-04-15 13:26:55 -07:00
Ishaan Jaff
464d812fa9
test - team based logging
2024-04-15 13:13:51 -07:00
Krrish Dholakia
8a32d8825c
fix(proxy_server.py): fix /team/update endpoint
...
this also removes the functionality for modifying membership via `/team/update`. This is not considered a breaking change, as we've been pointing to usage of `/team/member_add` and `/team/member_delete` for this. Previous behaviour for this endpoint was causing it to be non-functional. If you're impacted, please let us know.
2024-04-15 10:29:21 -07:00
Ishaan Jaff
ca95a64e05
fix test /health
2024-04-13 12:30:26 -07:00
Ishaan Jaff
6ad853c768
fix - test add models + call /health
2024-04-13 10:51:11 -07:00
Ishaan Jaff
4bf32cafa3
fix - add models + /health test
2024-04-13 10:49:41 -07:00
Ishaan Jaff
f0ebadbb6f
test - /health endpoints
2024-04-13 10:09:18 -07:00
Krrish Dholakia
ea1574c160
test(test_openai_endpoints.py): add concurrency testing for user defined rate limits on proxy
2024-04-12 18:56:13 -07:00
Krrish Dholakia
bdfb74f8a5
test(test_openai_endpoints.py): add local test, for proxy concurrency
2024-04-11 17:16:23 -07:00
Krrish Dholakia
a652a9b3e9
test(test_models.py): combine add + delete model test
2024-04-08 13:29:03 -07:00
Krrish Dholakia
0d1cca9aa0
fix(router.py): make router async calls coroutine safe
...
uses pre-call checks to check if a call is below it's rpm limit, works even if multiple async calls are
made simultaneously
2024-04-06 17:31:26 -07:00
Krish Dholakia
9119858f4a
Merge pull request #2798 from CLARKBENHAM/main
...
add test for rate limits - Router isn't coroutine safe
2024-04-06 08:47:40 -07:00
Krrish Dholakia
591bd9fcdd
test(test_openai_endpoints.py): add response header test - make sure it's always <4kb
2024-04-05 12:49:29 -07:00
Krrish Dholakia
1ee78c39e9
fix(proxy_server.py): skip to next model if invalid model (raise logger.error)
2024-04-04 19:22:17 -07:00
Krrish Dholakia
4b56f08cbe
test(test_models.py): fix delete model test
2024-04-04 08:46:08 -07:00
CLARKBENHAM
44cb0f352a
formating
2024-04-02 19:56:07 -07:00
Krrish Dholakia
d7601a4844
perf(proxy_server.py): batch write spend logs
...
reduces prisma client errors, by batch writing spend logs - max 1k logs at a time
2024-04-02 18:46:55 -07:00
CLARKBENHAM
6b0f3d3fd3
add routing strategy
2024-04-02 17:26:57 -07:00
CLARKBENHAM
2d7bafa12d
comments
2024-04-02 17:18:56 -07:00
CLARKBENHAM
75d029a91d
comments
2024-04-02 17:18:03 -07:00
CLARKBENHAM
f32d39aa93
check if in future None results are returned
2024-04-02 14:49:20 -07:00
CLARKBENHAM
a2bf891e48
test for rate limits
2024-04-02 13:49:13 -07:00
Krrish Dholakia
60f89faf1c
test: cleanup
2024-03-30 21:40:43 -07:00
Krrish Dholakia
c0204310ee
fix(main.py): fix translation to text_completions format for async text completion calls
2024-03-30 09:02:51 -07:00
Krrish Dholakia
af2eabba91
fix(proxy_server.py): fix /key/update endpoint to update key duration
...
also adds a test for this to our ci/cd
2024-03-29 21:47:10 -07:00
Ishaan Jaff
6b08be4c8e
Update README.MD
2024-03-29 14:56:41 -07:00
Ishaan Jaff
a951738849
Update README.MD
2024-03-29 14:56:04 -07:00
Krrish Dholakia
58bdbecbc6
fix(proxy_server.py): allow updating tpm/rpm limits via /team/update
2024-03-28 15:59:35 -07:00
Ishaan Jaff
75614bc29c
(fix) img gen endpoints unstable
2024-03-28 12:43:46 -07:00
Ishaan Jaff
2495587e7c
(fix) test img gen proxy
2024-03-28 12:33:08 -07:00
Krrish Dholakia
49e8cdbff9
fix(router.py): check for context window error when handling 400 status code errors
...
was causing proxy context window fallbacks to not work as expected
2024-03-26 08:08:15 -07:00
Krrish Dholakia
c81c9c2583
fix(proxy_server.py): fix model info check
2024-03-23 15:59:17 -07:00
Krrish Dholakia
066346b8c8
test: handle flaky tests locally - causing db timeout issues on circle ci
2024-03-19 20:39:53 -07:00
