Ishaan Jaff
cb386fda20
test - making mistral embedding request on proxy
2024-06-12 15:10:20 -07:00
Krrish Dholakia
79287a7584
feat(scheduler.py): add request prioritization scheduler
...
allow user to set priority for a request
2024-05-31 18:51:13 -07:00
Ishaan Jaff
d4288b134b
fix - use csv list for batch completions
2024-05-11 14:24:48 -07:00
Ishaan Jaff
e1f94fcbbb
test batch completions on litellm proxy
2024-05-11 13:32:30 -07:00
Krrish Dholakia
b014a72f7a
test(test_openai_endpoints.py): change key
2024-05-06 11:19:47 -07:00
Krrish Dholakia
e31eb6a6b2
test(test_openai_endpoints.py): add unit testing for streaming + logprobs on docker image
2024-04-23 19:26:19 -07:00
Krrish Dholakia
1507b23e30
test(test_openai_endpoints.py): make test stricter
2024-04-20 12:11:54 -07:00
Krrish Dholakia
01a1a8f731
fix(caching.py): dual cache async_batch_get_cache fix + testing
...
this fixes a bug in usage-based-routing-v2 which was caused b/c of how the result was being returned from dual cache async_batch_get_cache. it also adds unit testing for that function (and it's sync equivalent)
2024-04-19 15:03:25 -07:00
Ishaan Jaff
6df5337e65
test - wildcard openai models on proxy
2024-04-15 14:05:26 -07:00
Krrish Dholakia
ea1574c160
test(test_openai_endpoints.py): add concurrency testing for user defined rate limits on proxy
2024-04-12 18:56:13 -07:00
Krrish Dholakia
bdfb74f8a5
test(test_openai_endpoints.py): add local test, for proxy concurrency
2024-04-11 17:16:23 -07:00
Krrish Dholakia
591bd9fcdd
test(test_openai_endpoints.py): add response header test - make sure it's always <4kb
2024-04-05 12:49:29 -07:00
Krrish Dholakia
c0204310ee
fix(main.py): fix translation to text_completions format for async text completion calls
2024-03-30 09:02:51 -07:00
Ishaan Jaff
75614bc29c
(fix) img gen endpoints unstable
2024-03-28 12:43:46 -07:00
Ishaan Jaff
2495587e7c
(fix) test img gen proxy
2024-03-28 12:33:08 -07:00
Krrish Dholakia
20def3805c
test(test_openai_endpoints.py): fix test
2024-02-23 22:57:20 -08:00
Krrish Dholakia
83d43809a7
fix(proxy_server.py): fix proxy server team id bug
2024-02-13 22:33:56 -08:00
Krrish Dholakia
f8870fb48e
fix(utils.py): fix proxy streaming spend tracking
2024-01-23 15:59:03 -08:00
Krrish Dholakia
f5ced089d6
test(tests/): add unit testing for proxy server endpoints
2024-01-19 14:54:29 -08:00
