Krrish Dholakia
01a1a8f731
fix(caching.py): dual cache async_batch_get_cache fix + testing
...
this fixes a bug in usage-based-routing-v2 which was caused b/c of how the result was being returned from dual cache async_batch_get_cache. it also adds unit testing for that function (and it's sync equivalent)
2024-04-19 15:03:25 -07:00
Ishaan Jaff
6df5337e65
test - wildcard openai models on proxy
2024-04-15 14:05:26 -07:00
Krrish Dholakia
ea1574c160
test(test_openai_endpoints.py): add concurrency testing for user defined rate limits on proxy
2024-04-12 18:56:13 -07:00
Krrish Dholakia
bdfb74f8a5
test(test_openai_endpoints.py): add local test, for proxy concurrency
2024-04-11 17:16:23 -07:00
Krrish Dholakia
591bd9fcdd
test(test_openai_endpoints.py): add response header test - make sure it's always <4kb
2024-04-05 12:49:29 -07:00
Krrish Dholakia
c0204310ee
fix(main.py): fix translation to text_completions format for async text completion calls
2024-03-30 09:02:51 -07:00
Ishaan Jaff
75614bc29c
(fix) img gen endpoints unstable
2024-03-28 12:43:46 -07:00
Ishaan Jaff
2495587e7c
(fix) test img gen proxy
2024-03-28 12:33:08 -07:00
Krrish Dholakia
20def3805c
test(test_openai_endpoints.py): fix test
2024-02-23 22:57:20 -08:00
Krrish Dholakia
83d43809a7
fix(proxy_server.py): fix proxy server team id bug
2024-02-13 22:33:56 -08:00
Krrish Dholakia
f8870fb48e
fix(utils.py): fix proxy streaming spend tracking
2024-01-23 15:59:03 -08:00
Krrish Dholakia
f5ced089d6
test(tests/): add unit testing for proxy server endpoints
2024-01-19 14:54:29 -08:00
