15 commits

Author	SHA1	Message	Date
Krrish Dholakia	fba36714af	test(test_openai_endpoints.py): change key	2024-05-06 11:19:47 -07:00
Krrish Dholakia	264ec2de02	test(test_openai_endpoints.py): add unit testing for streaming + logprobs on docker image	2024-04-23 19:26:19 -07:00
Krrish Dholakia	1ea507419d	test(test_openai_endpoints.py): make test stricter	2024-04-20 12:11:54 -07:00
Krrish Dholakia	5da934099f	fix(caching.py): dual cache async_batch_get_cache fix + testing ... this fixes a bug in usage-based-routing-v2 which was caused b/c of how the result was being returned from dual cache async_batch_get_cache. it also adds unit testing for that function (and it's sync equivalent)	2024-04-19 15:03:25 -07:00
Ishaan Jaff	e33fd644f2	test - wildcard openai models on proxy	2024-04-15 14:05:26 -07:00
Krrish Dholakia	c177407f7b	test(test_openai_endpoints.py): add concurrency testing for user defined rate limits on proxy	2024-04-12 18:56:13 -07:00
Krrish Dholakia	200317ce06	test(test_openai_endpoints.py): add local test, for proxy concurrency	2024-04-11 17:16:23 -07:00
Krrish Dholakia	549ae397c6	test(test_openai_endpoints.py): add response header test - make sure it's always <4kb	2024-04-05 12:49:29 -07:00
Krrish Dholakia	5c199e4e4e	fix(main.py): fix translation to text_completions format for async text completion calls	2024-03-30 09:02:51 -07:00
Ishaan Jaff	ce9fd8b351	(fix) img gen endpoints unstable	2024-03-28 12:43:46 -07:00
Ishaan Jaff	5643f1c919	(fix) test img gen proxy	2024-03-28 12:33:08 -07:00
Krrish Dholakia	41f4f7f0df	test(test_openai_endpoints.py): fix test	2024-02-23 22:57:20 -08:00
Krrish Dholakia	b501af8f43	fix(proxy_server.py): fix proxy server team id bug	2024-02-13 22:33:56 -08:00
Krrish Dholakia	344e232549	fix(utils.py): fix proxy streaming spend tracking	2024-01-23 15:59:03 -08:00
Krrish Dholakia	beca1eaae7	test(tests/): add unit testing for proxy server endpoints	2024-01-19 14:54:29 -08:00