150 commits

Author	SHA1	Message	Date
Krish Dholakia	2df5ce4b7c	Merge pull request #1182 from sumanth13131/usage-based-routing-fix ... usage_based_routing_fix	2023-12-23 11:50:34 +05:30
Krish Dholakia	710c809478	Merge pull request #1195 from AllentDan/fix-routing ... fix least_busy router by updating min_traffic	2023-12-23 11:45:35 +05:30
Krrish Dholakia	979575a2a6	fix(proxy_server.py): handle misformatted json body in chat completion request	2023-12-22 12:30:36 +05:30
Krrish Dholakia	be68796eba	fix(router.py): add support for async image generation endpoints	2023-12-21 14:38:44 +05:30
Krrish Dholakia	04bbd0649f	fix(router.py): only do sync image gen fallbacks for now ... The customhttptransport we use for dall-e-2 only works for sync httpx calls, not async. Will need to spend some time writing the async version n	2023-12-20 19:10:59 +05:30
Krrish Dholakia	4040f60feb	feat(router.py): support async image generation on router	2023-12-20 17:24:20 +05:30
AllentDan	6b19db0327	fix least_busy router by updating min_traffic	2023-12-20 18:16:00 +08:00
sumanth	96e23daf4d	fix	2023-12-19 13:15:48 +05:30
Krrish Dholakia	071283c102	fix(router.py): init deployment_latency_map even if model_list is empty	2023-12-18 17:50:26 -08:00
Krrish Dholakia	50b741f8fa	fix(Dockerfile): support mac	2023-12-16 16:01:02 -08:00
Krrish Dholakia	3291de9e11	fix(proxy_server.py): setup depedencies on server startup	2023-12-16 11:56:11 -08:00
Krrish Dholakia	84ad9f441e	feat(router.py): support caching groups	2023-12-15 21:45:51 -08:00
Krrish Dholakia	4d8376a8e9	fix(router.py): init clients for azure/openai on set_model_list	2023-12-15 20:23:29 -08:00
Krrish Dholakia	e5268fa6bc	fix(router.py): support openai-compatible endpoints	2023-12-15 14:47:54 -08:00
Krrish Dholakia	d2e9798de9	fix(router.py): adding mistral to list of openai-compatible endpoints	2023-12-15 14:24:43 -08:00
ishaan-jaff	b6a78cd349	(feat) router - remove confusing -ModelID-XXXX	2023-12-15 07:07:53 +05:30
Krrish Dholakia	1608dd7e0b	fix(main.py): support async streaming for text completions endpoint	2023-12-14 13:56:32 -08:00
ishaan-jaff	9ee16bc962	(feat) caching - add supported call types	2023-12-14 22:27:14 +05:30
ishaan-jaff	bfe1e3fc02	(feat) add mistral to proxy + router	2023-12-14 18:20:08 +05:30
ishaan-jaff	b7a5ab5ffa	(feat) proxy - use model_group_alias	2023-12-14 13:08:14 +05:30
Krrish Dholakia	9cf5ab468f	fix(router.py): deepcopy initial model list, don't mutate it	2023-12-12 09:54:06 -08:00
ishaan-jaff	a5dd8b1d4a	(fix) use deepcopy for model list	2023-12-12 09:53:52 -08:00
Krrish Dholakia	3e908bf507	fix(router.py): reset caching correctly	2023-12-11 19:57:34 -08:00
Krrish Dholakia	3b6099633c	test(test_custom_callback_router.py): add async azure testing for router	2023-12-11 16:40:35 -08:00
ishaan-jaff	0b7f8265d6	(fix) router - only init cache when its none	2023-12-09 15:20:08 -08:00
Krrish Dholakia	4bf875d3ed	fix(router.py): fix least-busy routing	2023-12-08 20:29:49 -08:00
ishaan-jaff	9805269bd7	(feat) proxy: pass model_info as kwargs	2023-12-08 14:26:18 -08:00
Krrish Dholakia	2f1c5aa0c7	fix: setting cache responses on proxy	2023-12-07 20:39:40 -08:00
ishaan-jaff	762f28e4d7	(fix) make print_verbose non blocking	2023-12-07 17:31:32 -08:00
Krrish Dholakia	e5638e2c5d	fix(router.py): fix default caching response value	2023-12-07 13:44:31 -08:00
ishaan-jaff	7b389d10ca	(linting) fix	2023-12-06 20:30:33 -08:00
ishaan-jaff	ee70c4e822	(feat) router - add model_group_alias_map	2023-12-06 20:13:33 -08:00
ishaan-jaff	13f9e78799	(fix) router - errors with reading timeout, stream timeout, max retries	2023-12-06 09:19:51 -08:00
ishaan-jaff	aab6be654e	(fix) router - set read os.environ/ values	2023-12-06 08:59:33 -08:00
ishaan-jaff	ff028111cf	(fix) router len(num_retries)	2023-12-05 22:05:47 -08:00
ishaan-jaff	8e6c4c5310	(fix) router - allow users to call a specific_model explicit	2023-12-05 21:57:00 -08:00
Krrish Dholakia	acef6bd58d	refactor: linting fixes	2023-12-05 21:43:02 -08:00
Krrish Dholakia	7b83238cb5	fix(router.py): log when a call is retried or fallback happens	2023-12-05 21:29:58 -08:00
ishaan-jaff	56acded998	(router) better debugging using config.yaml	2023-12-05 18:07:27 -08:00
ishaan-jaff	e788a34da4	(chore) linting fix	2023-12-05 17:26:03 -08:00
ishaan-jaff	a532cf14ae	(feat) router - track original deployment names	2023-12-05 17:19:48 -08:00
ishaan-jaff	bb6a1968b3	(fix) router - allow user to call 1 deployment	2023-12-05 16:56:38 -08:00
ishaan-jaff	d2dab362df	(fix) proxy debugging display Init API key	2023-12-05 16:08:17 -08:00
Krrish Dholakia	d606a9cb4c	refactor(router.py): linting fixes	2023-12-05 13:33:44 -08:00
ishaan-jaff	63939c0a11	(fix) linting	2023-12-05 13:30:12 -08:00
Krrish Dholakia	58ab0a3f03	fix(router.py): fix cache init	2023-12-05 12:54:27 -08:00
ishaan-jaff	3f84ab04c4	(fix) router: Azure Client Init	2023-12-05 12:54:00 -08:00
ishaan-jaff	d9f083b5f8	(fix) router: remove misleading print statement	2023-12-05 12:54:00 -08:00
ishaan-jaff	951bcfc043	(fix) router init: raise error Azure API Base not set	2023-12-05 12:20:42 -08:00
Krrish Dholakia	88c95ca259	fix(_redis.py): support additional params for redis	2023-12-05 12:16:51 -08:00