135 commits

Author	SHA1	Message	Date
ishaan-jaff	5731b3bc23	(feat) router - remove confusing -ModelID-XXXX	2023-12-15 07:07:53 +05:30
Krrish Dholakia	bb5b883316	fix(main.py): support async streaming for text completions endpoint	2023-12-14 13:56:32 -08:00
ishaan-jaff	072fdac48c	(feat) caching - add supported call types	2023-12-14 22:27:14 +05:30
ishaan-jaff	18478382bf	(feat) add mistral to proxy + router	2023-12-14 18:20:08 +05:30
ishaan-jaff	0ea0e29bf1	(feat) proxy - use model_group_alias	2023-12-14 13:08:14 +05:30
Krrish Dholakia	0cf0c2d6dd	fix(router.py): deepcopy initial model list, don't mutate it	2023-12-12 09:54:06 -08:00
ishaan-jaff	5e9286ed41	(fix) use deepcopy for model list	2023-12-12 09:53:52 -08:00
Krrish Dholakia	bbf094dcf5	fix(router.py): reset caching correctly	2023-12-11 19:57:34 -08:00
Krrish Dholakia	4fd400015f	test(test_custom_callback_router.py): add async azure testing for router	2023-12-11 16:40:35 -08:00
ishaan-jaff	ae0cb38e47	(fix) router - only init cache when its none	2023-12-09 15:20:08 -08:00
Krrish Dholakia	a65c8919fc	fix(router.py): fix least-busy routing	2023-12-08 20:29:49 -08:00
ishaan-jaff	0740728be0	(feat) proxy: pass model_info as kwargs	2023-12-08 14:26:18 -08:00
Krrish Dholakia	30204497e0	fix: setting cache responses on proxy	2023-12-07 20:39:40 -08:00
ishaan-jaff	f744445db4	(fix) make print_verbose non blocking	2023-12-07 17:31:32 -08:00
Krrish Dholakia	69c34493ce	fix(router.py): fix default caching response value	2023-12-07 13:44:31 -08:00
ishaan-jaff	8920f12a8d	(linting) fix	2023-12-06 20:30:33 -08:00
ishaan-jaff	1177c54636	(feat) router - add model_group_alias_map	2023-12-06 20:13:33 -08:00
ishaan-jaff	2ecbd92f13	(fix) router - errors with reading timeout, stream timeout, max retries	2023-12-06 09:19:51 -08:00
ishaan-jaff	788da2ea68	(fix) router - set read os.environ/ values	2023-12-06 08:59:33 -08:00
ishaan-jaff	105da7bd58	(fix) router len(num_retries)	2023-12-05 22:05:47 -08:00
ishaan-jaff	905f366f43	(fix) router - allow users to call a specific_model explicit	2023-12-05 21:57:00 -08:00
Krrish Dholakia	077d2fbb2e	refactor: linting fixes	2023-12-05 21:43:02 -08:00
Krrish Dholakia	4ecd05df3e	fix(router.py): log when a call is retried or fallback happens	2023-12-05 21:29:58 -08:00
ishaan-jaff	cc2bb5daa6	(router) better debugging using config.yaml	2023-12-05 18:07:27 -08:00
ishaan-jaff	a73a2eedc0	(chore) linting fix	2023-12-05 17:26:03 -08:00
ishaan-jaff	aa795f3118	(feat) router - track original deployment names	2023-12-05 17:19:48 -08:00
ishaan-jaff	8c6103259b	(fix) router - allow user to call 1 deployment	2023-12-05 16:56:38 -08:00
ishaan-jaff	862d4bdef1	(fix) proxy debugging display Init API key	2023-12-05 16:08:17 -08:00
Krrish Dholakia	f2e8239678	refactor(router.py): linting fixes	2023-12-05 13:33:44 -08:00
ishaan-jaff	917d6bd7cb	(fix) linting	2023-12-05 13:30:12 -08:00
Krrish Dholakia	2e5eb56e7c	fix(router.py): fix cache init	2023-12-05 12:54:27 -08:00
ishaan-jaff	a781bb825c	(fix) router: Azure Client Init	2023-12-05 12:54:00 -08:00
ishaan-jaff	50e7de1ac6	(fix) router: remove misleading print statement	2023-12-05 12:54:00 -08:00
ishaan-jaff	aab443c30c	(fix) router init: raise error Azure API Base not set	2023-12-05 12:20:42 -08:00
Krrish Dholakia	94abb14b99	fix(_redis.py): support additional params for redis	2023-12-05 12:16:51 -08:00
ishaan-jaff	b7e98582f7	(fix) proxy: bug non OpenAI LLMs	2023-12-05 09:05:53 -08:00
ishaan-jaff	bd34c90b27	(feat) router: init stream, async stream, async, clients	2023-12-04 17:31:12 -08:00
Krrish Dholakia	d1a525b6c9	feat(utils.py): add async success callbacks for custom functions	2023-12-04 16:42:40 -08:00
ishaan-jaff	76715e09ee	(feat) router: set max_retries + timeout	2023-12-04 16:09:31 -08:00
Krrish Dholakia	284fb64f4d	feat: support for azure key vault	2023-12-01 19:36:06 -08:00
Krrish Dholakia	a4308fadce	fix(router.py): back-off if no models available	2023-11-30 18:42:29 -08:00
Krrish Dholakia	0e1c747f12	fix(router.py): support cloudflare ai gateway for azure models on router	2023-11-30 14:09:06 -08:00
ishaan-jaff	9ac8d403d6	(feat) proxy: config - azure allow users to pass in base_url	2023-11-30 10:56:55 -08:00
Krrish Dholakia	a1ea893a73	fix(main.py): don't pass stream to petals	2023-11-29 19:58:04 -08:00
ishaan-jaff	3ebc008c40	(feat) router: async client Azure, OpenAI	2023-11-29 19:45:08 -08:00
Krrish Dholakia	a05722571b	fix(replicate.py): fix custom prompt formatting	2023-11-29 19:44:09 -08:00
ishaan-jaff	cf9f3d0005	(fix) router: azure/embedding support	2023-11-29 19:06:36 -08:00
ishaan-jaff	e03ae74387	(fix) router: set default rpm/tpm when not set	2023-11-29 18:13:27 -08:00
ishaan-jaff	e365aea138	(feat) router: random pick based on tpm/rpm	2023-11-29 17:54:06 -08:00
ishaan-jaff	001ff8d11e	(fix) use weighted shuffle when rpm set	2023-11-29 17:13:11 -08:00