104 commits

Author	SHA1	Message	Date
ishaan-jaff	a781bb825c	(fix) router: Azure Client Init	2023-12-05 12:54:00 -08:00
ishaan-jaff	50e7de1ac6	(fix) router: remove misleading print statement	2023-12-05 12:54:00 -08:00
ishaan-jaff	aab443c30c	(fix) router init: raise error Azure API Base not set	2023-12-05 12:20:42 -08:00
Krrish Dholakia	94abb14b99	fix(_redis.py): support additional params for redis	2023-12-05 12:16:51 -08:00
ishaan-jaff	b7e98582f7	(fix) proxy: bug non OpenAI LLMs	2023-12-05 09:05:53 -08:00
ishaan-jaff	bd34c90b27	(feat) router: init stream, async stream, async, clients	2023-12-04 17:31:12 -08:00
Krrish Dholakia	d1a525b6c9	feat(utils.py): add async success callbacks for custom functions	2023-12-04 16:42:40 -08:00
ishaan-jaff	76715e09ee	(feat) router: set max_retries + timeout	2023-12-04 16:09:31 -08:00
Krrish Dholakia	284fb64f4d	feat: support for azure key vault	2023-12-01 19:36:06 -08:00
Krrish Dholakia	a4308fadce	fix(router.py): back-off if no models available	2023-11-30 18:42:29 -08:00
Krrish Dholakia	0e1c747f12	fix(router.py): support cloudflare ai gateway for azure models on router	2023-11-30 14:09:06 -08:00
ishaan-jaff	9ac8d403d6	(feat) proxy: config - azure allow users to pass in base_url	2023-11-30 10:56:55 -08:00
Krrish Dholakia	a1ea893a73	fix(main.py): don't pass stream to petals	2023-11-29 19:58:04 -08:00
ishaan-jaff	3ebc008c40	(feat) router: async client Azure, OpenAI	2023-11-29 19:45:08 -08:00
Krrish Dholakia	a05722571b	fix(replicate.py): fix custom prompt formatting	2023-11-29 19:44:09 -08:00
ishaan-jaff	cf9f3d0005	(fix) router: azure/embedding support	2023-11-29 19:06:36 -08:00
ishaan-jaff	e03ae74387	(fix) router: set default rpm/tpm when not set	2023-11-29 18:13:27 -08:00
ishaan-jaff	e365aea138	(feat) router: random pick based on tpm/rpm	2023-11-29 17:54:06 -08:00
ishaan-jaff	001ff8d11e	(fix) use weighted shuffle when rpm set	2023-11-29 17:13:11 -08:00
ishaan-jaff	8be91e2590	(feat) proxy+ router: support 1k request/second	2023-11-29 16:22:04 -08:00
ishaan-jaff	f32661661e	(feat) proxy: add weighted shuffle + set cooldown to 1s	2023-11-29 16:09:31 -08:00
Krrish Dholakia	e05063a399	fix(router.py): skip api key when generating model id for router deployments	2023-11-29 15:37:08 -08:00
Krrish Dholakia	ceadb1547d	fix(main.py): passing client as a litellm-specific kwarg	2023-11-28 21:20:05 -08:00
ishaan-jaff	8a22afd677	(feat) router: init client for OpenAI compatible providers	2023-11-28 17:49:53 -08:00
Krrish Dholakia	60d6b6bc37	fix(router.py): fix exponential backoff to use retry-after if present in headers	2023-11-28 17:25:03 -08:00
ishaan-jaff	d55df1c9c8	(fix) router red api_key, api_base, api_version	2023-11-28 17:10:20 -08:00
ishaan-jaff	9d69ea5b12	(fix) router: passing client	2023-11-28 16:34:16 -08:00
ishaan-jaff	493573f4d2	(router) re use client across requests	2023-11-28 16:21:16 -08:00
ishaan-jaff	57d774f3ad	(feat) router: re-use the same client for high trafic	2023-11-28 15:44:56 -08:00
ishaan-jaff	7ec948c9b9	(feat) router track total, success, failed calls per model	2023-11-28 15:44:56 -08:00
Krrish Dholakia	32a14e6cf0	fix(router.py): removing model id before making call	2023-11-28 10:09:45 -08:00
Krrish Dholakia	ae268a5c49	refactor(router.py): fix linting errors	2023-11-27 22:11:53 -08:00
Krrish Dholakia	861c66580f	refactor(router.py): fix linting errors	2023-11-27 22:08:48 -08:00
Krrish Dholakia	624ef83455	build: adding debug logs to gitignore	2023-11-27 22:05:07 -08:00
Krrish Dholakia	91e8ff439d	fix(main.py): fix linting errors	2023-11-27 19:11:38 -08:00
ishaan-jaff	741e923127	(feat) use api_base, api_key as model	2023-11-27 18:08:47 -08:00
Krrish Dholakia	e564f643f5	fix(router.py): speed improvements to the router	2023-11-27 17:35:26 -08:00
ishaan-jaff	5257fc1f9c	(fix) router: allow same model/name	2023-11-27 16:26:09 -08:00
Krrish Dholakia	2680f84cfc	fix(router.py): fix fallbacks	2023-11-25 19:34:20 -08:00
Krrish Dholakia	67fe8824b3	fix(router.py): check for fallbacks in completion params for router	2023-11-25 18:46:45 -08:00
Krrish Dholakia	8884ceb606	fix(proxy_server.py): expose a /health endpoint	2023-11-25 18:28:47 -08:00
Krrish Dholakia	2b9c5bf706	fix(router.py): check if fallbacks is none	2023-11-25 14:58:07 -08:00
Krrish Dholakia	b1b2d0c2b7	fix(utils.py): fix bedrock + cohere calls	2023-11-25 14:45:42 -08:00
Krrish Dholakia	e421642ba8	fix: fix linting issues	2023-11-24 15:46:25 -08:00
Krrish Dholakia	68168cc743	fix(router.py): fix retry logic	2023-11-24 13:27:44 -08:00
Krrish Dholakia	9618718080	test: refactor testing order	2023-11-24 12:47:28 -08:00
Krrish Dholakia	27fd144950	docs(simple_proxy.md): add tutorial for doing fallbacks + retries + timeouts on the proxy	2023-11-24 12:20:38 -08:00
Krrish Dholakia	9185d57f6d	fix(router.py): fixing embedding call	2023-11-23 21:07:02 -08:00
Krrish Dholakia	f3bef86848	fix(router.py): use an older version of async for compatibility	2023-11-23 21:00:53 -08:00
Krrish Dholakia	3a8d7ec835	fix(router.py): add modelgroup to call metadata	2023-11-23 20:55:49 -08:00