105 commits

Author	SHA1	Message	Date
Ishaan Jaff	d8788bbe9b	test - router re-use openai client	2024-04-06 11:33:17 -07:00
Krrish Dholakia	a7da061ed8	test(test_router.py): fix casting	2024-04-04 13:54:16 -07:00
Krrish Dholakia	99e9b29526	test(test_router.py): fix test to check cast	2024-04-04 13:32:50 -07:00
Krrish Dholakia	0294c3f8a9	test(test_router.py): fix test to check type	2024-04-04 11:45:12 -07:00
Krrish Dholakia	129bb52e9d	fix(proxy_server.py): persist models added via /model/new to db ... allows models to be used across instances https://github.com/BerriAI/litellm/issues/2319 , https://github.com/BerriAI/litellm/issues/2329	2024-04-03 20:16:41 -07:00
Krrish Dholakia	b2b8375987	fix(router.py): support context window fallbacks for pre-call checks	2024-04-01 10:51:54 -07:00
Ishaan Jaff	11c922a966	(fix) test aimg gen on router	2024-03-28 12:27:26 -07:00
Krrish Dholakia	00d27a324d	fix(router.py): check for context window error when handling 400 status code errors ... was causing proxy context window fallbacks to not work as expected	2024-03-26 08:08:15 -07:00
Krrish Dholakia	8c6402b02d	docs(routing.md): add pre-call checks to docs	2024-03-23 19:10:34 -07:00
Krrish Dholakia	292cdd81e4	fix(router.py): fix pre call check logic	2024-03-23 18:56:08 -07:00
Krrish Dholakia	4e70a3e09a	feat(router.py): enable pre-call checks ... filter models outside of context window limits of a given message for a model group https://github.com/BerriAI/litellm/issues/872	2024-03-23 18:03:30 -07:00
Krrish Dholakia	dad65ca602	fix(bedrock.py): support anthropic messages api on bedrock (claude-3)	2024-03-04 17:15:47 -08:00
ishaan-jaff	b5fcc89682	(test) claude-instant-1	2024-03-04 08:32:13 -08:00
Krrish Dholakia	129e84f0c5	test: removing aws tests - account suspended - pending their approval	2024-02-28 13:46:20 -08:00
ishaan-jaff	6c1c2e8c7d	(feat) add moderation on router	2024-02-14 11:00:09 -08:00
ishaan-jaff	b7c458f25d	(ci/cd) add more logging to timeout test	2024-01-23 18:39:19 -08:00
ishaan-jaff	7276849499	(test) dynamic timeouts - router	2024-01-23 13:27:49 -08:00
Krrish Dholakia	ea9bb27e52	test(test_router.py): add more testing for dynamically passing params to router	2024-01-23 10:31:49 -08:00
ishaan-jaff	9aa40c63ee	(test) router.py revert to stable version	2024-01-23 09:05:19 -08:00
Krrish Dholakia	0b794cbd8f	test(test_router.py): fix test check	2024-01-23 07:07:30 -08:00
Krrish Dholakia	2912c3dcbb	fix(router.py): ensure no unsupported args are passed to completion()	2024-01-22 22:33:06 -08:00
Krrish Dholakia	9aaec44dff	fix(router.py): fix client init	2024-01-22 22:15:39 -08:00
Krrish Dholakia	253842bc7a	test(test_router.py): handle rate limiting error	2024-01-11 00:00:17 +05:30
Krrish Dholakia	4e719850bc	test(test_router.py): handle image gen timeouts	2024-01-10 20:56:52 +05:30
Krrish Dholakia	11ff324ff7	test(test_router.py): fix router test	2024-01-09 11:08:35 +05:30
Krrish Dholakia	1ed96b1fc8	test(test_router.py): add retries	2023-12-30 15:54:46 +05:30
Krrish Dholakia	2cea8b0e83	fix(router.py): periodically re-initialize azure/openai clients to solve max conn issue	2023-12-30 15:48:34 +05:30
Krrish Dholakia	79978c44ba	refactor: add black formatting	2023-12-25 14:11:20 +05:30
Krrish Dholakia	158325497d	feat(proxy_server.py): add new images/generation endpoint	2023-12-21 15:39:09 +05:30
Krrish Dholakia	c084f04a35	fix(router.py): add support for async image generation endpoints	2023-12-21 14:38:44 +05:30
Krrish Dholakia	b873833340	fix(router.py): only do sync image gen fallbacks for now ... The customhttptransport we use for dall-e-2 only works for sync httpx calls, not async. Will need to spend some time writing the async version n	2023-12-20 19:10:59 +05:30
Krrish Dholakia	89b808d767	feat(router.py): support async image generation on router	2023-12-20 17:24:20 +05:30
Krrish Dholakia	0a20b92c12	refactor(test_router_caching.py): add tests for router caching	2023-12-15 20:38:51 -08:00
Krrish Dholakia	6081602bf8	fix(router.py): support openai-compatible endpoints	2023-12-15 14:47:54 -08:00
ishaan-jaff	b8d51ee3ab	(test) router - acompletion + cache	2023-12-14 19:46:56 +05:30
Krrish Dholakia	fa74447cc8	test: testing fixes	2023-12-12 10:57:51 -08:00
Krrish Dholakia	bbf094dcf5	fix(router.py): reset caching correctly	2023-12-11 19:57:34 -08:00
ishaan-jaff	96c0f1495b	(test) router + caching	2023-12-09 15:41:36 -08:00
ishaan-jaff	db750475c3	(test) async completion + caching on router	2023-12-09 14:25:38 -08:00
Krrish Dholakia	f5dbddef39	test(test_router.py): fix init issue	2023-12-09 12:28:51 -08:00
Krrish Dholakia	3f045501c2	test(test_router.py): run as asyncio.run	2023-12-09 12:28:28 -08:00
Krrish Dholakia	a65c8919fc	fix(router.py): fix least-busy routing	2023-12-08 20:29:49 -08:00
Krrish Dholakia	461788636f	test: test_router.py ... fix router testing n	2023-12-07 20:10:36 -08:00
ishaan-jaff	f5d9eb773e	(test) router - read os.environ/ OpenAI	2023-12-06 10:56:27 -08:00
ishaan-jaff	dd474dfb5a	(chore) linting fix	2023-12-06 10:48:01 -08:00
ishaan-jaff	52a95a755e	(test) router - set sync stream client	2023-12-06 10:08:37 -08:00
ishaan-jaff	179db2a1a7	(test) router - reading os.environ/ with client	2023-12-06 09:26:21 -08:00
ishaan-jaff	7b88420223	(test) router - reading os.environ/ variables	2023-12-06 09:19:51 -08:00
ishaan-jaff	1fd1396f06	(test) router - explcitly call one deployment	2023-12-05 21:57:00 -08:00
ishaan-jaff	e851892dcb	(test) router: call 1 deployment	2023-12-05 17:35:35 -08:00