616 commits

Author	SHA1	Message	Date
ishaan-jaff	5efe336968	(fix) linting	2023-12-08 17:26:14 -08:00
Krrish Dholakia	5fa2b6e5ad	fix(proxy_server.py): enable pre+post-call hooks and max parallel request limits	2023-12-08 17:11:30 -08:00
ishaan-jaff	be94a8c478	(feat) pass model_info, proxy_server_request to callback	2023-12-08 14:26:18 -08:00
Krrish Dholakia	bab958561e	fix(proxy/utils.py): correctly raise exceptions on db call fail	2023-12-08 13:35:42 -08:00
Krrish Dholakia	9f0b83248a	fix(proxy/utils.py): add backoff/retry logic to db read/writes	2023-12-08 13:34:31 -08:00
Krrish Dholakia	f06c3e0f08	fix(proxy_server.py): initializing sentry in proxy logging before db init	2023-12-08 11:58:16 -08:00
Krrish Dholakia	7aec95ed7c	feat(proxy_server.py): add sentry logging for db read/writes	2023-12-08 11:40:19 -08:00
Krrish Dholakia	4e6a8d09d0	fix(proxy/utils.py): tracking db failed writes	2023-12-08 10:40:02 -08:00
Krrish Dholakia	1b35736797	fix(utils.py): fix cost calculation to handle tool input	2023-12-08 09:53:46 -08:00
Krrish Dholakia	9494c2cd9e	fix(requirements.txt): add ollama async_generator in requirements.txt	2023-12-08 09:20:04 -08:00
Krrish Dholakia	2f1c5aa0c7	fix: setting cache responses on proxy	2023-12-07 20:39:40 -08:00
Krrish Dholakia	f5afc429b3	fix(proxy_server.py): add call hooks pre+post completion and embedding calls	2023-12-07 20:35:32 -08:00
Krrish Dholakia	9cf3051ea2	feat(proxy_server.py): enable background health checks	2023-12-07 19:40:06 -08:00
ishaan-jaff	fd04b48764	(feat) async callbacks with litellm.completion()	2023-12-07 18:09:57 -08:00
ishaan-jaff	762f28e4d7	(fix) make print_verbose non blocking	2023-12-07 17:31:32 -08:00
Krrish Dholakia	e5638e2c5d	fix(router.py): fix default caching response value	2023-12-07 13:44:31 -08:00
ishaan-jaff	2bc583c2a6	(test) proxy - async custom logger	2023-12-07 13:19:17 -08:00
Krrish Dholakia	d77e0cc716	docs(config.md): adding docs on parallel request rate limiting	2023-12-07 11:27:48 -08:00
Krrish Dholakia	c7aaa4adf8	docs(deploy.md): add docker instructions to deploy docs	2023-12-07 09:22:54 -08:00
Krrish Dholakia	bd8d59e693	refactor(proxy_server.py): linting fix	2023-12-06 22:49:30 -08:00
Krrish Dholakia	c1e95740b0	fix(bedrock.py): fix output format for cohere embeddings	2023-12-06 22:47:01 -08:00
ishaan-jaff	fa70b1f85b	(test) unset model_group_alias_map after test	2023-12-06 20:35:14 -08:00
ishaan-jaff	900b8d66f3	(feat) proxy use model_group_alias_map	2023-12-06 20:23:24 -08:00
Krrish Dholakia	c0eedf28fc	test: fix proxy server testing	2023-12-06 18:38:53 -08:00
ishaan-jaff	19b1deb200	(feat) proxy: protect health endpoint	2023-12-06 18:14:54 -08:00
Krrish Dholakia	45b4140615	test: fix config import for proxy testing	2023-12-06 17:40:38 -08:00
Krrish Dholakia	d814184bc3	test: fix test imports	2023-12-06 17:21:47 -08:00
ishaan-jaff	b60dc20f4b	(fix) proxy edit custom logger	2023-12-06 17:16:24 -08:00
ishaan-jaff	1bac052eca	(fix) proxy use async logging	2023-12-06 17:16:24 -08:00
ishaan-jaff	dfb30d38fa	(feat) proxy print set callbacks	2023-12-06 17:16:24 -08:00
Krrish Dholakia	58848841e1	fix(proxy_server.py): make headers json serializable	2023-12-06 17:09:02 -08:00
Krrish Dholakia	ad922b205b	fix(proxy_server.py): enable rate limiting concurrent user requests	2023-12-06 15:11:05 -08:00
ishaan-jaff	44bf51601a	(feat) proxy - custom on failure callback	2023-12-06 14:43:47 -08:00
ishaan-jaff	3b17fd3821	(feat) proxy - async_on_fail_logger	2023-12-06 14:43:47 -08:00
ishaan-jaff	be15cf20b9	(chore) print verbose	2023-12-06 14:14:20 -08:00
ishaan-jaff	e1230627d0	(fix) print statements	2023-12-06 14:11:23 -08:00
ishaan-jaff	0598ab9b63	(fix) proxy /model/new writing to config	2023-12-06 14:11:23 -08:00
Krrish Dholakia	346551da29	fix(proxy_server.py): allow worker config to just be the config filepath	2023-12-06 14:03:25 -08:00
ishaan-jaff	368934d160	(feat) proxy: use async_callback function	2023-12-06 13:51:24 -08:00
Krrish Dholakia	d962d5d4c0	fix(bedrock.py): adding support for cohere embeddings	2023-12-06 13:25:18 -08:00
ishaan-jaff	cf6ecc03a5	(fix) linting	2023-12-06 13:14:26 -08:00
ishaan-jaff	f3c3a9860a	(feat) /v1/model/info	2023-12-06 13:03:29 -08:00
ishaan-jaff	06255c6590	(feat) proxy add ext-embedding-ada-002 as a base model	2023-12-06 12:19:47 -08:00
ishaan-jaff	29fb97f88a	(feat) proxy - define model info	2023-12-06 12:06:30 -08:00
Krrish Dholakia	102de97960	refactor: fix linting errors	2023-12-06 11:46:15 -08:00
ishaan-jaff	de58dcc016	(feat) proxy - allow setting cost, context window	2023-12-06 11:42:56 -08:00
ishaan-jaff	aefa4f36f9	(docs) update yaml with chat/embedding/completion mode	2023-12-06 11:36:16 -08:00
ishaan-jaff	fd86876164	(feat) proxy: add mode in model info	2023-12-06 11:29:59 -08:00
ishaan-jaff	7c77cc3cfa	(feat) add mode for config.yaml health checks	2023-12-06 11:16:29 -08:00
ishaan-jaff	1e2a8869a9	(docs) proxy config with azure, openai embedding models	2023-12-06 10:45:07 -08:00