456 commits

Author	SHA1	Message	Date
Krrish Dholakia	2f1c5aa0c7	fix: setting cache responses on proxy	2023-12-07 20:39:40 -08:00
Krrish Dholakia	f5afc429b3	fix(proxy_server.py): add call hooks pre+post completion and embedding calls	2023-12-07 20:35:32 -08:00
Krrish Dholakia	9cf3051ea2	feat(proxy_server.py): enable background health checks	2023-12-07 19:40:06 -08:00
ishaan-jaff	fd04b48764	(feat) async callbacks with litellm.completion()	2023-12-07 18:09:57 -08:00
ishaan-jaff	762f28e4d7	(fix) make print_verbose non blocking	2023-12-07 17:31:32 -08:00
Krrish Dholakia	e5638e2c5d	fix(router.py): fix default caching response value	2023-12-07 13:44:31 -08:00
ishaan-jaff	2bc583c2a6	(test) proxy - async custom logger	2023-12-07 13:19:17 -08:00
Krrish Dholakia	d77e0cc716	docs(config.md): adding docs on parallel request rate limiting	2023-12-07 11:27:48 -08:00
Krrish Dholakia	c7aaa4adf8	docs(deploy.md): add docker instructions to deploy docs	2023-12-07 09:22:54 -08:00
Krrish Dholakia	bd8d59e693	refactor(proxy_server.py): linting fix	2023-12-06 22:49:30 -08:00
Krrish Dholakia	c1e95740b0	fix(bedrock.py): fix output format for cohere embeddings	2023-12-06 22:47:01 -08:00
ishaan-jaff	fa70b1f85b	(test) unset model_group_alias_map after test	2023-12-06 20:35:14 -08:00
ishaan-jaff	900b8d66f3	(feat) proxy use model_group_alias_map	2023-12-06 20:23:24 -08:00
Krrish Dholakia	c0eedf28fc	test: fix proxy server testing	2023-12-06 18:38:53 -08:00
ishaan-jaff	19b1deb200	(feat) proxy: protect health endpoint	2023-12-06 18:14:54 -08:00
Krrish Dholakia	45b4140615	test: fix config import for proxy testing	2023-12-06 17:40:38 -08:00
Krrish Dholakia	d814184bc3	test: fix test imports	2023-12-06 17:21:47 -08:00
ishaan-jaff	b60dc20f4b	(fix) proxy edit custom logger	2023-12-06 17:16:24 -08:00
ishaan-jaff	1bac052eca	(fix) proxy use async logging	2023-12-06 17:16:24 -08:00
ishaan-jaff	dfb30d38fa	(feat) proxy print set callbacks	2023-12-06 17:16:24 -08:00
Krrish Dholakia	58848841e1	fix(proxy_server.py): make headers json serializable	2023-12-06 17:09:02 -08:00
Krrish Dholakia	ad922b205b	fix(proxy_server.py): enable rate limiting concurrent user requests	2023-12-06 15:11:05 -08:00
ishaan-jaff	44bf51601a	(feat) proxy - custom on failure callback	2023-12-06 14:43:47 -08:00
ishaan-jaff	3b17fd3821	(feat) proxy - async_on_fail_logger	2023-12-06 14:43:47 -08:00
ishaan-jaff	be15cf20b9	(chore) print verbose	2023-12-06 14:14:20 -08:00
ishaan-jaff	e1230627d0	(fix) print statements	2023-12-06 14:11:23 -08:00
ishaan-jaff	0598ab9b63	(fix) proxy /model/new writing to config	2023-12-06 14:11:23 -08:00
Krrish Dholakia	346551da29	fix(proxy_server.py): allow worker config to just be the config filepath	2023-12-06 14:03:25 -08:00
ishaan-jaff	368934d160	(feat) proxy: use async_callback function	2023-12-06 13:51:24 -08:00
Krrish Dholakia	d962d5d4c0	fix(bedrock.py): adding support for cohere embeddings	2023-12-06 13:25:18 -08:00
ishaan-jaff	cf6ecc03a5	(fix) linting	2023-12-06 13:14:26 -08:00
ishaan-jaff	f3c3a9860a	(feat) /v1/model/info	2023-12-06 13:03:29 -08:00
ishaan-jaff	06255c6590	(feat) proxy add ext-embedding-ada-002 as a base model	2023-12-06 12:19:47 -08:00
ishaan-jaff	29fb97f88a	(feat) proxy - define model info	2023-12-06 12:06:30 -08:00
Krrish Dholakia	102de97960	refactor: fix linting errors	2023-12-06 11:46:15 -08:00
ishaan-jaff	de58dcc016	(feat) proxy - allow setting cost, context window	2023-12-06 11:42:56 -08:00
ishaan-jaff	aefa4f36f9	(docs) update yaml with chat/embedding/completion mode	2023-12-06 11:36:16 -08:00
ishaan-jaff	fd86876164	(feat) proxy: add mode in model info	2023-12-06 11:29:59 -08:00
ishaan-jaff	7c77cc3cfa	(feat) add mode for config.yaml health checks	2023-12-06 11:16:29 -08:00
ishaan-jaff	1e2a8869a9	(docs) proxy config with azure, openai embedding models	2023-12-06 10:45:07 -08:00
ishaan-jaff	caf2a6b279	(fix) proxy - move new health check import	2023-12-06 10:13:06 -08:00
Ishaan Jaff	a4cf4e7ca9	Merge pull request #1023 from PSU3D0/speedup_health_endpoint ... (feat) Speedup health endpoint	2023-12-06 09:52:13 -08:00
Krrish Dholakia	92b2cbcdc5	feat(proxy_server.py): adding /model/delete endpoint	2023-12-05 22:38:38 -08:00
ishaan-jaff	8e6c4c5310	(fix) router - allow users to call a specific_model explicit	2023-12-05 21:57:00 -08:00
Frank Colson	fc31221b8a	Speedup health endpoint	2023-12-05 22:09:01 -07:00
ishaan-jaff	642c62f7b7	(fix) proxy: better debugging when -debug is on	2023-12-05 18:19:15 -08:00
ishaan-jaff	48aa00d6c0	(fix) proxy - clean up print statement	2023-12-05 18:14:01 -08:00
ishaan-jaff	27d7d7ba9c	(feat) proxy cli, better description of config yaml param	2023-12-05 18:11:29 -08:00
ishaan-jaff	155e99b9a3	(fix) prox cli: remove deprecated param	2023-12-05 18:04:08 -08:00
ishaan-jaff	cb52e3347e	(fix) proxy: make yaml load print_verbose	2023-12-05 18:00:00 -08:00