253 commits

Author	SHA1	Message	Date
Krrish Dholakia	d77e0cc716	docs(config.md): adding docs on parallel request rate limiting	2023-12-07 11:27:48 -08:00
Krrish Dholakia	bd8d59e693	refactor(proxy_server.py): linting fix	2023-12-06 22:49:30 -08:00
Krrish Dholakia	c1e95740b0	fix(bedrock.py): fix output format for cohere embeddings	2023-12-06 22:47:01 -08:00
ishaan-jaff	fa70b1f85b	(test) unset model_group_alias_map after test	2023-12-06 20:35:14 -08:00
ishaan-jaff	900b8d66f3	(feat) proxy use model_group_alias_map	2023-12-06 20:23:24 -08:00
Krrish Dholakia	c0eedf28fc	test: fix proxy server testing	2023-12-06 18:38:53 -08:00
ishaan-jaff	19b1deb200	(feat) proxy: protect health endpoint	2023-12-06 18:14:54 -08:00
Krrish Dholakia	45b4140615	test: fix config import for proxy testing	2023-12-06 17:40:38 -08:00
ishaan-jaff	dfb30d38fa	(feat) proxy print set callbacks	2023-12-06 17:16:24 -08:00
Krrish Dholakia	58848841e1	fix(proxy_server.py): make headers json serializable	2023-12-06 17:09:02 -08:00
Krrish Dholakia	ad922b205b	fix(proxy_server.py): enable rate limiting concurrent user requests	2023-12-06 15:11:05 -08:00
ishaan-jaff	44bf51601a	(feat) proxy - custom on failure callback	2023-12-06 14:43:47 -08:00
ishaan-jaff	be15cf20b9	(chore) print verbose	2023-12-06 14:14:20 -08:00
ishaan-jaff	e1230627d0	(fix) print statements	2023-12-06 14:11:23 -08:00
ishaan-jaff	0598ab9b63	(fix) proxy /model/new writing to config	2023-12-06 14:11:23 -08:00
Krrish Dholakia	346551da29	fix(proxy_server.py): allow worker config to just be the config filepath	2023-12-06 14:03:25 -08:00
ishaan-jaff	368934d160	(feat) proxy: use async_callback function	2023-12-06 13:51:24 -08:00
Krrish Dholakia	d962d5d4c0	fix(bedrock.py): adding support for cohere embeddings	2023-12-06 13:25:18 -08:00
ishaan-jaff	cf6ecc03a5	(fix) linting	2023-12-06 13:14:26 -08:00
ishaan-jaff	f3c3a9860a	(feat) /v1/model/info	2023-12-06 13:03:29 -08:00
Krrish Dholakia	102de97960	refactor: fix linting errors	2023-12-06 11:46:15 -08:00
ishaan-jaff	caf2a6b279	(fix) proxy - move new health check import	2023-12-06 10:13:06 -08:00
Ishaan Jaff	a4cf4e7ca9	Merge pull request #1023 from PSU3D0/speedup_health_endpoint ... (feat) Speedup health endpoint	2023-12-06 09:52:13 -08:00
Krrish Dholakia	92b2cbcdc5	feat(proxy_server.py): adding /model/delete endpoint	2023-12-05 22:38:38 -08:00
ishaan-jaff	8e6c4c5310	(fix) router - allow users to call a specific_model explicit	2023-12-05 21:57:00 -08:00
Frank Colson	fc31221b8a	Speedup health endpoint	2023-12-05 22:09:01 -07:00
ishaan-jaff	642c62f7b7	(fix) proxy: better debugging when -debug is on	2023-12-05 18:19:15 -08:00
ishaan-jaff	48aa00d6c0	(fix) proxy - clean up print statement	2023-12-05 18:14:01 -08:00
ishaan-jaff	cb52e3347e	(fix) proxy: make yaml load print_verbose	2023-12-05 18:00:00 -08:00
ishaan-jaff	4d5313343b	(feat) proxy /embedding check 1 deploy call	2023-12-05 17:22:07 -08:00
ishaan-jaff	3af4f7fb0f	(fix) proxy: /chat/cmp - check 1 deployment	2023-12-05 17:19:48 -08:00
ishaan-jaff	d2dab362df	(fix) proxy debugging display Init API key	2023-12-05 16:08:17 -08:00
Krrish Dholakia	4d7ff1b33b	fix(proxy_server.py): don't override exceptions if they're of type httpexception	2023-12-05 14:33:28 -08:00
Krrish Dholakia	88c95ca259	fix(_redis.py): support additional params for redis	2023-12-05 12:16:51 -08:00
Krrish Dholakia	ddea62fdb1	refactor(proxy_server.py): clean up print statements in proxy server	2023-12-05 11:14:16 -08:00
Krrish Dholakia	b46c73a46e	fix: fix proxy testing	2023-12-05 11:13:09 -08:00
Krrish Dholakia	030bd22078	feat(proxy_server.py): allow user to override api key auth	2023-12-04 18:32:58 -08:00
Krrish Dholakia	e0ccb281d8	feat(utils.py): add async success callbacks for custom functions	2023-12-04 16:42:40 -08:00
ishaan-jaff	4ef0378e6e	(fix) proxy: custom callbacks	2023-12-04 11:29:26 -08:00
ishaan-jaff	e96a608938	(feat) proxy: set custom headers in metadata	2023-12-04 10:49:01 -08:00
Krrish Dholakia	63e55f1865	fix(proxy_server.py): fix /key/generate post endpoint	2023-12-04 10:44:13 -08:00
ishaan-jaff	d7d8c5f6e6	(fix) proxy	2023-12-04 10:23:55 -08:00
ishaan-jaff	88cec3b9ab	(fix) proxy: don't overwrite user	2023-12-04 10:17:56 -08:00
ishaan-jaff	65e00b438e	(feat) proxy-read litellm custom callback class	2023-12-04 09:09:54 -08:00
ishaan-jaff	ed8c666922	(feat) proxy: custom_logger for I/O logging	2023-12-04 08:44:20 -08:00
Krrish Dholakia	add4dfc528	fix(proxy_server.py): support model info augmenting for azure models	2023-12-02 21:33:54 -08:00
ishaan-jaff	ecddb852a2	(fix) proxy: pydantic error / warning message	2023-12-02 21:30:35 -08:00
Krrish Dholakia	47c1aa27c7	fix(proxy_server.py): add litellm model cost map info to /model/info	2023-12-02 21:24:28 -08:00
Krrish Dholakia	f0d8a87c48	fix(proxy_server.py): fix user param name sent to litellm.completion	2023-12-02 19:56:22 -08:00
Krrish Dholakia	5200818af1	fix(proxy_server.py): Passing user IDs to OpenAI to identify abusive virtual keys	2023-12-02 19:55:11 -08:00