621 commits

Author	SHA1	Message	Date
Krrish Dholakia	062ede96e3	refactor(proxy_server.py): fix linting issues	2023-11-30 09:24:59 -08:00
Krrish Dholakia	b4b7acdb72	fix(utils.py): fix azure completion cost calculation	2023-11-30 09:19:35 -08:00
Krrish Dholakia	7ee089b5ca	fix(proxy_server.py): provide an endpoint that gives model-specific info from proxy	2023-11-30 09:08:19 -08:00
ishaan-jaff	a56d4a1e83	(fix) proxy: print cwd()	2023-11-30 08:52:06 -08:00
ishaan-jaff	ecdc5bdad6	(dos) config.yaml	2023-11-30 08:34:36 -08:00
ishaan-jaff	e5ce45dc2c	(cleanup) proxy/health	2023-11-29 20:15:52 -08:00
Krrish Dholakia	50cc4a8595	fix(proxy_server.py): have /health and /routes be router endpoints	2023-11-29 19:59:56 -08:00
ishaan-jaff	3cc8305ec6	(fix) proxy: /health	2023-11-29 16:23:37 -08:00
ishaan-jaff	d3672452ce	(test) 1k requests	2023-11-29 16:22:18 -08:00
ishaan-jaff	66bc0fc343	(fix) proxy: /health works with router updates	2023-11-29 16:09:31 -08:00
ishaan-jaff	f307e82a41	(fix) proxy: making receiving data print_verbose	2023-11-29 07:50:52 -08:00
Krrish Dholakia	2b06fea4a8	fix(proxy_server.py): ensure /models returns unique model names	2023-11-28 17:32:20 -08:00
ishaan-jaff	ee6f5a84db	(test) load test completion	2023-11-28 15:44:56 -08:00
ishaan-jaff	ae7f0ae0b6	(feat) proxy: add logs on router performance	2023-11-28 15:44:56 -08:00
Krrish Dholakia	4ea52dd571	fix(proxy_server.py): support reading master key from os environment	2023-11-28 14:05:17 -08:00
ishaan-jaff	3ca4487e77	(feat) proxy set num_retries=3	2023-11-27 19:33:59 -08:00
ishaan-jaff	40d9e8ab23	(test) load test	2023-11-27 18:08:47 -08:00
ishaan-jaff	8560794963	(test) load test router	2023-11-27 16:37:57 -08:00
ishaan-jaff	ba228a9e0a	(fix) proxy set litellm attributes	2023-11-27 13:39:18 -08:00
ishaan-jaff	5e2c13fb11	(test) load test proxy completion	2023-11-27 12:13:21 -08:00
ishaan-jaff	9747cc5aad	(feat) --health for checking config models	2023-11-27 12:13:21 -08:00
Krrish Dholakia	56bb39e52c	fix(acompletion): fix acompletion raise exception issue when custom llm provider is none	2023-11-27 11:34:48 -08:00
Krrish Dholakia	aafba24e84	fix(proxy_server.py): fix user model returned in /models	2023-11-27 08:04:49 -08:00
Krrish Dholakia	e4f302a8e2	fix(proxy_server.py): expose a /health endpoint	2023-11-25 18:28:47 -08:00
ishaan-jaff	a688df79b1	(feat) proxy: make chat/completions async	2023-11-25 12:54:03 -08:00
ishaan-jaff	dca0a5ad0f	(test) load test embedding: proxy	2023-11-24 17:14:44 -08:00
ishaan-jaff	111c7afaca	(docs) proxy performance	2023-11-24 17:07:46 -08:00
Krrish Dholakia	d62da29cbe	fix: fix linting issues	2023-11-24 15:46:25 -08:00
Krrish Dholakia	bc84b38154	feat(proxy_server.py): new /key/info endpoint to access key information (master key only)	2023-11-24 15:24:50 -08:00
Krrish Dholakia	4f22e7de18	feat(proxy_server.py): tracking spend per api key	2023-11-24 15:14:06 -08:00
Krrish Dholakia	16e1070dbe	test: refactor testing order	2023-11-24 12:47:28 -08:00
Krrish Dholakia	2a033fd8a2	test(test_router_cooldowns.py): adding logging	2023-11-24 12:30:08 -08:00
Krrish Dholakia	2e8d582a34	fix(proxy_server.py): fix linting issues	2023-11-24 11:39:01 -08:00
ishaan-jaff	8edfcd8e5d	(fix) prisma using: secrets.compare_digest	2023-11-24 10:02:08 -08:00
David Manouchehri	ac08e3616c	Fix timing attack on master_key.	2023-11-24 12:12:29 -05:00
David Manouchehri	5b6f227170	Fix master key check.	2023-11-24 12:03:30 -05:00
David Manouchehri	3fa3a767b3	Fix OpenAPI auth spec.	2023-11-24 11:59:33 -05:00
ishaan-jaff	0a23af7990	(feat) proxy: cost tracking add track_cost_callback	2023-11-24 05:53:23 -08:00
Kumaran Rajendhiran	01fad94485	Fail gracefully if ollama is already being served	2023-11-24 16:52:55 +05:30
Krrish Dholakia	c9e37b8fb8	fix(proxy_server.py): fix linting errors	2023-11-23 21:42:39 -08:00
Krrish Dholakia	9a44433844	feat(proxy_server.py): /key/delete endpoint	2023-11-23 21:37:53 -08:00
ishaan-jaff	486037ff8b	(fix) proxy: /embeddings	2023-11-23 21:16:51 -08:00
ishaan-jaff	31c5b4572b	(fix) proxy: prisma.schema	2023-11-23 20:11:42 -08:00
Krrish Dholakia	c273d6f0d6	fix(router.py): add support for context window fallbacks on router	2023-11-23 16:43:02 -08:00
ishaan-jaff	9b53ea4b0f	(feat) proxy: cost tracking per completion request	2023-11-23 16:08:59 -08:00
Krrish Dholakia	8c4e8d6c62	feat(proxy_server.py): add in-memory caching for user api keys	2023-11-23 13:21:45 -08:00
ishaan-jaff	8291f239a4	(feat) proxy server: add spend column	2023-11-23 11:46:59 -08:00
Krrish Dholakia	61fc76a8c4	fix(router.py): fix caching for tracking cooldowns + usage	2023-11-23 11:13:32 -08:00
ishaan-jaff	9d3d10da4c	(Feat) update config.yaml example on proxy	2023-11-23 10:54:30 -08:00
ishaan-jaff	1dcab433b4	(feat) proxy: caching - show redis settings when initializing	2023-11-23 10:52:50 -08:00