24 commits

Author	SHA1	Message	Date
Krrish Dholakia	d2f47ee45b	fix(parallel_request_limiter.py): handle metadata being none	2024-03-14 10:02:41 -07:00
Krrish Dholakia	c963e2761b	feat(proxy_server.py): retry if virtual key is rate limited ... currently for chat completions	2024-03-05 19:00:03 -08:00
Krrish Dholakia	f72b84f6e0	fix(parallel_request_limiter.py): handle none scenario	2024-02-26 20:09:06 -08:00
Krrish Dholakia	7fff5119de	fix(parallel_request_limiter.py): fix team rate limit enforcement	2024-02-26 18:06:13 -08:00
Krrish Dholakia	5213fd2e1e	feat(parallel_request_limiter.py): enforce team based tpm / rpm limits	2024-02-26 16:20:41 -08:00
ishaan-jaff	5ec69a0ca5	(fix) failing parallel_Request_limiter test	2024-02-22 19:16:22 -08:00
ishaan-jaff	b728ded300	(fix) don't double check curr data and time	2024-02-22 18:50:02 -08:00
ishaan-jaff	74d66d5ac5	(feat) tpm/rpm limit by User	2024-02-22 18:44:03 -08:00
Krrish Dholakia	07aa05bf17	fix(test_parallel_request_limiter.py): use mock responses for streaming	2024-02-08 21:45:38 -08:00
ishaan-jaff	c8b2f0fd5d	(fix) parallel_request_limiter debug	2024-02-06 12:43:28 -08:00
Krrish Dholakia	dbf2b0b2c8	fix(utils.py): override default success callbacks with dynamic callbacks if set	2024-02-02 06:21:43 -08:00
Krrish Dholakia	c91ab81fde	fix(test_parallel_request_limiter): increase time limit for waiting for success logging event to happen	2024-01-30 13:26:17 -08:00
Krrish Dholakia	e957f41ab7	fix(utils.py): add metadata to logging obj on setup, if exists	2024-01-19 17:29:47 -08:00
Krrish Dholakia	f73a4ae7c2	fix(parallel_request_limiter.py): handle tpm/rpm limits being null	2024-01-19 10:22:27 -08:00
Krrish Dholakia	34c3b33b37	test(test_parallel_request_limiter.py): unit testing for tpm/rpm rate limits	2024-01-18 15:28:28 -08:00
Krrish Dholakia	13b013b28d	feat(parallel_request_limiter.py): add support for tpm/rpm limits	2024-01-18 13:52:15 -08:00
Krrish Dholakia	44553bcc3a	fix(parallel_request_limiter.py): decrement count for failed llm calls ... https://github.com/BerriAI/litellm/issues/1477	2024-01-18 12:42:14 -08:00
Krrish Dholakia	79978c44ba	refactor: add black formatting	2023-12-25 14:11:20 +05:30
Krrish Dholakia	018405b956	fix(proxy/utils.py): return different exceptions if key is invalid vs. expired ... https://github.com/BerriAI/litellm/issues/1230	2023-12-25 10:29:44 +05:30
Krrish Dholakia	72e8c84914	build(test_streaming.py): fix linting issues	2023-12-25 07:34:54 +05:30
Krrish Dholakia	1da7d35218	feat(proxy_server.py): enable infinite retries on rate limited requests	2023-12-15 20:03:41 -08:00
Krrish Dholakia	3fbeca134f	fix(custom_logger.py): enable pre_call hooks to modify incoming data to proxy	2023-12-13 16:20:37 -08:00
Krrish Dholakia	8eb7dc6393	fix(proxy_server.py): support for streaming	2023-12-09 16:23:04 -08:00
Krrish Dholakia	9c6584a376	fix(proxy_server.py): enable pre+post-call hooks and max parallel request limits	2023-12-08 17:11:30 -08:00