23 commits

Author	SHA1	Message	Date
Ishaan Jaff	94e74b9ede	inly write model tpm/rpm tracking when user set it	2024-08-18 09:58:09 -07:00
Ishaan Jaff	45cc7291f5	fix parallel request limiter tests	2024-08-17 14:21:59 -07:00
Ishaan Jaff	cf807d1541	fix parallel request limiter	2024-08-17 14:14:12 -07:00
Ishaan Jaff	8ae626b31f	feat add settings for rpm/tpm limits for a model	2024-08-17 09:16:01 -07:00
Ishaan Jaff	824ea32452	track rpm/tpm usage per key+model	2024-08-16 18:28:58 -07:00
Krrish Dholakia	17635450cd	feat(proxy_server.py): return 'retry-after' param for rate limited requests ... Closes https://github.com/BerriAI/litellm/issues/4695	2024-07-13 17:15:20 -07:00
Krrish Dholakia	b8c6f3dd61	test(test_parallel_request_limiter.py): fix test	2024-06-13 17:13:44 -07:00
Krrish Dholakia	56fd0c60d1	fix(parallel_request_limiter.py): fix user+team tpm/rpm limit check ... Closes https://github.com/BerriAI/litellm/issues/3788	2024-05-27 08:48:23 -07:00
Krrish Dholakia	3f339cb694	fix(parallel_request_limiter.py): fix max parallel request limiter on retries	2024-05-15 20:16:11 -07:00
Krrish Dholakia	29368b5d71	test: refactor testing to handle the hash token fix	2024-04-17 17:31:39 -07:00
Krrish Dholakia	6346839574	test(test_parallel_request_limiter.py): add more verbose logging	2024-03-06 19:21:57 -08:00
Krrish Dholakia	7fff5119de	fix(parallel_request_limiter.py): fix team rate limit enforcement	2024-02-26 18:06:13 -08:00
ishaan-jaff	74d66d5ac5	(feat) tpm/rpm limit by User	2024-02-22 18:44:03 -08:00
Krrish Dholakia	07aa05bf17	fix(test_parallel_request_limiter.py): use mock responses for streaming	2024-02-08 21:45:38 -08:00
ishaan-jaff	f33815aaf8	(fix) test_normal_router_tpm_limit	2024-02-06 12:44:30 -08:00
ishaan-jaff	de4d36cf96	(fix) test_normal_router_tpm_limit	2024-02-06 06:46:49 -08:00
ishaan-jaff	f9916cc6e4	(fix) test_parallel limiter fix	2024-02-03 21:03:15 -08:00
Krrish Dholakia	6bdb081e2c	test(test_parallel_request_limiter.py): fix test to handle minute changes	2024-02-03 20:39:31 -08:00
Krrish Dholakia	0302e4b27d	test(test_parallel_request_limiter.py): fix test import	2024-02-02 10:15:33 -08:00
Krrish Dholakia	c91ab81fde	fix(test_parallel_request_limiter): increase time limit for waiting for success logging event to happen	2024-01-30 13:26:17 -08:00
Krrish Dholakia	3957a8303a	test(test_parallel_request_limiter.py): unit testing for tpm/rpm rate limits	2024-01-18 15:25:29 -08:00
Krrish Dholakia	13b013b28d	feat(parallel_request_limiter.py): add support for tpm/rpm limits	2024-01-18 13:52:15 -08:00
Krrish Dholakia	44553bcc3a	fix(parallel_request_limiter.py): decrement count for failed llm calls ... https://github.com/BerriAI/litellm/issues/1477	2024-01-18 12:42:14 -08:00