24 commits

Author	SHA1	Message	Date
Ishaan Jaff	3966c545b4	fix flaky tests	2024-08-27 22:47:11 -07:00
Ishaan Jaff	398295116f	inly write model tpm/rpm tracking when user set it	2024-08-18 09:58:09 -07:00
Ishaan Jaff	653d2e6ce0	fix parallel request limiter tests	2024-08-17 14:21:59 -07:00
Ishaan Jaff	221e5b829b	fix parallel request limiter	2024-08-17 14:14:12 -07:00
Ishaan Jaff	785482f023	feat add settings for rpm/tpm limits for a model	2024-08-17 09:16:01 -07:00
Ishaan Jaff	1ee33478c9	track rpm/tpm usage per key+model	2024-08-16 18:28:58 -07:00
Krrish Dholakia	fde434be66	feat(proxy_server.py): return 'retry-after' param for rate limited requests ... Closes https://github.com/BerriAI/litellm/issues/4695	2024-07-13 17:15:20 -07:00
Krrish Dholakia	8d56f72d5a	test(test_parallel_request_limiter.py): fix test	2024-06-13 17:13:44 -07:00
Krrish Dholakia	4408b717f0	fix(parallel_request_limiter.py): fix user+team tpm/rpm limit check ... Closes https://github.com/BerriAI/litellm/issues/3788	2024-05-27 08:48:23 -07:00
Krrish Dholakia	594ca947c8	fix(parallel_request_limiter.py): fix max parallel request limiter on retries	2024-05-15 20:16:11 -07:00
Krrish Dholakia	473e667bdf	test: refactor testing to handle the hash token fix	2024-04-17 17:31:39 -07:00
Krrish Dholakia	2163e43b9a	test(test_parallel_request_limiter.py): add more verbose logging	2024-03-06 19:21:57 -08:00
Krrish Dholakia	f86ab19067	fix(parallel_request_limiter.py): fix team rate limit enforcement	2024-02-26 18:06:13 -08:00
ishaan-jaff	b5900099af	(feat) tpm/rpm limit by User	2024-02-22 18:44:03 -08:00
Krrish Dholakia	b9393fb769	fix(test_parallel_request_limiter.py): use mock responses for streaming	2024-02-08 21:45:38 -08:00
ishaan-jaff	1489fd369b	(fix) test_normal_router_tpm_limit	2024-02-06 12:44:30 -08:00
ishaan-jaff	47bed68c7f	(fix) test_normal_router_tpm_limit	2024-02-06 06:46:49 -08:00
ishaan-jaff	c353161456	(fix) test_parallel limiter fix	2024-02-03 21:03:15 -08:00
Krrish Dholakia	3e35041758	test(test_parallel_request_limiter.py): fix test to handle minute changes	2024-02-03 20:39:31 -08:00
Krrish Dholakia	b8c2cbdd08	test(test_parallel_request_limiter.py): fix test import	2024-02-02 10:15:33 -08:00
Krrish Dholakia	bbe71c8375	fix(test_parallel_request_limiter): increase time limit for waiting for success logging event to happen	2024-01-30 13:26:17 -08:00
Krrish Dholakia	22c3321208	test(test_parallel_request_limiter.py): unit testing for tpm/rpm rate limits	2024-01-18 15:25:29 -08:00
Krrish Dholakia	aef59c554f	feat(parallel_request_limiter.py): add support for tpm/rpm limits	2024-01-18 13:52:15 -08:00
Krrish Dholakia	1ea3833ef7	fix(parallel_request_limiter.py): decrement count for failed llm calls ... https://github.com/BerriAI/litellm/issues/1477	2024-01-18 12:42:14 -08:00