13 commits

Author	SHA1	Message	Date
Ishaan Jaff	4acb76d7c5	sleep before checi g	2024-08-17 19:50:37 -07:00
Krrish Dholakia	8e4b014f3a	test(test_least_busy_routing.py): avoid deployments with low rate limits	2024-05-09 13:54:24 -07:00
Krrish Dholakia	d1e58193d0	test: replace flaky endpoint	2024-04-27 16:37:09 -07:00
Krrish Dholakia	48a5948081	fix(router.py): handle id being passed in as int	2024-04-04 14:23:10 -07:00
Krrish Dholakia	30c9c91520	test(test_least_busy_routing.py): fix test	2023-12-30 16:12:52 +05:30
Krrish Dholakia	11b039193a	test(test_least_busy_routing.py): fix test init	2023-12-30 12:39:13 +05:30
Krrish Dholakia	e1925d0e29	fix(router.py): support retry and fallbacks for atext_completion	2023-12-30 11:19:32 +05:30
Krrish Dholakia	a11940f4eb	fix(router.py): handle initial scenario for tpm/rpm routing	2023-12-30 07:28:45 +05:30
Krrish Dholakia	a30f00276b	refactor(lowest_tpm_rpm.py): move tpm/rpm based routing to a separate file for better testing	2023-12-29 18:33:43 +05:30
Krrish Dholakia	3fa1bb9f08	test(test_least_busy_router.py): add better testing for least busy routing	2023-12-29 17:16:00 +05:30
Krrish Dholakia	ffe2350428	fix(least_busy.py): support consistent use of model id instead of deployment name	2023-12-29 17:05:26 +05:30
Krrish Dholakia	79978c44ba	refactor: add black formatting	2023-12-25 14:11:20 +05:30
Krrish Dholakia	a65c8919fc	fix(router.py): fix least-busy routing	2023-12-08 20:29:49 -08:00