6 commits

Author	SHA1	Message	Date
Krrish Dholakia	d717fa2588	test(test_tpm_rpm_routing_v2.py): fix test - bump number of iteration s	2024-04-30 08:48:55 -07:00
Krrish Dholakia	a978f2d881	fix(lowest_tpm_rpm_v2.py): shuffle deployments with same tpm values	2024-04-29 15:23:47 -07:00
Krrish Dholakia	01a1a8f731	fix(caching.py): dual cache async_batch_get_cache fix + testing ... this fixes a bug in usage-based-routing-v2 which was caused b/c of how the result was being returned from dual cache async_batch_get_cache. it also adds unit testing for that function (and it's sync equivalent)	2024-04-19 15:03:25 -07:00
Krrish Dholakia	9c42c847a5	fix(router.py): instrument pre-call-checks for all openai endpoints	2024-04-18 21:54:25 -07:00
Krrish Dholakia	81573b2dd9	fix(test_lowest_tpm_rpm_routing_v2.py): unit testing for usage-based-routing-v2	2024-04-18 21:38:00 -07:00
Krrish Dholakia	a05f148c17	fix(tpm_rpm_routing_v2.py): fix tpm rpm routing	2024-04-18 20:01:22 -07:00