6 commits

Author	SHA1	Message	Date
Krrish Dholakia	f21ec71caf	test(test_tpm_rpm_routing_v2.py): fix test - bump number of iteration s	2024-04-30 08:48:55 -07:00
Krrish Dholakia	3afe7ab1a1	fix(lowest_tpm_rpm_v2.py): shuffle deployments with same tpm values	2024-04-29 15:23:47 -07:00
Krrish Dholakia	5da934099f	fix(caching.py): dual cache async_batch_get_cache fix + testing ... this fixes a bug in usage-based-routing-v2 which was caused b/c of how the result was being returned from dual cache async_batch_get_cache. it also adds unit testing for that function (and it's sync equivalent)	2024-04-19 15:03:25 -07:00
Krrish Dholakia	5bb73dc9c0	fix(router.py): instrument pre-call-checks for all openai endpoints	2024-04-18 21:54:25 -07:00
Krrish Dholakia	376ee4e9d7	fix(test_lowest_tpm_rpm_routing_v2.py): unit testing for usage-based-routing-v2	2024-04-18 21:38:00 -07:00
Krrish Dholakia	72691e05f4	fix(tpm_rpm_routing_v2.py): fix tpm rpm routing	2024-04-18 20:01:22 -07:00