2 commits

Author	SHA1	Message	Date
Ishaan Jaff	c8d15544c8	[Fix] Router cooldown logic - use % thresholds instead of allowed fails to cooldown deployments (#5698) ... * move cooldown logic to it's own helper * add new track deployment metrics folder * increment success, fails for deployment in current minute * fix cooldown logic * fix test_aaarouter_dynamic_cooldown_message_retry_time * fix test_single_deployment_no_cooldowns_test_prod_mock_completion_calls * clean up get from deployment test * fix _async_get_healthy_deployments * add mock InternalServerError * test deployment failing 25% requests * add test_high_traffic_cooldowns_one_bad_deployment * fix vertex load test * add test for rate limit error models in cool down * change default cooldown time * fix cooldown message time * fix cooldown on 429 error * fix doc string for _should_cooldown_deployment * fix sync cooldown logic router	2024-09-14 18:01:19 -07:00
Ishaan Jaff	cd8d7ca915	[Fix] Performance - use in memory cache when downloading images from a url (#5657) ... * fix use in memory cache when getting images * fix linting * fix load testing * fix load test size * fix load test size * trigger ci/cd again	2024-09-13 07:23:42 -07:00