Ishaan Jaff c8d15544c8 [Fix] Router cooldown logic - use % thresholds instead of allowed fails to cooldown deployments (#5698) ... * move cooldown logic to it's own helper * add new track deployment metrics folder * increment success, fails for deployment in current minute * fix cooldown logic * fix test_aaarouter_dynamic_cooldown_message_retry_time * fix test_single_deployment_no_cooldowns_test_prod_mock_completion_calls * clean up get from deployment test * fix _async_get_healthy_deployments * add mock InternalServerError * test deployment failing 25% requests * add test_high_traffic_cooldowns_one_bad_deployment * fix vertex load test * add test for rate limit error models in cool down * change default cooldown time * fix cooldown message time * fix cooldown on 429 error * fix doc string for _should_cooldown_deployment * fix sync cooldown logic router		2024-09-14 18:01:19 -07:00
..
track_deployment_metrics.py	[Fix] Router cooldown logic - use % thresholds instead of allowed fails to cooldown deployments (#5698)	2024-09-14 18:01:19 -07:00