* add testing coverage for pattern match router
* fix add_pattern
* fix typo on router_cooldown_event_callback
* add testing for pattern match router
* fix add explanation for pattern match router
* add flake 8 check
* split up litellm _acompletion
* fix get model client
* refactor use commong func to add metadata to kwargs
* use common func to get timeout
* re-use helper to _get_async_model_client
* use _handle_mock_testing_rate_limit_error
* fix docstring for _handle_mock_testing_rate_limit_error
* fix function_with_retries
* use helper for mock testing fallbacks
* router - use 1 func for simple_shuffle
* add doc string for simple_shuffle
* use 1 function for filtering cooldown deployments
* fix use common helper to _get_fallback_model_group_from_fallbacks
* track api key and team in prom latency metric
* add test for latency metric
* test prometheus success metrics for latency
* track team and key labels for deployment failures
* add test for litellm_deployment_failure_responses_total
* fix checks for premium user on prometheus
* log_success_fallback_event and log_failure_fallback_event
* log original_exception in log_success_fallback_event
* track key, team and exception status and class on fallback metrics
* use get_standard_logging_metadata
* fix import error
* track litellm_deployment_successful_fallbacks
* add test test_proxy_fallback_metrics
* add log log_success_fallback_event
* fix test prometheus
* feat(router.py): initial commit for loadbalancing azure batch api endpoints
Closes https://github.com/BerriAI/litellm/issues/5396
* fix(router.py): working `router.acreate_file()`
* feat(router.py): working router.acreate_batch endpoint
* feat(router.py): expose router.aretrieve_batch function
Make it easy for user to retrieve the batch information
* feat(router.py): support 'router.alist_batches' endpoint
Adds support for getting all batches across all endpoints
* feat(router.py): working loadbalancing on `/v1/files`
* feat(proxy_server.py): working loadbalancing on `/v1/batches`
* feat(proxy_server.py): working loadbalancing on Retrieve + List batch
* Azure Service Principal with Secret authentication workflow. (#5131)
* Implement Azure Service Principal with Secret authentication workflow.
* Use `ClientSecretCredential` instead of `DefaultAzureCredential`.
* Move imports into the function.
* Add type hint for `azure_ad_token_provider`.
* Add unit test for router initialization and sample completion using Azure Service Principal with Secret authentication workflow.
* Add unit test for router initialization with neither API key nor using Azure Service Principal with Secret authentication workflow.
* fix(client_initializtion_utils.py): fix typing + overrides
* test: fix linting errors
* fix(client_initialization_utils.py): fix client init azure ad token logic
* fix(router_client_initialization.py): add flag check for reading azure ad token from environment
* test(test_streaming.py): skip end of life bedrock model
* test(test_router_client_init.py): add correct flag to test
---------
Co-authored-by: kzych-inpost <142029278+kzych-inpost@users.noreply.github.com>
Without this change, the default version of the Azure OpenAI API is hardcoded in
the code as an old version, `"2024-02-01"`. This change allows the user to set
the default version of the Azure OpenAI API by setting the environment variable
`AZURE_API_VERSION` or by using the command-line parameter `--api_version`.
