diff --git a/docs/my-website/docs/routing.md b/docs/my-website/docs/routing.md index 6e7a076bd5..ac5dcbe29e 100644 --- a/docs/my-website/docs/routing.md +++ b/docs/my-website/docs/routing.md @@ -2,22 +2,9 @@ import Image from '@theme/IdealImage'; # Reliability - Fallbacks, Azure Deployments, etc. +Prevent failed calls and slow response times with multiple deployments for API calls (E.g. multiple azure-openai deployments). -HF_Dashboard - -# Reliability - -LiteLLM helps prevent failed requests in 3 ways: -- Retries -- Fallbacks: Context Window + General -- RateLimitManager - -## Helper utils -LiteLLM supports the following functions for reliability: -* `litellm.longer_context_model_fallback_dict`: Dictionary which has a mapping for those models which have larger equivalents -* `num_retries`: use tenacity retries -* `completion()` with fallbacks: switch between models/keys/api bases in case of errors. -* `router()`: An abstraction on top of completion + embeddings to route the request to a deployment with capacity (available tpm/rpm). +HF_Dashboard ## Manage Multiple Deployments @@ -135,6 +122,13 @@ response = completion( ## Fallbacks +## Helper utils +LiteLLM supports the following functions for reliability: +* `litellm.longer_context_model_fallback_dict`: Dictionary which has a mapping for those models which have larger equivalents +* `num_retries`: use tenacity retries +* `completion()` with fallbacks: switch between models/keys/api bases in case of errors. + + ### Context Window Fallbacks ```python from litellm import completion diff --git a/docs/my-website/img/multiple_deployment.png b/docs/my-website/img/multiple_deployment.png deleted file mode 100644 index 5f0c309427..0000000000 Binary files a/docs/my-website/img/multiple_deployment.png and /dev/null differ diff --git a/docs/my-website/img/multiple_deployments.png b/docs/my-website/img/multiple_deployments.png new file mode 100644 index 0000000000..d28fce8d9b Binary files /dev/null and b/docs/my-website/img/multiple_deployments.png differ