From d7dd9f030723dbfff7457df3465d55c48310ff7e Mon Sep 17 00:00:00 2001
From: ishaan-jaff <ishaanjaffer0324@gmail.com>
Date: Mon, 27 Nov 2023 12:20:15 -0800
Subject: [PATCH] (docs) health check proxy llms

---
 docs/my-website/docs/simple_proxy.md | 43 ++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/docs/my-website/docs/simple_proxy.md b/docs/my-website/docs/simple_proxy.md
index b55f3cfafc..dd9bbaf48e 100644
--- a/docs/my-website/docs/simple_proxy.md
+++ b/docs/my-website/docs/simple_proxy.md
@@ -1050,6 +1050,41 @@ https://api.openai.com/v1/chat/completions \
 -d '{"model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "this is a test request, write a short poem"}]}'
 ```
 
+## Health Check LLMs on Proxy
+Use this to health check all LLMs defined in your config.yaml
+#### Request
+```shell
+curl --location 'http://0.0.0.0:8000/health'
+```
+
+You can also run `litellm -health` it makes a `get` request to `http://0.0.0.0:8000/health` for you
+```
+litellm --health
+```
+#### Response
+```shell
+{
+    "healthy_endpoints": [
+        {
+            "model": "azure/gpt-35-turbo",
+            "api_base": "https://my-endpoint-canada-berri992.openai.azure.com/"
+        },
+        {
+            "model": "azure/gpt-35-turbo",
+            "api_base": "https://my-endpoint-europe-berri-992.openai.azure.com/"
+        }
+    ],
+    "unhealthy_endpoints": [
+        {
+            "model": "azure/gpt-35-turbo",
+            "api_base": "https://openai-france-1234.openai.azure.com/"
+        }
+    ]
+}
+```
+
+
+
 ## Logging Proxy Input/Output - Langfuse
 We will use the `--config` to set `litellm.success_callback = ["langfuse"]` this will log all successfull LLM calls to langfuse
 
@@ -1156,6 +1191,14 @@ LiteLLM proxy adds **0.00325 seconds** latency as compared to using the Raw Open
      litellm --test
      ```
 
+#### --health
+   - **Type:** `bool` (Flag)
+   - Runs a health check on all models in config.yaml
+   - **Usage:** 
+     ```shell
+     litellm --health
+     ```
+
 #### --alias
    - **Default:** `None`
    - An alias for the model, for user-friendly reference.