From 7b38dbbbe930f8045c8023b1b22386d7734d33eb Mon Sep 17 00:00:00 2001
From: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Tue, 26 Mar 2024 18:02:44 -0700
Subject: [PATCH] docs(enterprise.md): update docs to turn on/off llm guard per
 key

---
 docs/my-website/docs/proxy/enterprise.md | 92 +++++++++++++++++-------
 1 file changed, 66 insertions(+), 26 deletions(-)

diff --git a/docs/my-website/docs/proxy/enterprise.md b/docs/my-website/docs/proxy/enterprise.md
index 7071239248..0c72077ee6 100644
--- a/docs/my-website/docs/proxy/enterprise.md
+++ b/docs/my-website/docs/proxy/enterprise.md
@@ -12,9 +12,9 @@ Features here are behind a commercial license in our `/enterprise` folder. [**Se
 :::
 
 Features: 
+- ✅ Content Moderation with LLM Guard
 - ✅ Content Moderation with LlamaGuard 
 - ✅ Content Moderation with Google Text Moderations 
-- ✅ Content Moderation with LLM Guard
 - ✅ Reject calls from Blocked User list 
 - ✅ Reject calls (incoming / outgoing) with Banned Keywords (e.g. competitors)
 - ✅ Don't log/store specific requests (eg confidential LLM requests)
@@ -23,6 +23,71 @@ Features:
 
 
 ## Content Moderation
+### Content Moderation with LLM Guard
+
+Set the LLM Guard API Base in your environment 
+
+```env
+LLM_GUARD_API_BASE = "http://0.0.0.0:8192" # deployed llm guard api
+```
+
+Add `llmguard_moderations` as a callback 
+
+```yaml
+litellm_settings:
+    callbacks: ["llmguard_moderations"]
+```
+
+Now you can easily test it
+
+- Make a regular /chat/completion call 
+
+- Check your proxy logs for any statement with `LLM Guard:`
+
+Expected results: 
+
+```
+LLM Guard: Received response - {"sanitized_prompt": "hello world", "is_valid": true, "scanners": { "Regex": 0.0 }}
+```
+#### Turn on/off per key
+
+**1. Update config**
+```yaml
+litellm_settings:
+    callbacks: ["llmguard_moderations"]
+    llm_guard_mode: "key-specific"
+```
+
+**2. Create new key**
+
+```bash
+curl --location 'http://localhost:4000/key/generate' \
+--header 'Authorization: Bearer sk-1234' \
+--header 'Content-Type: application/json' \
+--data '{
+    "models": ["fake-openai-endpoint"],
+    "permissions": {
+        "enable_llm_guard_check": true # 👈 KEY CHANGE
+    }
+}'
+
+# Returns {..'key': 'my-new-key'}
+```
+
+**2. Test it!**
+
+```bash
+curl --location 'http://0.0.0.0:4000/v1/chat/completions' \
+--header 'Content-Type: application/json' \
+--header 'Authorization: Bearer my-new-key' \ # 👈 TEST KEY
+--data '{"model": "fake-openai-endpoint", "messages": [
+        {"role": "system", "content": "Be helpful"},
+        {"role": "user", "content": "What do you know?"}
+    ]
+    }'
+```
+
+
 ### Content Moderation with LlamaGuard 
 
 Currently works with Sagemaker's LlamaGuard endpoint. 
@@ -55,32 +120,7 @@ callbacks: ["llamaguard_moderations"]
   llamaguard_unsafe_content_categories: /path/to/llamaguard_prompt.txt
 ```
 
-### Content Moderation with LLM Guard
 
-Set the LLM Guard API Base in your environment 
-
-```env
-LLM_GUARD_API_BASE = "http://0.0.0.0:8192" # deployed llm guard api
-```
-
-Add `llmguard_moderations` as a callback 
-
-```yaml
-litellm_settings:
-    callbacks: ["llmguard_moderations"]
-```
-
-Now you can easily test it
-
-- Make a regular /chat/completion call 
-
-- Check your proxy logs for any statement with `LLM Guard:`
-
-Expected results: 
-
-```
-LLM Guard: Received response - {"sanitized_prompt": "hello world", "is_valid": true, "scanners": { "Regex": 0.0 }}
-```
 
 ### Content Moderation with Google Text Moderation