diff --git a/docs/my-website/docs/enterprise.md b/docs/my-website/docs/enterprise.md index b1dda8769..f5cd77aea 100644 --- a/docs/my-website/docs/enterprise.md +++ b/docs/my-website/docs/enterprise.md @@ -23,7 +23,7 @@ This covers: - ✅ [Audit Logs with retention policy](./proxy/enterprise#audit-logs) - ✅ [JWT-Auth](../docs/proxy/token_auth.md) - ✅ [Control available public, private routes](./proxy/enterprise#control-available-public-private-routes) - - ✅ [[BETA] AWS Key Manager v2 - Key Decryption](./proxy/enterprise#beta-aws-key-manager---key-decryption) + - ✅ [**Secret Managers** AWS Key Manager, Google Secret Manager, Azure Key](./secret) - ✅ IP address‑based access control lists - ✅ Track Request IP Address - ✅ [Use LiteLLM keys/authentication on Pass Through Endpoints](./proxy/pass_through#✨-enterprise---use-litellm-keysauthentication-on-pass-through-endpoints) diff --git a/docs/my-website/docs/proxy/enterprise.md b/docs/my-website/docs/proxy/enterprise.md index a432c72ae..3ff160225 100644 --- a/docs/my-website/docs/proxy/enterprise.md +++ b/docs/my-website/docs/proxy/enterprise.md @@ -17,7 +17,7 @@ Features: - ✅ [Audit Logs with retention policy](#audit-logs) - ✅ [JWT-Auth](../docs/proxy/token_auth.md) - ✅ [Control available public, private routes](#control-available-public-private-routes) - - ✅ [[BETA] AWS Key Manager v2 - Key Decryption](#beta-aws-key-manager---key-decryption) + - ✅ [**Secret Managers** AWS Key Manager, Google Secret Manager, Azure Key](../secret) - ✅ IP address‑based access control lists - ✅ Track Request IP Address - ✅ [Use LiteLLM keys/authentication on Pass Through Endpoints](pass_through#✨-enterprise---use-litellm-keysauthentication-on-pass-through-endpoints) diff --git a/docs/my-website/docs/secret.md b/docs/my-website/docs/secret.md index c2b6774c0..db5ec6910 100644 --- a/docs/my-website/docs/secret.md +++ b/docs/my-website/docs/secret.md @@ -1,9 +1,22 @@ # Secret Manager -LiteLLM supports reading secrets from Azure Key Vault and Infisical +LiteLLM supports reading secrets from Azure Key Vault, Google Secret Manager -- AWS Key Managemenet Service +:::info + +✨ **This is an Enterprise Feature** + +[Enterprise Pricing](https://www.litellm.ai/#pricing) + +[Contact us here to get a free trial](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat) + +::: + +## Supported Secret Managers + +- AWS Key Management Service - AWS Secret Manager - [Azure Key Vault](#azure-key-vault) +- [Google Secret Manager](#google-secret-manager) - Google Key Management Service - [Infisical Secret Manager](#infisical-secret-manager) - [.env Files](#env-files) @@ -125,6 +138,45 @@ litellm --config /path/to/config.yaml [Quick Test Proxy](./proxy/quick_start#using-litellm-proxy---curl-request-openai-package-langchain-langchain-js) +## Google Secret Manager + +Support for [Google Secret Manager](https://cloud.google.com/security/products/secret-manager) + + +1. Save Google Secret Manager details in your environment + +```shell +GOOGLE_SECRET_MANAGER_PROJECT_ID="your-project-id-on-gcp" # example: adroit-crow-413218 +``` + +Optional Params + +```shell +export GOOGLE_SECRET_MANAGER_REFRESH_INTERVAL = "" # (int) defaults to 86400 +export GOOGLE_SECRET_MANAGER_ALWAYS_READ_SECRET_MANAGER = "" # (str) set to "true" if you want to always read from google secret manager without using in memory caching. NOT RECOMMENDED in PROD +``` + +2. Add to proxy config.yaml +```yaml +model_list: + - model_name: fake-openai-endpoint + litellm_params: + model: openai/fake + api_base: https://exampleopenaiendpoint-production.up.railway.app/ + api_key: os.environ/OPENAI_API_KEY # this will be read from Google Secret Manager + +general_settings: + key_management_system: "google_secret_manager" +``` + +You can now test this by starting your proxy: +```bash +litellm --config /path/to/config.yaml +``` + +[Quick Test Proxy](./proxy/quick_start#using-litellm-proxy---curl-request-openai-package-langchain-langchain-js) + + ## Google Key Management Service Use encrypted keys from Google KMS on the proxy diff --git a/litellm/secret_managers/google_secret_manager.py b/litellm/secret_managers/google_secret_manager.py index 7da9a13e6..0d0c31005 100644 --- a/litellm/secret_managers/google_secret_manager.py +++ b/litellm/secret_managers/google_secret_manager.py @@ -48,9 +48,15 @@ class GoogleSecretManager(GCSBucketBase): _always_read_secret_manager = os.environ.get( "GOOGLE_SECRET_MANAGER_ALWAYS_READ_SECRET_MANAGER", - always_read_secret_manager, ) - self.always_read_secret_manager = _always_read_secret_manager + if ( + _always_read_secret_manager + and _always_read_secret_manager.lower() == "true" + ): + self.always_read_secret_manager = True + else: + # by default this should be False, we want to use in memory caching for this. It's a bad idea to fetch from secret manager for all requests + self.always_read_secret_manager = always_read_secret_manager or False def get_secret_from_google_secret_manager(self, secret_name: str) -> Optional[str]: """