phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-26 11:14:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
litellm-mirror/docs/my-website/docs/proxy/jwt_auth_arch.md
Krish Dholakia dad24f2b52
Litellm dev 01 29 2025 p2 (#8102) 
* docs: cleanup doc

* feat(bedrock/): initial commit adding bedrock/converse_like/<model> route support

allows routing to a converse like endpoint

Resolves https://github.com/BerriAI/litellm/issues/8085

* feat(bedrock/chat/converse_transformation.py): make converse config base config compatible

enables new 'converse_like' route

* feat(converse_transformation.py): enables using the proxy with converse like api endpoint

Resolves https://github.com/BerriAI/litellm/issues/8085
2025-01-29 20:53:37 -08:00

3.3 KiB
Raw Blame History

import Image from '@theme/IdealImage'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';

Control Model Access with OIDC (Azure AD/Keycloak/etc.)

:::info

✨ JWT Auth is on LiteLLM Enterprise

Enterprise Pricing

Get free 7-day trial key

:::

<Image img={require('../../img/control_model_access_jwt.png')} style={{ width: '100%', maxWidth: '4000px' }} />

Example Token

{
  "sub": "1234567890",
  "name": "John Doe",
  "email": "john.doe@example.com",
  "roles": ["basic_user"] # 👈 ROLE
}

{
  "sub": "1234567890",
  "name": "John Doe",
  "email": "john.doe@example.com",
  "resource_access": {
    "litellm-test-client-id": {
      "roles": ["basic_user"] # 👈 ROLE
    }
  }
}

Proxy Configuration

general_settings:
  enable_jwt_auth: True 
  litellm_jwtauth:
    user_roles_jwt_field: "roles" # the field in the JWT that contains the roles 
    user_allowed_roles: ["basic_user"] # roles that map to an 'internal_user' role on LiteLLM 
    enforce_rbac: true # if true, will check if the user has the correct role to access the model
  
  role_permissions: # control what models are allowed for each role
    - role: internal_user
      models: ["anthropic-claude"]

model_list:
    - model: anthropic-claude
      litellm_params:
        model: claude-3-5-haiku-20241022
    - model: openai-gpt-4o
      litellm_params:
        model: gpt-4o

general_settings:
  enable_jwt_auth: True 
  litellm_jwtauth:
    user_roles_jwt_field: "resource_access.litellm-test-client-id.roles" # the field in the JWT that contains the roles
    user_allowed_roles: ["basic_user"] # roles that map to an 'internal_user' role on LiteLLM 
    enforce_rbac: true # if true, will check if the user has the correct role to access the model
  
  role_permissions: # control what models are allowed for each role
    - role: internal_user
      models: ["anthropic-claude"]

model_list:
    - model: anthropic-claude
      litellm_params:
        model: claude-3-5-haiku-20241022
    - model: openai-gpt-4o
      litellm_params:
        model: gpt-4o

How it works

  1. Specify JWT_PUBLIC_KEY_URL - This is the public keys endpoint of your OpenID provider. For Azure AD it's https://login.microsoftonline.com/{tenant_id}/discovery/v2.0/keys. For Keycloak it's {keycloak_base_url}/realms/{your-realm}/protocol/openid-connect/certs.

  2. Map JWT roles to LiteLLM roles - Done via user_roles_jwt_field and user_allowed_roles

    • Currently just internal_user is supported for role mapping.

  3. Specify model access:

    • role_permissions: control what models are allowed for each role.
      • role: the LiteLLM role to control access for. Allowed roles = ["internal_user", "proxy_admin", "team"]
      • models: list of models that the role is allowed to access.
    • model_list: parent list of models on the proxy. Learn more

  4. Model Checks: The proxy will run validation checks on the received JWT. Code