* fix(cost_calculator.py): handle custom pricing at deployment level for router
* test: add unit tests
* fix(router.py): show custom pricing on UI
check correct model str
* fix: fix linting error
* docs(custom_pricing.md): clarify custom pricing for proxy
Fixes https://github.com/BerriAI/litellm/issues/8573#issuecomment-2790420740
* test: update code qa test
* fix: cleanup traceback
* fix: handle litellm param custom pricing
* test: update test
* fix(cost_calculator.py): add router model id to list of potential model names
* fix(cost_calculator.py): fix router model id check
* fix: router.py - maintain older model registry approach
* fix: fix ruff check
* fix(router.py): router get deployment info
add custom values to mapped dict
* test: update test
* fix(utils.py): update only if value is non-null
* test: add unit test
* build(pyproject.toml): add new dev dependencies - for type checking
* build: reformat files to fit black
* ci: reformat to fit black
* ci(test-litellm.yml): make tests run clear
* build(pyproject.toml): add ruff
* fix: fix ruff checks
* build(mypy/): fix mypy linting errors
* fix(hashicorp_secret_manager.py): fix passing cert for tls auth
* build(mypy/): resolve all mypy errors
* test: update test
* fix: fix black formatting
* build(pre-commit-config.yaml): use poetry run black
* fix(proxy_server.py): fix linting error
* fix: fix ruff safe representation error
* feat(create_key_button.tsx): initial commit using openapi.json to ensure all values via api are supported on ui for `/key/generate`
Closes https://github.com/BerriAI/litellm/issues/7763
* style(create_key_button.tsx): put openapi settings inside 'advanced setting' accordion
* fix(check_openapi_schema.tsx): style improvements for advanced settings
* style(create_key_button.tsx): add tooltip explaining what the settings mean
* fix(team_info.tsx): render metadata field on team update
allow updating a team's metadata
* fix(networking.tsx): add 'metadata' field to create team form
* refactor: cleanup dead codeblock
* fix(organization_endpoints.py): fix metadata param support on `/organization/new`
* feat(organization_endpoints.py): support updating metadata for organization on api + ui
* test: mark flaky test
* feat(organization_endpoints.py): expose new `/organization/delete` endpoint. Cascade org deletion to member, teams and keys
Ensures any org deletion is handled correctly
* test(test_organizations.py): add simple test to ensure org deletion works
* feat(organization_endpoints.py): expose /organization/update endpoint, and define response models for org delete + update
* fix(organizations.tsx): support org delete on UI + move org/delete endpoint to use DELETE
* feat(organization_endpoints.py): support `/organization/member_update` endpoint
Allow admin to update member's role within org
* feat(organization_endpoints.py): support deleting member from org
* test(test_organizations.py): add e2e test to ensure org member flow works
* fix(organization_endpoints.py): fix code qa check
* fix(schema.prisma): don't introduce ondelete:cascade - breaking change
* docs(organization_endpoints.py): document missing params
* refactor(organization_view.tsx): initial commit creating a generic update member component shared between org and team member classes
* feat(organization_view.tsx): support updating org member role on UI
* feat(organization_view.tsx): allow proxy admin to delete members from org
* refactor(organization.tsx): initial commit with orgs tab refactor
make it similar to 'Teams' tab - simplifies org management actions
* fix(page.tsx): pass user orgs to component
* fix(organization_view.tsx): fix to pull info from org info endpoint
* feat(organization_endpoints.py): return org members when calling /org/info
* fix(organization_view.tsx): show org members on info page
* feat(organization_view.tsx): allow adding user to org via user email
Resolves https://github.com/BerriAI/litellm/issues/8330
* fix(organization_endpoints.py): raise better error when duplicate user_email found in db
* fix(organization_view.tsx): cleanup user_email for now
not in initial org info - will need to prefetch
* fix(page.tsx): fix getting user models on page load
allows passing down the user models to org
* fix(organizations.tsx): fix creating org on ui
* fix(proxy/_types.py): include org created at and updated at
cleans up ui
* fix(navbar.tsx): cleanup
* fix(organizations.tsx): fix tpm/rpm limits on org
* fix(organizations.tsx): fix linting error
* fix(organizations.tsx): fix linting \
* (Feat) - Add `/bedrock/meta.llama3-3-70b-instruct-v1:0` tool calling support + cost tracking + base llm unit test for tool calling (#8545)
* Add support for bedrock meta.llama3-3-70b-instruct-v1:0 tool calling (#8512)
* fix(converse_transformation.py): fixing bedrock meta.llama3-3-70b tool calling
* test(test_bedrock_completion.py): adding llama3.3 tool compatibility check
* add TestBedrockTestSuite
* add bedrock llama 3.3 to base llm class
* us.meta.llama3-3-70b-instruct-v1:0
* test_basic_tool_calling
* TestAzureOpenAIO1
* test_basic_tool_calling
* test_basic_tool_calling
---------
Co-authored-by: miraclebakelaser <65143272+miraclebakelaser@users.noreply.github.com>
* fix(general_settings.tsx): filter out empty dictionaries post fallback delete (#8550)
Fixes https://github.com/BerriAI/litellm/issues/8331
* bump: version 1.61.3 → 1.61.4
* (perf) Fix memory leak on `/completions` route (#8551)
* initial mem util test
* fix _cached_get_model_info_helper
* test memory usage
* fix tests
* fix mem usage
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: miraclebakelaser <65143272+miraclebakelaser@users.noreply.github.com>
* feat(team_endpoints.py): support returning teams filtered by organization_id
allows user to just get teams they belong to, within the org
Enables org admin to see filtered list of teams on UI
* fix(teams.tsx): simple filter for team on ui - just filter team based on selected org id
* feat(ui/organizations): show 'default org' in switcher, filter teams based on selected org
* feat(user_dashboard.tsx): update team in switcher when org changes
* feat(schema.prisma): add new 'organization_id' value to key table
allow org admin to directly issue keys to a user within their org
* fix(view_key_table.tsx): fix regression where admin couldn't see keys
caused by bad console log statement
* fix(team_endpoints.py): handle default org value in /team/list
* fix(key_management_endpoints.py): allow proxy admin to create keys for team they're not in
* fix(team_endpoints.py): fix team endpoint to handle org id not being passed in
* build(config.yml): investigate what pkg is installing posthog in ci/cd
* ci(config.yml): uninstall posthog
prevent it from being added in ci/cd
* ci: auto-install ci
* fix(client_initialization_utils.py): handle custom llm provider set with valid value not from model name
* fix(handle_jwt.py): handle groups not existing in jwt token
if user not in group, this won't exist
* fix(handle_jwt.py): add new `enforce_team_based_model_access` flag to jwt auth
allows proxy admin to enforce user can only call model if team has access
* feat(navbar.tsx): expose new dropdown in navbar - allow org admin to create teams within org context
* fix(navbar.tsx): remove non-functional cogicon
* fix(proxy/utils.py): include user-org memberships in `/user/info` response
return orgs user is a member of and the user role within org
* feat(organization_endpoints.py): allow internal user to query `/organizations/list` and get all orgs they belong to
enables org admin to select org they belong to, to create teams
* fix(navbar.tsx): show change in ui when org switcher clicked
* feat(page.tsx): update user role based on org they're in
allows org admin to create teams in the org context
* feat(teams.tsx): working e2e flow for allowing org admin to add new teams
* style(navbar.tsx): clarify switching orgs on UI is in BETA
* fix(organization_endpoints.py): handle getting but not setting members
* test: fix test
* fix(client_initialization_utils.py): revert custom llm provider handling fix - causing unintended issues
* docs(token_auth.md): cleanup docs
* feat: initial commit for new 'organizations' tab on ui
* build(ui/): create generic card for rendering complete org data table
can be reused in teams as well
simplifies things
* build(ui/): display created orgs on ui
* build(ui/): support adding orgs via UI
* build(ui/): add org in selection dropdown
* build(organizations.tsx): allow assigning org admins
* build(ui/): show org members on ui
* build(ui/): cleanup + show actual models on org dropdown
* build(ui/): explain user roles within organization
* feat(pass_through_endpoints/): support logging anthropic/gemini pass through calls to langfuse/s3/etc.
* fix(utils.py): allow disabling end user cost tracking with new param
Allows proxy admin to disable cost tracking for end user - keeps prometheus metrics small
* docs(configs.md): add disable_end_user_cost_tracking reference to docs
* feat(key_management_endpoints.py): add support for restricting access to `/key/generate` by team/proxy level role
Enables admin to restrict key creation, and assign team admins to handle distributing keys
* test(test_key_management.py): add unit testing for personal / team key restriction checks
* docs: add docs on restricting key creation
* docs(finetuned_models.md): add new guide on calling finetuned models
* docs(input.md): cleanup anthropic supported params
Closes https://github.com/BerriAI/litellm/issues/6856
* test(test_embedding.py): add test for passing extra headers via embedding
* feat(cohere/embed): pass client to async embedding
* feat(rerank.py): add `/v1/rerank` if missing for cohere base url
Closes https://github.com/BerriAI/litellm/issues/6844
* fix(main.py): pass extra_headers param to openai
Fixes https://github.com/BerriAI/litellm/issues/6836
* fix(litellm_logging.py): don't disable global callbacks when dynamic callbacks are set
Fixes issue where global callbacks - e.g. prometheus were overriden when langfuse was set dynamically
* fix(handler.py): fix linting error
* fix: fix typing
* build: add conftest to proxy_admin_ui_tests/
* test: fix test
* fix: fix linting errors
* test: fix test
* fix: fix pass through testing
* feat(customer_endpoints.py): support passing budget duration via `/customer/new` endpoint
Closes https://github.com/BerriAI/litellm/issues/5651
* docs: add missing params to swagger + api documentation test
* docs: add documentation for all key endpoints
documents all params on swagger
* docs(internal_user_endpoints.py): document all /user/new params
Ensures all params are documented
* docs(team_endpoints.py): add missing documentation for team endpoints
Ensures 100% param documentation on swagger
* docs(organization_endpoints.py): document all org params
Adds documentation for all params in org endpoint
* docs(customer_endpoints.py): add coverage for all params on /customer endpoints
ensures all /customer/* params are documented
* ci(config.yml): add endpoint doc testing to ci/cd
* fix: fix internal_user_endpoints.py
* fix(internal_user_endpoints.py): support 'duration' param
* fix(partner_models/main.py): fix anthropic re-raise exception on vertex
* fix: fix pydantic obj
* Update organization_endpoints.py to be able to list organizations (#6473)
* Update organization_endpoints.py to be able to list organizations
* Update test_organizations.py
* Update test_organizations.py
add test for list
* Update test_organizations.py
correct indentation
* Add unreleased Claude 3.5 Haiku models. (#6476)
---------
Co-authored-by: superpoussin22 <vincent.nadal@orange.fr>
Co-authored-by: David Manouchehri <david.manouchehri@ai.moda>
* track LiteLLM_OrganizationMembership
* add add_internal_user_to_organization
* add org membership to schema
* read organization membership when reading user info in auth checks
* add check for valid organization_id
* add test for test_create_new_user_in_organization
* test test_create_new_user_in_organization
* add new ADMIN role
* add test for org admins creating teams
* add test for test_org_admin_create_user_permissions
* test_org_admin_create_user_team_wrong_org_permissions
* test_org_admin_create_user_team_wrong_org_permissions
* fix organization_role_based_access_check
* fix getting user members
* fix TeamBase
* fix types used for use role
* fix type checks
* sync prisma schema
* docs - organization admins
* fix use organization_endpoints for /organization management
* add types for org member endpoints
* fix role name for org admin
* add type for member add response
* add organization/member_add
* add error handling for adding members to an org
* add nice doc string for oranization/member_add
* fix test_create_new_user_in_organization
* linting fix
* use simple route changes
* fix types
* add organization member roles
* add org admin auth checks
* add auth checks for orgs
* test for creating teams as org admin
* simplify org id usage
* fix typo
* test test_org_admin_create_user_team_wrong_org_permissions
* fix type check issue
* code quality fix
* fix schema.prisma
