Commit graph

204 commits

Author SHA1 Message Date
Ishaan Jaff
b41311bb21
(UI) - Fix show correct count of internal user keys on Users Page (#9082)
* get_user_key_counts

* fix get_user_key_counts

* fix get_user_key_counts

* test_get_users_filters_dashboard_keys

* remove unused func
2025-03-08 16:13:18 -08:00
Ishaan Jaff
7f70bdd99b
(Feat) - add pricing for eu.amazon.nova models (#9056)
* add pricing for eu.amazon.nova models

* fix typo in key management endpoints.py
2025-03-07 07:06:17 -08:00
Krish Dholakia
5591354309
Support master key rotations (#9041)
* feat(key_management_endpoints.py): adding support for rotating master key

* feat(key_management_endpoints.py): support decryption-re-encryption of models in db, when master key rotated

* fix(user_api_key_auth.py): raise valid token is None error earlier

enables easier debugging with api key hash in error message

* feat(key_management_endpoints.py): rotate any env vars

* fix(key_management_endpoints.py): uncomment check

* fix: fix linting error
2025-03-06 23:13:30 -08:00
Krish Dholakia
274147bc5e
fix(team_endpoints.py): ensure 404 raised when team not found (#9038)
* fix(team_endpoints.py): ensure 404 raised when team not found

* fix(key_management_endpoints.py): fix adding tags to key when metadata is empty

* fix(key_management_endpoints.py): refactor set metadata field to use common function across keys + teams

reduces scope for errors + easier testing

* fix: fix linting error
2025-03-06 22:04:36 -08:00
Krrish Dholakia
3be3b802c8 fix: fix linting error 2025-03-05 10:10:53 -08:00
Ishaan Jaff
8d6815ce98 Revert "(UI) - Security Improvement, move to JWT Auth for Admin UI Sessions (#8995)"
This reverts commit 01a44a4e47.
2025-03-05 08:49:20 -08:00
Krrish Dholakia
313b315791 fix: fix linting error 2025-03-05 08:26:26 -08:00
Krish Dholakia
c69ec66dc5
fix(base_aws_llm.py): remove region name before sending in args (#8998)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 12s
* fix(base_aws_llm.py): remove region name before sending in args

* fix(base_aws_llm.py): fix optional param pop position

* fix: fix linting error
2025-03-04 23:05:28 -08:00
Krrish Dholakia
e715c376b9 fix: fix linting error 2025-03-04 22:34:07 -08:00
Krish Dholakia
5e386c28b2
Litellm dev 03 04 2025 p3 (#8997)
* fix(core_helpers.py): handle litellm_metadata instead of 'metadata'

* feat(batches/): ensure batches logs are written to db

makes batches response dict compatible

* fix(cost_calculator.py): handle batch response being a dictionary

* fix(batches/main.py): modify retrieve endpoints to use @client decorator

enables logging to work on retrieve call

* fix(batches/main.py): fix retrieve batch response type to be 'dict' compatible

* fix(spend_tracking_utils.py): send unique uuid for retrieve batch call type

create batch and retrieve batch share the same id

* fix(spend_tracking_utils.py): prevent duplicate retrieve batch calls from being double counted

* refactor(batches/): refactor cost tracking for batches - do it on retrieve, and within the established litellm_logging pipeline

ensures cost is always logged to db

* fix: fix linting errors

* fix: fix linting error
2025-03-04 21:58:03 -08:00
Ishaan Jaff
01a44a4e47
(UI) - Security Improvement, move to JWT Auth for Admin UI Sessions (#8995)
* (UI) - Improvements to session handling logic  (#8970)

* add cookieUtils

* use utils for clearing cookies

* on logout use clearTokenCookies

* ui use correct clearTokenCookies

* navbar show userEmail on UserID page

* add timestamp on token cookie

* update generate_authenticated_redirect_response

* use common getAuthToken

* fix clearTokenCookies

* fixes for get auth token

* fix invitation link sign in logic

* Revert "fix invitation link sign in logic"

This reverts commit 30e5308cb3.

* fix getAuthToken

* update setAuthToken

* fix ui session handling

* fix ui session handler

* bug fix stop generating LiteLLM Virtual keys for access

* working JWT insert into cookies

* use central place to build UI JWT token

* add _validate_ui_token

* fix ui session handler

* fix fetchWithCredentials

* check allowed routes for ui session tokens

* expose validate_session endpoint

* validate session endpoint

* call sso/session/validate

* getUISessionDetails

* ui move to getUISessionDetails

* /sso/session/validate

* fix cookie utils

* use getUISessionDetails

* use ui_session_id

* "/spend/logs/ui" in spend_tracking_routes

* working sign in JWT flow for proxy admin

* allow proxy admin to access ui routes

* use check_route_access

* update types

* update login method

* fixes to ui session handler

* working flow for admin and internal users

* fixes for invite links

* use JWTs for SSO sign in

* fix /invitation/new flow

* fix code quality checks

* fix _get_ui_session_token_from_cookies

* /organization/list

* ui sso sign in

* TestUISessionHandler

* TestUISessionHandler
2025-03-04 21:48:23 -08:00
Ishaan Jaff
4c8b4fefc9 Revert "(UI) - Improvements to session handling logic (#8970)"
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 15s
This reverts commit c015fb34f1.
2025-03-04 13:29:08 -08:00
Krrish Dholakia
9baf4f7e56 fix: fix linting errors 2025-03-04 06:13:53 -08:00
Ishaan Jaff
c015fb34f1
(UI) - Improvements to session handling logic (#8970)
* add cookieUtils

* use utils for clearing cookies

* on logout use clearTokenCookies

* ui use correct clearTokenCookies

* navbar show userEmail on UserID page

* add timestamp on token cookie

* update generate_authenticated_redirect_response

* use common getAuthToken

* fix clearTokenCookies

* fixes for get auth token

* fix invitation link sign in logic

* Revert "fix invitation link sign in logic"

This reverts commit 30e5308cb3.

* fix getAuthToken

* update setAuthToken

* fix ui session handling

* fix ui session handler
2025-03-03 22:17:21 -08:00
Krish Dholakia
c1527ebf52
UI - Allow admin to control default model access for internal users (#8912)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 36s
* fix(create_user_button.tsx): allow admin to set models user has access to, on invite

Enables controlling model access on invite

* feat(auth_checks.py): enforce 'no-model-access' special model name on backend

prevent user from calling models if default key has no model access

* fix(chat_ui.tsx): allow user to input custom model

* fix(chat_ui.tsx): pull available models based on models key has access to

* style(create_user_button.tsx): move default model inside 'personal key creation' accordion

* fix(chat_ui.tsx): fix linting error

* test(test_auth_checks.py): add unit-test for special model name

* docs(internal_user_endpoints.py): update docstring

* fix test_moderations_bad_model

* Litellm dev 02 27 2025 p6 (#8891)

* fix(http_parsing_utils.py): orjson can throw errors on some emoji's in text, default to json.loads

* fix(sagemaker/handler.py): support passing model id on async streaming

* fix(litellm_pre_call_utils.py): Fixes https://github.com/BerriAI/litellm/issues/7237

* Fix calling claude via invoke route + response_format support for claude on invoke route (#8908)

* fix(anthropic_claude3_transformation.py): fix amazon anthropic claude 3 tool calling transformation on invoke route

move to using anthropic config as base

* fix(utils.py): expose anthropic config via providerconfigmanager

* fix(llm_http_handler.py): support json mode on async completion calls

* fix(invoke_handler/make_call): support json mode for anthropic called via bedrock invoke

* fix(anthropic/): handle 'response_format: {"type": "text"}` + migrate amazon claude 3 invoke config to inherit from anthropic config

Prevents error when passing in 'response_format: {"type": "text"}

* test: fix test

* fix(utils.py): fix base invoke provider check

* fix(anthropic_claude3_transformation.py): don't pass 'stream' param

* fix: fix linting errors

* fix(converse_transformation.py): handle response_format type=text for converse

* converse_transformation: pass 'description' if set in response_format (#8907)

* test(test_bedrock_completion.py): e2e test ensuring tool description is passed in

* fix(converse_transformation.py): pass description, if set

* fix(transformation.py): Fixes https://github.com/BerriAI/litellm/issues/8767#issuecomment-2689887663

* Fix bedrock passing `response_format: {"type": "text"}` (#8900)

* fix(converse_transformation.py): ignore type: text, value in response_format

no-op for bedrock

* fix(converse_transformation.py): handle adding response format value to tools

* fix(base_invoke_transformation.py): fix 'get_bedrock_invoke_provider' to handle cross-region-inferencing models

* test(test_bedrock_completion.py): add unit testing for bedrock invoke provider logic

* test: update test

* fix(exception_mapping_utils.py): add context window exceeded error handling for databricks provider route

* fix(fireworks_ai/): support passing tools + response_format together

* fix: cleanup

* fix(base_invoke_transformation.py): fix imports

* (Feat) - Show Error Logs on LiteLLM UI  (#8904)

* fix test_moderations_bad_model

* use async_post_call_failure_hook

* basic logging errors in DB

* show status on ui

* show status on ui

* ui show request / response side by side

* stash fixes

* working, track raw request

* track error info in metadata

* fix showing error / request / response logs

* show traceback on error viewer

* ui with traceback of error

* fix async_post_call_failure_hook

* fix(http_parsing_utils.py): orjson can throw errors on some emoji's in text, default to json.loads

* test_get_error_information

* fix code quality

* rename proxy track cost callback test

* _should_store_errors_in_spend_logs

* feature flag error logs

* Revert "_should_store_errors_in_spend_logs"

This reverts commit 7f345df477.

* Revert "feature flag error logs"

This reverts commit 0e90c022bb.

* test_spend_logs_payload

* fix OTEL log_db_metrics

* fix import json

* fix ui linting error

* test_async_post_call_failure_hook

* test_chat_completion_bad_model_with_spend_logs

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>

* ui new build

* test_chat_completion_bad_model_with_spend_logs

* docs(release_cycle.md): document release cycle

* bump: version 1.62.0 → 1.62.1

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-02-28 23:23:03 -08:00
Krish Dholakia
887c66c6b7
Show 'user_email' on key table on UI (#8887)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 35s
* refactor(internal_user_endpoints.py): refactor `/user/list` to accept 'user_ids' and use prisma for db calls

enables bulk search from UI

* fix(internal_user_endpoints.py): fix linting errors

* fix(all_keys_table.tsx): show user email on create key table

make it easier for admin to know which key is associated to which user

* docs(internal_user_endpoints.py): improve docstring

* fix: sync schema with main

* fix(columns.tsx): display SSO ID on Internal User Table

make it easy to identify what the SSO ID for a user is

* fix(columns.tsx): add tooltip to header

help user understand what SSO ID means

* style: add more tooltips in the management flows

make it easier to understand what you're seeing

* style(all_keys_table.tsx): replace 'Not Set' with '-'

reduces words on table

* fix(internal_user_endpoints.py): fix user ids check

* test: fix test

* fix(internal_user_endpoints.py): maintain returning key count in `/user/list`
2025-02-27 21:56:14 -08:00
Krish Dholakia
2d2d1b9df5
Add created_by and updated_by fields to Keys table (#8885)
* fix(proxy/_types.py): return created_by and updated_by on /key/list

enables better trail of who made a key

* fix(all_keys_table.tsx): add created by to key table

allows easier tracking of who generated the key

* fix(key_management_endpoints.py): track 'created_by' and 'updated_by' fields

enable easier tracking of who created proxy keys
2025-02-27 18:12:58 -08:00
Ishaan Jaff
f8e43296fb
(UI) Fixes for managing Internal Users (#8786)
* allow bulk adding internal users

* allow sorting users by created at

* cleanup

* clean up user table

* show total num users

* show per user error when bulk adding users

* fix - don't allow creating duplicate internal users in DB

* ui flow fix for bulk adding users

* allow adding user in multiple teams with models

* correctly extract info

* working invitation link

* fix fill in table after bulk add

* fix the results from creating new users in bulkd

* bulk invite users

* fix view user flow

* fix ui type errors

* fix type errors

* fix type errors
2025-02-24 23:40:13 -08:00
Krish Dholakia
566d9354aa
fix(proxy/_types.py): fixes issue where internal user able to escalat… (#8740)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 41s
* fix(proxy/_types.py): fixes issue where internal user able to escalate their role with ui key

Fixes https://github.com/BerriAI/litellm/issues/8029

* style: cleanup

* test: handle bedrock instability
2025-02-22 22:59:58 -08:00
Krish Dholakia
c2aec21b4d
fix(amazon_deepseek_transformation.py): remove </think> from stream o… (#8717)
* fix(amazon_deepseek_transformation.py): remove </think> from stream output - cleanup user facing stream

* fix(key_managenet_endpoints.py): return `/key/list` sorted by created_at

makes it easier to see created key

* style: cleanup team table

* feat(key_edit_view.tsx): support setting model specific tpm/rpm limits on keys
2025-02-22 21:46:55 -08:00
Krish Dholakia
cc77138b37
Add all /key/generate api params to UI + add metadata fields on team AND org add/update (#8667)
* feat(create_key_button.tsx): initial commit using openapi.json to ensure all values via api are supported on ui for `/key/generate`

Closes https://github.com/BerriAI/litellm/issues/7763

* style(create_key_button.tsx): put openapi settings inside 'advanced setting' accordion

* fix(check_openapi_schema.tsx): style improvements for advanced settings

* style(create_key_button.tsx): add tooltip explaining what the settings mean

* fix(team_info.tsx): render metadata field on team update

allow updating a team's metadata

* fix(networking.tsx): add 'metadata' field to create team form

* refactor: cleanup dead codeblock

* fix(organization_endpoints.py): fix metadata param support on `/organization/new`

* feat(organization_endpoints.py): support updating metadata for organization on api + ui

* test: mark flaky test
2025-02-19 21:13:06 -08:00
Krish Dholakia
0319559295
fix(team_endpoints.py): allow team member to view team info (#8644)
* fix(team_endpoints.py): allow team member to view team info

* test: handle model overloaded in tool calling test

* test: handle internal server error
2025-02-18 22:28:57 -08:00
Ishaan Jaff
b88762b63c
(Polish/Fixes) - Fixes for Adding Team Specific Models (#8645)
* refactor get model info for team models

* allow adding a model to a team when creating team specific model

* ui update selected Team on Team Dropdown

* test_team_model_association

* testing for team specific models

* test_get_team_specific_model

* test: skip on internal server error

* remove model alias card on teams page

* linting fix _get_team_specific_model

* fix DeploymentTypedDict

* fix linting error

* fix code quality

* fix model info checks

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
2025-02-18 21:11:57 -08:00
Krish Dholakia
2b7755f8d8
Litellm dev 02 18 2025 p3 (#8640)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 14s
* fix(team_endpoints.py): cleanup user <-> team association on team delete

Fixes issue where user table still listed team membership post delete

* test(test_team.py): update e2e test - ensure user/team membership is deleted on team delete

* fix(base_invoke_transformation.py): fix deepseek r1 transformation

remove deepseek name from model url

* test(test_completion.py): assert model route not in url

* feat(base_invoke_transformation.py): infer region name from model arn

prevent errors due to different region name in env var vs. model arn, respect if explicitly set in call though

* test: fix test

* test: skip on internal server error
2025-02-18 19:14:20 -08:00
Ishaan Jaff
8024300825
(UI) Improvements to Add Team Model Flow (#8603)
* ui - use common team dropdown component

* re-use team component

* rename org field on add model

* handle add model submit

* working view model_id and team_id on root models page

* cleaner

* show all fields

* working model info view

* working team info selector

* clean up team id

* new component for model dashboard

* ui show table with dropdown

* make public model names like email

* revert changes to litellm model name

* fix litellm model name

* ui fix public model

* fix mappings

* fix conditional text input

* fix message

* ui fix bulk add models

* _add_team_model_to_db

* move model mgmt helper funcs

* test_add_team_model_to_db

* ui - display model team model name

* fix add model tab

* fix remove redundant info tab on models page

* dont pass model mappings all the way through

* fix jarring model name when adding team models

* fix edit model button

* delete button on model info

* ui fix model dashboard

* fix DeploymentTypedDict

* _is_model_access_group_for_wildcard_route

* test _get_public_model_name

* ui fix viewing public model name

* fix linting error

* fix linting errors

* fix selectedModel logic
2025-02-17 18:37:14 -08:00
Krish Dholakia
51cb3c84e3
Litellm stable UI 02 17 2025 p1 (#8599)
* fix(key_management_endpoints.py): initial commit with logic to get all keys for teams user is an admin for

* fix(key_managements_endpoints.py): return all keys for teams user is an admin for

* fix(key_management_endpoints.py): add query param to ensure user opts into seeing all team keys (not just their own)

* fix(regenerate_key_modal.tsx): fix key regenerate

* fix(proxy_server.py): fix model metrics check on none api base

* test(test_key_generate_prisma.py): remove redundant test

* test(test_proxy_utils.py): add unit test covering new management endpoint helper util

* fix: fix test

* test(test_proxy_server.py): fix test
2025-02-17 17:55:05 -08:00
Krish Dholakia
bdc1a72542
refactor(teams.tsx): refactor to display all teams, across all orgs (#8565)
* refactor(teams.tsx): refactor to display all teams, across all orgs

removes org switcher from navbar, simplifies viewing/creating teams on UI

* fix(key_list.tsx): show user keys across all orgs

make it easy to see flat list of keys across orgs on key table

* style(all_keys_table.tsx): cleanup keys table

* fix(user_dashboard.tsx): remove overflow-hidden in dashboard component

* fix(teams.tsx): move org id placement in create team flow

* fix(teams.tsx): support model selection on create team based on selected org

* feat(view_key_table.tsx): move to using a filter component on keys page

allows filtering keys by org and team

* fix(filter.tsx): handle reset filter

* fix: fix linting error

* (Feat) - return `x-litellm-attempted-fallbacks` in responses from litellm proxy  (#8558)

* add_fallback_headers_to_response

* test x-litellm-attempted-fallbacks

* unit test attempted fallbacks

* fix add_fallback_headers_to_response

* docs document response headers

* fix file name

* test fix use mock endpoints for e2e files and ft tests

* Revert "test fix use mock endpoints for e2e files and ft tests"

This reverts commit c921d8dd81.

* cleanup_azure_files

* Add remaining org CRUD endpoints + support deleting orgs on UI  (#8561)

* feat(organization_endpoints.py): expose new `/organization/delete` endpoint. Cascade org deletion to member, teams and keys

Ensures any org deletion is handled correctly

* test(test_organizations.py): add simple test to ensure org deletion works

* feat(organization_endpoints.py): expose /organization/update endpoint, and define response models for org delete + update

* fix(organizations.tsx): support org delete on UI + move org/delete endpoint to use DELETE

* feat(organization_endpoints.py): support `/organization/member_update` endpoint

Allow admin to update member's role within org

* feat(organization_endpoints.py): support deleting member from org

* test(test_organizations.py): add e2e test to ensure org member flow works

* fix(organization_endpoints.py): fix code qa check

* fix(schema.prisma): don't introduce ondelete:cascade - breaking change

* docs(organization_endpoints.py): document missing params

* refactor(organization_view.tsx): initial commit creating a generic update member component shared between org and team member classes

* feat(organization_view.tsx): support updating org member role on UI

* feat(organization_view.tsx): allow proxy admin to delete members from org

* Enable update/delete org members on UI  (#8560)

* feat(organization_endpoints.py): expose new `/organization/delete` endpoint. Cascade org deletion to member, teams and keys

Ensures any org deletion is handled correctly

* test(test_organizations.py): add simple test to ensure org deletion works

* feat(organization_endpoints.py): expose /organization/update endpoint, and define response models for org delete + update

* fix(organizations.tsx): support org delete on UI + move org/delete endpoint to use DELETE

* feat(organization_endpoints.py): support `/organization/member_update` endpoint

Allow admin to update member's role within org

* feat(organization_endpoints.py): support deleting member from org

* test(test_organizations.py): add e2e test to ensure org member flow works

* fix(organization_endpoints.py): fix code qa check

* fix(schema.prisma): don't introduce ondelete:cascade - breaking change

* docs(organization_endpoints.py): document missing params

* (Bug Fix) - Add Regenerate Key on Virtual Keys Tab (#8567)

* add regenerate key to ui

* ui fix key info

* (Bug Fix + Better Observability) - BudgetResetJob:  (#8562)

* use class ResetBudgetJob

* refactor reset budget job

* update reset_budget job

* refactor reset budget job

* fix LiteLLM_UserTable

* refactor reset budget job

* add telemetry for reset budget job

* dd - log service success/failure on DD

* add detailed reset budget reset info on DD

* initialize_scheduled_background_jobs

* refactor reset budget job

* trigger service failure hook when fails to reset a budget for team, key, user

* fix resetBudgetJob

* unit testing for ResetBudgetJob

* test_duration_in_seconds_basic

* testing for triggering service logging

* fix logs on test teams fail

* remove unused imports

* fix import duration in s

* duration_in_seconds

* (Patch/bug fix) - UI, filter out litellm ui session tokens on Virtual Keys Page (#8568)

* fix key list endpoint

* _get_condition_to_filter_out_ui_session_tokens

* duration_in_seconds

* test_list_key_helper_team_filtering

* bump: version 1.61.4 → 1.61.5

* ui fix tsx linting

* ui new build

* test_list_key_helper_team_filtering

* ui new build

* test_openai_fine_tuning

* test_openai_fine_tuning

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-02-15 23:50:11 -08:00
Ishaan Jaff
4d49b50f71
(Patch/bug fix) - UI, filter out litellm ui session tokens on Virtual Keys Page (#8568)
* fix key list endpoint

* _get_condition_to_filter_out_ui_session_tokens

* duration_in_seconds

* test_list_key_helper_team_filtering
2025-02-15 16:25:57 -08:00
Ishaan Jaff
2753de1458
(Bug Fix + Better Observability) - BudgetResetJob: (#8562)
* use class ResetBudgetJob

* refactor reset budget job

* update reset_budget job

* refactor reset budget job

* fix LiteLLM_UserTable

* refactor reset budget job

* add telemetry for reset budget job

* dd - log service success/failure on DD

* add detailed reset budget reset info on DD

* initialize_scheduled_background_jobs

* refactor reset budget job

* trigger service failure hook when fails to reset a budget for team, key, user

* fix resetBudgetJob

* unit testing for ResetBudgetJob

* test_duration_in_seconds_basic

* testing for triggering service logging

* fix logs on test teams fail

* remove unused imports

* fix import duration in s

* duration_in_seconds
2025-02-15 16:13:08 -08:00
Krish Dholakia
01ace78dcc
Add remaining org CRUD endpoints + support deleting orgs on UI (#8561)
* feat(organization_endpoints.py): expose new `/organization/delete` endpoint. Cascade org deletion to member, teams and keys

Ensures any org deletion is handled correctly

* test(test_organizations.py): add simple test to ensure org deletion works

* feat(organization_endpoints.py): expose /organization/update endpoint, and define response models for org delete + update

* fix(organizations.tsx): support org delete on UI + move org/delete endpoint to use DELETE

* feat(organization_endpoints.py): support `/organization/member_update` endpoint

Allow admin to update member's role within org

* feat(organization_endpoints.py): support deleting member from org

* test(test_organizations.py): add e2e test to ensure org member flow works

* fix(organization_endpoints.py): fix code qa check

* fix(schema.prisma): don't introduce ondelete:cascade - breaking change

* docs(organization_endpoints.py): document missing params

* refactor(organization_view.tsx): initial commit creating a generic update member component shared between org and team member classes

* feat(organization_view.tsx): support updating org member role on UI

* feat(organization_view.tsx): allow proxy admin to delete members from org
2025-02-15 15:48:06 -08:00
Krish Dholakia
f91e9728b0
fix(team_endpoints.py): fix team info check to handle team keys (#8529) 2025-02-14 21:48:28 -08:00
Krish Dholakia
138ef0a5a9
Org Flow Improvements (#8549)
* refactor(organization.tsx): initial commit with orgs tab refactor

make it similar to 'Teams' tab - simplifies org management actions

* fix(page.tsx): pass user orgs to component

* fix(organization_view.tsx): fix to pull info from org info endpoint

* feat(organization_endpoints.py): return org members when calling /org/info

* fix(organization_view.tsx): show org members on info page

* feat(organization_view.tsx): allow adding user to org via user email

Resolves https://github.com/BerriAI/litellm/issues/8330

* fix(organization_endpoints.py): raise better error when duplicate user_email found in db

* fix(organization_view.tsx): cleanup user_email for now

not in initial org info - will need to prefetch

* fix(page.tsx): fix getting user models on page load

allows passing down the user models to org

* fix(organizations.tsx): fix creating org on ui

* fix(proxy/_types.py): include org created at and updated at

cleans up ui

* fix(navbar.tsx): cleanup

* fix(organizations.tsx): fix tpm/rpm limits on org

* fix(organizations.tsx): fix linting error

* fix(organizations.tsx): fix linting \

* (Feat) - Add `/bedrock/meta.llama3-3-70b-instruct-v1:0` tool calling support + cost tracking + base llm unit test for tool calling (#8545)

* Add support for bedrock meta.llama3-3-70b-instruct-v1:0 tool calling (#8512)

* fix(converse_transformation.py): fixing bedrock meta.llama3-3-70b tool calling

* test(test_bedrock_completion.py): adding llama3.3 tool compatibility check

* add TestBedrockTestSuite

* add bedrock llama 3.3 to base llm class

* us.meta.llama3-3-70b-instruct-v1:0

* test_basic_tool_calling

* TestAzureOpenAIO1

* test_basic_tool_calling

* test_basic_tool_calling

---------

Co-authored-by: miraclebakelaser <65143272+miraclebakelaser@users.noreply.github.com>

* fix(general_settings.tsx): filter out empty dictionaries post fallback delete (#8550)

Fixes https://github.com/BerriAI/litellm/issues/8331

* bump: version 1.61.3 → 1.61.4

* (perf) Fix memory leak on `/completions` route (#8551)

* initial mem util test

* fix _cached_get_model_info_helper

* test memory usage

* fix tests

* fix mem usage

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: miraclebakelaser <65143272+miraclebakelaser@users.noreply.github.com>
2025-02-14 19:49:03 -08:00
Krish Dholakia
3f0438a59d
UI Fixes p2 (#8502)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 13s
* refactor(admin.tsx): cleanup add new admin flow

removes buggy flow. Ensures just 1 simple way to add users / update roles.

* fix(user_search_modal.tsx): ensure 'add member' button is always visible

* fix(edit_membership.tsx): ensure 'save changes' button always visible

* fix(internal_user_endpoints.py): ensure user in org can be deleted

Fixes issue where user couldn't be deleted if they were a member of an org

* fix: fix linting error
2025-02-12 22:51:54 -08:00
Krish Dholakia
aee90f1dfe
fix: fix test (#8501) 2025-02-12 18:38:15 -08:00
Krish Dholakia
1195fe2a44
Litellm UI stable version 02 12 2025 (#8497)
* fix(key_management_endpoints.py): fix `/key/list` to include `return_full_object` as a top-level query param

Allows user to specify they want the keys as a list of objects

* refactor(key_list.tsx): initial refactor of key table in user dashboard

offloads key filtering logic to backend api

prevents common error of user not being able to see their keys

* fix(key_management_endpoints.py): allow internal user to query `/key/list` to see their keys

* fix(key_management_endpoints.py): add validation checks and filtering to `/key/list` endpoint

allow internal user to see their keys. not anybody else's

* fix(view_key_table.tsx): fix issue where internal user could not see default team keys

* fix: fix linting error

* fix: fix linting error

* fix: fix linting error

* fix: fix linting error

* fix: fix linting error

* fix: fix linting error

* fix: fix linting error
2025-02-12 18:01:57 -08:00
Ishaan Jaff
5cd20d2abc
(UI) allow adding model aliases for teams (#8471)
* update team info endpoint

* clean up model alias

* fix model alias

* fix model alias card

* clean up naming on docs

* fix model alias card

* fix _model_in_team_aliases

* fix key_model_access_denied

* test_can_key_call_model_with_aliases

* fix test_aview_spend_per_user
2025-02-11 16:18:43 -08:00
Krish Dholakia
13a3e8630e
Org UI Improvements (#8436)
* feat(team_endpoints.py): support returning teams filtered by organization_id

allows user to just get teams they belong to, within the org

Enables org admin to see filtered list of teams on UI

* fix(teams.tsx): simple filter for team on ui - just filter team based on selected org id

* feat(ui/organizations): show 'default org' in switcher, filter teams based on selected org

* feat(user_dashboard.tsx): update team in switcher when org changes

* feat(schema.prisma): add new 'organization_id' value to key table

allow org admin to directly issue keys to a user within their org

* fix(view_key_table.tsx): fix regression where admin couldn't see keys

caused by bad console log statement

* fix(team_endpoints.py): handle default org value in /team/list

* fix(key_management_endpoints.py): allow proxy admin to create keys for team they're not in

* fix(team_endpoints.py): fix team endpoint to handle org id not being passed in

* build(config.yml): investigate what pkg is installing posthog in ci/cd

* ci(config.yml): uninstall posthog

prevent it from being added in ci/cd

* ci: auto-install ci
2025-02-10 19:13:32 -08:00
Krish Dholakia
9c4c7813fb
Allow org admin to create teams on UI (#8407)
* fix(client_initialization_utils.py): handle custom llm provider set with valid value not from model name

* fix(handle_jwt.py): handle groups not existing in jwt token

if user not in group, this won't exist

* fix(handle_jwt.py): add new `enforce_team_based_model_access` flag to jwt auth

allows proxy admin to enforce user can only call model if team has access

* feat(navbar.tsx): expose new dropdown in navbar - allow org admin to create teams within org context

* fix(navbar.tsx): remove non-functional cogicon

* fix(proxy/utils.py): include user-org memberships in `/user/info` response

return orgs user is a member of and the user role within org

* feat(organization_endpoints.py): allow internal user to query `/organizations/list` and get all orgs they belong to

enables org admin to select org they belong to, to create teams

* fix(navbar.tsx): show change in ui when org switcher clicked

* feat(page.tsx): update user role based on org they're in

allows org admin to create teams in the org context

* feat(teams.tsx): working e2e flow for allowing org admin to add new teams

* style(navbar.tsx): clarify switching orgs on UI is in BETA

* fix(organization_endpoints.py): handle getting but not setting members

* test: fix test

* fix(client_initialization_utils.py): revert custom llm provider handling fix - causing unintended issues

* docs(token_auth.md): cleanup docs
2025-02-09 00:07:15 -08:00
Krish Dholakia
e4411e4815
Allow editing model api key + provider on UI (#8406)
* fix(parallel_request_limiter.py): add back parallel request information to max parallel request limiter

Resolves https://github.com/BerriAI/litellm/issues/8392

* test: mark flaky test to handle time based tracking issues

* feat(model_management_endpoints.py): expose new patch `/model/{model_id}/update` endpoint

Allows updating specific values of a model in db - makes it easy for admin to know this by calling it a PA
TCH

* feat(edit_model_modal.tsx): allow user to update llm provider + api key on the ui

* fix: fix linting error
2025-02-08 23:50:47 -08:00
Krrish Dholakia
7f06b88192 fix(internal_user_endpoints.py): fix try-except for team not in db 2025-02-04 21:57:43 -08:00
Krish Dholakia
df93debbc7
Internal User Endpoint - vulnerability fix + response type fix (#8228)
* fix(key_management_endpoints.py): fix vulnerability where a user could update another user's keys

Resolves https://github.com/BerriAI/litellm/issues/8031

* test(key_management_endpoints.py): return consistent 403 forbidden error when modifying key that doesn't belong to user

* fix(internal_user_endpoints.py): return model max budget in internal user create response

Fixes https://github.com/BerriAI/litellm/issues/7047

* test: fix test

* test: update test to handle gemini token counter change

* fix(factory.py): fix bedrock http:// handling

* docs: fix typo in lm_studio.md (#8222)

* test: fix testing

* test: fix test

---------

Co-authored-by: foreign-sub <51928805+foreign-sub@users.noreply.github.com>
2025-02-04 06:41:14 -08:00
Krish Dholakia
6834c5ecaf
Easier user onboarding via SSO (#8187)
* fix(ui_sso.py): use common `get_user_object` logic across jwt + ui sso auth

Allows finding users by their email, and attaching the sso user id to the user if found

* Improve Team Management flow on UI  (#8204)

* build(teams.tsx): refactor teams page to make it easier to add members to a team

make a row in table clickable -> allows user to add users to team they intended

* build(teams.tsx): make it clear user should click on team id to view team details

simplifies team management by putting team details on separate page

* build(team_info.tsx): separately show user id and user email

make it easy for user to understand the information they're seeing

* build(team_info.tsx): add back in 'add member' button

* build(team_info.tsx): working team member update on team_info.tsx

* build(team_info.tsx): enable team member delete on ui

allow user to delete accidental adds

* build(internal_user_endpoints.py): expose new endpoint for ui to allow filtering on user table

allows proxy admin to quickly find user they're looking for

* feat(team_endpoints.py): expose new team filter endpoint for ui

allows proxy admin to easily find team they're looking for

* feat(user_search_modal.tsx): allow admin to filter on users when adding new user to teams

* test: mark flaky test

* test: mark flaky test

* fix(exception_mapping_utils.py): fix anthropic text route error

* fix(ui_sso.py): handle situation when user not in db
2025-02-02 23:02:33 -08:00
Krish Dholakia
9e65f867ab
test: add more unit testing for team member endpoints (#8170)
* test: add more unit testing for team member add

* fix(team_endpoints.py): add validation check to prevent same user from being added to team again

prevents duplicates

* fix(team_endpoints.py): raise error if `/team/member_delete` called on member that's not in team

prevent being able to call delete on same member multiple times

* test: update initial tests

* test: fix test

* test: update test to handle no member duplication
2025-02-01 11:23:00 -08:00
Krish Dholakia
de261e2120
Doc updates + management endpoint fixes (#8138)
* Litellm dev 01 29 2025 p4 (#8107)

* fix(key_management_endpoints.py): always get db team

Fixes https://github.com/BerriAI/litellm/issues/7983

* test(test_key_management.py): add unit test enforcing check_db_only is always true on key generate checks

* test: fix test

* test: skip gemini thinking

* Litellm dev 01 29 2025 p3 (#8106)

* fix(__init__.py): reduces size of __init__.py and reduces scope for errors by using correct param

* refactor(__init__.py): refactor init by cleaning up redundant params

* refactor(__init__.py): move more constants into constants.py

cleanup root

* refactor(__init__.py): more cleanup

* feat(__init__.py): expose new 'disable_hf_tokenizer_download' param

enables hf model usage in offline env

* docs(config_settings.md): document new disable_hf_tokenizer_download param

* fix: fix linting error

* fix: fix unsafe comparison

* test: fix test

* docs(public_teams.md): add doc showing how to expose public teams for users to join

* docs: add beta disclaimer on public teams

* test: update tests
2025-01-30 22:56:41 -08:00
Ishaan Jaff
a7b3c664d1
(Feat) set guardrails per team (#7993)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 35s
* _add_guardrails_from_key_or_team_metadata

* e2e test test_guardrails_with_team_controls

* add try/except on team new

* test_guardrails_with_team_controls

* test_guardrails_with_api_key_controls
2025-01-25 10:41:11 -08:00
Ishaan Jaff
669b4fc955
(Prometheus) - emit key budget metrics on startup (#8002)
* add UI_SESSION_TOKEN_TEAM_ID

* add type KeyListResponseObject

* add _list_key_helper

* _initialize_api_key_budget_metrics

* key / budget metrics

* init key budget metrics on startup

* test_initialize_api_key_budget_metrics

* fix linting

* test_list_key_helper

* test_initialize_remaining_budget_metrics_exception_handling
2025-01-25 10:37:52 -08:00
Krish Dholakia
1ab10d8f72
Litellm dev 01 24 2025 p4 (#7992)
* feat(team_endpoints.py): new `/teams/available` endpoint - allows proxy admin to expose available teams for users to join on UI

* build(ui/): available_teams.tsx

allow user to join available teams on UI

makes it easier to onboard new users to teams

* fix(navbar.tsx): cleanup title

* fix(team_endpoints.py): fix linting error

* test: update groq model in test

* build(model_prices_and_context_window.json): update groq 3.3 model with 'supports function calling'
2025-01-24 21:29:37 -08:00
Ishaan Jaff
74caef0843
(Feat) - Add GCS Pub/Sub Logging integration for sending DB SpendLogs to BigQuery (#7976)
* add pub_sub

* fix custom batch logger for GCS PUB/SUB

* GCS_PUBSUB_PROJECT_ID

* e2e gcs pub sub

* add gcs pub sub

* fix logging

* add GcsPubSubLogger

* fix pub sub

* add pub sub

* docs gcs pub / sub

* docs on pub sub controls

* test_gcs_pub_sub

* fix publish_message

* test_async_gcs_pub_sub

* test_async_gcs_pub_sub
2025-01-24 20:57:20 -08:00
Ishaan Jaff
70a3683552 fix LiteLLM_ManagementEndpoint_MetadataFields 2025-01-23 20:59:38 -08:00
Ishaan Jaff
085920aa1c
(Feat) allow setting guardrails on a team on the API (#7959)
* allow setting guardrails on a team

* test set guardrails on team

* set guardrails on a team

* fix LiteLLM_ManagementEndpoint_MetadataFields_Premium
2025-01-23 20:26:51 -08:00