Commit graph

19792 commits

Author SHA1 Message Date
Krish Dholakia
305049a968
Litellm dev 02 12 2025 p1 (#8494)
* Resolves https://github.com/BerriAI/litellm/issues/6625 (#8459)

- enables no auth for SMTP

Signed-off-by: Regli Daniel <daniel.regli1@sanitas.com>

* add sonar pricings (#8476)

* add sonar pricings

* Update model_prices_and_context_window.json

* Update model_prices_and_context_window.json

* Update model_prices_and_context_window_backup.json

* test: fix test

---------

Signed-off-by: Regli Daniel <daniel.regli1@sanitas.com>
Co-authored-by: Dani Regli <1daniregli@gmail.com>
Co-authored-by: Lucca Zenóbio <luccazen@gmail.com>
2025-02-12 22:39:29 -08:00
Krish Dholakia
aee90f1dfe
fix: fix test (#8501) 2025-02-12 18:38:15 -08:00
Ishaan Jaff
54811cf595 test_async_router_context_window_fallback 2025-02-12 18:14:35 -08:00
Ishaan Jaff
6f5440b5d8 test_supports_tool_choice 2025-02-12 18:13:27 -08:00
Krish Dholakia
1195fe2a44
Litellm UI stable version 02 12 2025 (#8497)
* fix(key_management_endpoints.py): fix `/key/list` to include `return_full_object` as a top-level query param

Allows user to specify they want the keys as a list of objects

* refactor(key_list.tsx): initial refactor of key table in user dashboard

offloads key filtering logic to backend api

prevents common error of user not being able to see their keys

* fix(key_management_endpoints.py): allow internal user to query `/key/list` to see their keys

* fix(key_management_endpoints.py): add validation checks and filtering to `/key/list` endpoint

allow internal user to see their keys. not anybody else's

* fix(view_key_table.tsx): fix issue where internal user could not see default team keys

* fix: fix linting error

* fix: fix linting error

* fix: fix linting error

* fix: fix linting error

* fix: fix linting error

* fix: fix linting error

* fix: fix linting error
2025-02-12 18:01:57 -08:00
Ishaan Jaff
40e3af0428
(Redis Cluster) - Fixes for using redis cluster + pipeline (#8442)
* update RedisCluster creation

* update RedisClusterCache

* add redis ClusterCache

* update async_set_cache_pipeline

* cleanup redis cluster usage

* fix redis pipeline

* test_init_async_client_returns_same_instance

* fix redis cluster

* update mypy_path

* fix init_redis_cluster

* remove stub

* test redis commit

* ClusterPipeline

* fix import

* RedisCluster import

* fix redis cluster

* Potential fix for code scanning alert no. 2129: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix naming of redis cluster integration

* test_redis_caching_ttl_pipeline

* fix async_set_cache_pipeline

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-02-12 18:01:32 -08:00
Krish Dholakia
b710407763
ci(config.yml): mark daily docker builds with -nightly (#8499)
Resolves https://github.com/BerriAI/litellm/discussions/8495
2025-02-12 18:01:08 -08:00
Ishaan Jaff
78f6fdcd92 test_async_router_context_window_fallback 2025-02-12 17:33:27 -08:00
Ishaan Jaff
9307f39daf
fix prom check startup (#8492) 2025-02-12 17:24:37 -08:00
Ishaan Jaff
ca7c5e8773
pplx - fix supports tool choice openai param (#8496) 2025-02-12 17:21:16 -08:00
Ishaan Jaff
faee508d1f fix test_async_router_context_window_fallback 2025-02-12 16:55:49 -08:00
Ishaan Jaff
a04204db40 update load testing script
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 12s
2025-02-12 16:07:06 -08:00
Lucca Zenóbio
dfb5a9985d
add sonar pricings (#8476)
* add sonar pricings

* Update model_prices_and_context_window.json

* Update model_prices_and_context_window.json

* Update model_prices_and_context_window_backup.json
2025-02-12 08:45:29 -08:00
Krrish Dholakia
9f93ed110a docs: fix docs
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 13s
2025-02-12 07:28:21 -08:00
Krrish Dholakia
c4a5e2c5c7 docs(token_auth.md): clarify scopes can be a list or comma separated string 2025-02-12 07:26:47 -08:00
Krrish Dholakia
d918b089c6 build(model_prices_and_context_window.json): handle azure model update
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 36s
2025-02-11 22:40:32 -08:00
Ishaan Jaff
bfae5c4161 fix naming docker stable release 2025-02-11 20:53:52 -08:00
Ishaan Jaff
caa848649c
(Bug fix) - Using include_usage for /completions requests + unit testing (#8484)
* pass stream options (#8419)

* test_completion_streaming_usage_metrics

* test_text_completion_include_usage

---------

Co-authored-by: Kaushik Deka <55996465+Kaushikdkrikhanu@users.noreply.github.com>
2025-02-11 20:29:04 -08:00
Krish Dholakia
57e5ec07cc
Improved wildcard route handling on /models and /model_group/info (#8473)
* fix(model_checks.py): update returning known model from wildcard to filter based on given model prefix

ensures wildcard route - `vertex_ai/gemini-*` just returns known vertex_ai/gemini- models

* test(test_proxy_utils.py): add unit testing for new 'get_known_models_from_wildcard' helper

* test(test_models.py): add e2e testing for `/model_group/info` endpoint

* feat(prometheus.py): support tracking total requests by user_email on prometheus

adds initial support for tracking total requests by user_email

* test(test_prometheus.py): add testing to ensure user email is always tracked

* test: update testing for new prometheus metric

* test(test_prometheus_unit_tests.py): add user email to total proxy metric

* test: update tests

* test: fix spend tests

* test: fix test

* fix(pagerduty.py): fix linting error
2025-02-11 19:37:43 -08:00
Ishaan Jaff
5e58ae0347 bump: version 1.61.0 → 1.61.1 2025-02-11 16:43:29 -08:00
Ishaan Jaff
946c6640a5 ui new build 2025-02-11 16:42:57 -08:00
Ishaan Jaff
81109893ec
(round 4 fixes) - Team model alias setting (#8474)
* update team info endpoint

* clean up model alias

* fix model alias

* fix model alias card

* clean up naming on docs

* fix model alias card

* fix _model_in_team_aliases

* team alias - fix litellm.model_alias_map

* fix _update_model_if_team_alias_exists

* fix test_aview_spend_per_user

* Test model alias functionality with teams:

* complete e2e test

* test_update_model_if_team_alias_exists
2025-02-11 16:40:01 -08:00
Ishaan Jaff
f1dd0f62b5 fix e2e ui testing linting error 2025-02-11 16:20:28 -08:00
Ishaan Jaff
5cd20d2abc
(UI) allow adding model aliases for teams (#8471)
* update team info endpoint

* clean up model alias

* fix model alias

* fix model alias card

* clean up naming on docs

* fix model alias card

* fix _model_in_team_aliases

* fix key_model_access_denied

* test_can_key_call_model_with_aliases

* fix test_aview_spend_per_user
2025-02-11 16:18:43 -08:00
Ishaan Jaff
ac14cfc25e fix test_aview_spend_per_user 2025-02-11 15:49:21 -08:00
Taha Ali
727283a3bc
Ui Fixes Teams Setting #8347 (#8353)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 14s
* [Bug] UI: Newly created key does not display on the View Key Page (#8039)

- Fixed issue where all keys appeared blank for admin users.
- Implemented filtering of data via team settings to ensure all keys are displayed correctly.

* Fix:
- Updated the validator to allow model editing when `keyTeam.team_alias === "Default Team"`.
- Ensured other teams still follow the original validation rules.

* - added some classes in global.css
- added text wrap in output of request,response and metadata in index.tsx
- fixed styles of table in table.tsx

* - added full payload when we open single log entry
- added Combined Info Card in index.tsx

* fix: keys not showing on refresh for internal user

* fix: btns and input fix
2025-02-11 12:39:03 -08:00
Krish Dholakia
ce3ead6f91
Log applied guardrails on LLM API call (#8452)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 40s
* fix(litellm_logging.py): support saving applied guardrails in logging object

allows list of applied guardrails to be logged for proxy admin's knowledge

* feat(spend_tracking_utils.py): log applied guardrails to spend logs

makes it easy for admin to know what guardrails were applied on a request

* ci(config.yml): uninstall posthog from ci/cd

* test: fix tests

* test: update test
2025-02-10 22:57:30 -08:00
Krish Dholakia
8e32713637
Show Guardrails on UI (#8447)
* feat(leftnav.tsx): initial commit adding guardrails tab on UI

allow proxy admin to see currently set up guardrails

* feat(guardrails.tsx): show configured guardrails on proxy ui

'

* fix(guardrails.tsx): style improvements

* feat(guardrails.py): return specific litellm params in `/guardrails/list` endpoint

support returning mode, default_on and guardrail name on `/guardrails/list` endpoint

* fix(guardrails.tsx): show guardrail name, status, mode on ui

* style(guardrails.tsx): style improvements to guardrails ui

* fix: fix linting error

* ci(config.yml): fix ci/cd error

* test: update test

* test: skip redundant test
2025-02-10 22:14:14 -08:00
Krrish Dholakia
b5ff15b3d3 test: skip redundant test 2025-02-10 22:13:58 -08:00
Krrish Dholakia
a552da5eef test: update test 2025-02-10 22:13:58 -08:00
Krrish Dholakia
197b1db6ec fix: fix linting error 2025-02-10 22:13:58 -08:00
Krrish Dholakia
184ee6c252 style(guardrails.tsx): style improvements to guardrails ui 2025-02-10 22:13:58 -08:00
Krrish Dholakia
0dfcf325b4 fix(guardrails.tsx): show guardrail name, status, mode on ui 2025-02-10 22:13:58 -08:00
Krrish Dholakia
e9a861ec32 feat(guardrails.py): return specific litellm params in /guardrails/list endpoint
support returning mode, default_on and guardrail name on `/guardrails/list` endpoint
2025-02-10 22:13:58 -08:00
Krrish Dholakia
93595826fb fix(guardrails.tsx): style improvements 2025-02-10 22:13:58 -08:00
Krrish Dholakia
c7a3e5b4b2 feat(guardrails.tsx): show configured guardrails on proxy ui
'
2025-02-10 22:13:58 -08:00
Krrish Dholakia
892c32cd39 feat(leftnav.tsx): initial commit adding guardrails tab on UI
allow proxy admin to see currently set up guardrails
2025-02-10 22:13:54 -08:00
Ishaan Jaff
40f51bf81f new ui build 2025-02-10 20:42:23 -08:00
Ishaan Jaff
00c596a852
(Feat) - Allow viewing Request/Response Logs stored in GCS Bucket (#8449)
* BaseRequestResponseFetchFromCustomLogger

* get_active_base_request_response_fetch_from_custom_logger

* get_request_response_payload

* ui_view_request_response_for_request_id

* fix uiSpendLogDetailsCall

* fix get_request_response_payload

* ui fix RequestViewer

* use 1 class AdditionalLoggingUtils

* ui_view_request_response_for_request_id

* cache the prefetch logs details

* refactor prefetch

* test view request/resp logs

* fix code quality

* fix get_request_response_payload

* uninstall posthog
prevent it from being added in ci/cd

* fix posthog

* fix traceloop test

* fix linting error
2025-02-10 20:38:55 -08:00
Ishaan Jaff
64a4229606
(e2e testing) - add tests for using litellm /team/ updates in multi-instance deployments with Redis (#8440)
* add team block/unblock test

* test_team_blocking_behavior_multi_instance

* proxy_multi_instance_tests

* test - Run Docker container 2
2025-02-10 19:33:27 -08:00
Krish Dholakia
13a3e8630e
Org UI Improvements (#8436)
* feat(team_endpoints.py): support returning teams filtered by organization_id

allows user to just get teams they belong to, within the org

Enables org admin to see filtered list of teams on UI

* fix(teams.tsx): simple filter for team on ui - just filter team based on selected org id

* feat(ui/organizations): show 'default org' in switcher, filter teams based on selected org

* feat(user_dashboard.tsx): update team in switcher when org changes

* feat(schema.prisma): add new 'organization_id' value to key table

allow org admin to directly issue keys to a user within their org

* fix(view_key_table.tsx): fix regression where admin couldn't see keys

caused by bad console log statement

* fix(team_endpoints.py): handle default org value in /team/list

* fix(key_management_endpoints.py): allow proxy admin to create keys for team they're not in

* fix(team_endpoints.py): fix team endpoint to handle org id not being passed in

* build(config.yml): investigate what pkg is installing posthog in ci/cd

* ci(config.yml): uninstall posthog

prevent it from being added in ci/cd

* ci: auto-install ci
2025-02-10 19:13:32 -08:00
Krish Dholakia
e26d7df91b
Litellm dev 02 10 2025 p2 (#8443)
* Fixed issue #8246 (#8250)

* Fixed issue #8246

* Added unit tests for discard() and for remove_callback_from_list_by_object()

* fix(openai.py): support dynamic passing of organization param to openai

handles scenario where client-side org id is passed to openai

---------

Co-authored-by: Erez Hadad <erezh@il.ibm.com>
2025-02-10 17:53:46 -08:00
Krish Dholakia
47f46f92c8
Litellm dev 02 10 2025 p1 (#8438)
* fix(azure/chat/gpt_transformation.py): fix str compare to use int - ensure correct api version check is done

Resolves https://github.com/BerriAI/litellm/issues/8241#issuecomment-2647142891

* test(test_azure_openai.py): add better testing
2025-02-10 16:25:04 -08:00
Krrish Dholakia
6a4013a075 bump: version 1.60.9 → 1.61.0
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 40s
2025-02-09 00:08:38 -08:00
Krrish Dholakia
802c6e58cc build: ui updates 2025-02-09 00:08:25 -08:00
Krish Dholakia
9c4c7813fb
Allow org admin to create teams on UI (#8407)
* fix(client_initialization_utils.py): handle custom llm provider set with valid value not from model name

* fix(handle_jwt.py): handle groups not existing in jwt token

if user not in group, this won't exist

* fix(handle_jwt.py): add new `enforce_team_based_model_access` flag to jwt auth

allows proxy admin to enforce user can only call model if team has access

* feat(navbar.tsx): expose new dropdown in navbar - allow org admin to create teams within org context

* fix(navbar.tsx): remove non-functional cogicon

* fix(proxy/utils.py): include user-org memberships in `/user/info` response

return orgs user is a member of and the user role within org

* feat(organization_endpoints.py): allow internal user to query `/organizations/list` and get all orgs they belong to

enables org admin to select org they belong to, to create teams

* fix(navbar.tsx): show change in ui when org switcher clicked

* feat(page.tsx): update user role based on org they're in

allows org admin to create teams in the org context

* feat(teams.tsx): working e2e flow for allowing org admin to add new teams

* style(navbar.tsx): clarify switching orgs on UI is in BETA

* fix(organization_endpoints.py): handle getting but not setting members

* test: fix test

* fix(client_initialization_utils.py): revert custom llm provider handling fix - causing unintended issues

* docs(token_auth.md): cleanup docs
2025-02-09 00:07:15 -08:00
Krish Dholakia
e4411e4815
Allow editing model api key + provider on UI (#8406)
* fix(parallel_request_limiter.py): add back parallel request information to max parallel request limiter

Resolves https://github.com/BerriAI/litellm/issues/8392

* test: mark flaky test to handle time based tracking issues

* feat(model_management_endpoints.py): expose new patch `/model/{model_id}/update` endpoint

Allows updating specific values of a model in db - makes it easy for admin to know this by calling it a PA
TCH

* feat(edit_model_modal.tsx): allow user to update llm provider + api key on the ui

* fix: fix linting error
2025-02-08 23:50:47 -08:00
Krish Dholakia
0d2e723e95
Revert "Added filter in Teams and fixed spacing & height issues in Teams tabs…" (#8416)
This reverts commit 352cfaa12c.
2025-02-08 23:25:56 -08:00
Taha Ali
352cfaa12c
Added filter in Teams and fixed spacing & height issues in Teams tabs (#6192) (#8357)
* [Bug] UI: Newly created key does not display on the View Key Page (#8039)

- Fixed issue where all keys appeared blank for admin users.
- Implemented filtering of data via team settings to ensure all keys are displayed correctly.

* Fix:
- Updated the validator to allow model editing when `keyTeam.team_alias === "Default Team"`.
- Ensured other teams still follow the original validation rules.

* - added some classes in global.css
- added text wrap in output of request,response and metadata in index.tsx
- fixed styles of table in table.tsx

* - added full payload when we open single log entry
- added Combined Info Card in index.tsx

* fix: keys not showing on refresh for internal user

* add: search added in teams
2025-02-08 23:25:41 -08:00
Ishaan Jaff
e25806d070 bump: version 1.60.8 → 1.60.9
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 37s
2025-02-08 22:39:51 -08:00