Commit graph

4664 commits

Author SHA1 Message Date
Ishaan Jaff
6e88d1e662 Merge branch 'main' into litellm_metrics_pod_lock_manager 2025-04-04 21:11:39 -07:00
Chaos Yu
1127fb4b7a make sure metadata available and have a value (#9764) 2025-04-04 20:39:12 -07:00
Ishaan Jaff
b541f17e89 PrometheusAuthMiddleware 2025-04-04 20:37:53 -07:00
Ishaan Jaff
5b6b1c0fb8 allow adding auth on /metrics endpoint 2025-04-04 20:37:17 -07:00
Krish Dholakia
88df504375 Allow team members to see team models (#9742)
* fix(proxy_server.py): allow team member to see team models

* fix(model_dashboard.tsx): show edit + delete icons to be disabled if user is not admin and did not create models

* fix(proxy_server.py): fix ruff function size error

* fix(proxy_server.py): fix user model filter check
2025-04-04 20:36:48 -07:00
Ishaan Jaff
dea96e17bb _should_run_auth_on_metrics_endpoint 2025-04-04 20:32:04 -07:00
Ishaan Jaff
c4c2d4c40e PrometheusAuthMiddleware 2025-04-04 20:27:17 -07:00
Ishaan Jaff
1260d616dc allow requiring auth for /metrics endpoint 2025-04-04 17:35:02 -07:00
Ishaan Jaff
2fe3b08425 Merge branch 'main' into litellm_reliability_fix_db_txs 2025-04-04 16:46:46 -07:00
Ishaan Jaff
ab6c9e0313 add operational metrics for pod lock manager v2 arch 2025-04-04 16:41:07 -07:00
Ishaan Jaff
98bc54b428 Merge branch 'main' into litellm_metrics_pod_lock_manager 2025-04-04 16:33:16 -07:00
Ishaan Jaff
4b314a9aa1 test_queue_flush_limit 2025-04-04 16:29:06 -07:00
Ishaan Jaff
6948b62a3a test_queue_size_reduction_with_large_volume 2025-04-04 16:21:29 -07:00
Ishaan Jaff
4198d17544 aggregate_queue_updates 2025-04-04 15:54:07 -07:00
Ishaan Jaff
528f9799fd Merge pull request #9745 from BerriAI/litellm_sso_fixes_dev
[Feat] Allow assigning SSO users to teams on MSFT SSO
2025-04-04 15:40:19 -07:00
Ishaan Jaff
462827c9ec flush_all_updates_from_in_memory_queue 2025-04-04 15:34:56 -07:00
Ishaan Jaff
87b834585b add _get_aggregated_spend_update_queue_item 2025-04-04 15:32:27 -07:00
Ishaan Jaff
b42b3bf6c8 add logic for max size in memory queue 2025-04-04 15:31:40 -07:00
Ishaan Jaff
8e80928efd add MAX_SIZE_IN_MEMORY_QUEUE 2025-04-04 15:31:09 -07:00
Ishaan Jaff
a852b7e273 ui new build 2025-04-04 14:39:55 -07:00
Ishaan Jaff
04d3cead3f fix sso callback 2025-04-03 22:13:46 -07:00
Ishaan Jaff
6ca245e41c MSFT make it easier for using group ids with MSFT 2025-04-03 20:43:22 -07:00
Ishaan Jaff
8aa2e7db69 debug show SSO callback result 2025-04-03 20:06:21 -07:00
Krish Dholakia
0ce878e804 LiteLLM Minor Fixes & Improvements (04/02/2025) (#9725)
* Add date picker to usage tab + Add reasoning_content token tracking across all providers on streaming (#9722)

* feat(new_usage.tsx): add date picker for new usage tab

allow user to look back on their usage data

* feat(anthropic/chat/transformation.py): report reasoning tokens in completion token details

allows usage tracking on how many reasoning tokens are actually being used

* feat(streaming_chunk_builder.py): return reasoning_tokens in anthropic/openai streaming response

allows tracking reasoning_token usage across providers

* Fix update team metadata + fix bulk adding models on Ui  (#9721)

* fix(handle_add_model_submit.tsx): fix bulk adding models

* fix(team_info.tsx): fix team metadata update

Fixes https://github.com/BerriAI/litellm/issues/9689

* (v0) Unified file id - allow calling multiple providers with same file id (#9718)

* feat(files_endpoints.py): initial commit adding 'target_model_names' support

allow developer to specify all the models they want to call with the file

* feat(files_endpoints.py): return unified files endpoint

* test(test_files_endpoints.py): add validation test - if invalid purpose submitted

* feat: more updates

* feat: initial working commit of unified file id translation

* fix: additional fixes

* fix(router.py): remove model replace logic in jsonl on acreate_file

enables file upload to work for chat completion requests as well

* fix(files_endpoints.py): remove whitespace around model name

* fix(azure/handler.py): return acreate_file with correct response type

* fix: fix linting errors

* test: fix mock test to run on github actions

* fix: fix ruff errors

* fix: fix file too large error

* fix(utils.py): remove redundant var

* test: modify test to work on github actions

* test: update tests

* test: more debug logs to understand ci/cd issue

* test: fix test for respx

* test: skip mock respx test

fails on ci/cd - not clear why

* fix: fix ruff check

* fix: fix test

* fix(model_connection_test.tsx): fix linting error

* test: update unit tests
2025-04-03 11:48:52 -07:00
Ishaan Jaff
49f2cee5b6 Merge branch 'main' into litellm_metrics_pod_lock_manager 2025-04-02 21:35:55 -07:00
Krish Dholakia
354a75fb59 Squashed commit of the following: (#9709)
commit b12a9892b7
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Apr 2 08:09:56 2025 -0700

    fix(utils.py): don't modify openai_token_counter

commit 294de31803
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 21:22:40 2025 -0700

    fix: fix linting error

commit cb6e9fbe40
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 19:52:45 2025 -0700

    refactor: complete migration

commit bfc159172d
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 19:09:59 2025 -0700

    refactor: refactor more constants

commit 43ffb6a558
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:45:24 2025 -0700

    fix: test

commit 04dbe4310c
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:28:58 2025 -0700

    refactor: refactor: move more constants into constants.py

commit 3c26284aff
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:14:46 2025 -0700

    refactor: migrate hardcoded constants out of __init__.py

commit c11e0de69d
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:11:21 2025 -0700

    build: migrate all constants into constants.py

commit 7882bdc787
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:07:37 2025 -0700

    build: initial test banning hardcoded numbers in repo
2025-04-02 21:24:54 -07:00
Ishaan Jaff
db890abe18 Merge branch 'main' into litellm_metrics_pod_lock_manager 2025-04-02 21:04:44 -07:00
Ishaan Jaff
fd4e397da7 prom emit size of DB TX queues for observability 2025-04-02 18:39:29 -07:00
Ishaan Jaff
1165758740 clean up service metrics 2025-04-02 17:50:41 -07:00
Ishaan Jaff
6a1552ee03 emit lock acquired and released events 2025-04-02 17:40:25 -07:00
Ishaan Jaff
bd26fc05fe fix pod lock manager 2025-04-02 14:52:55 -07:00
Ishaan Jaff
4a70e9ad8b test pod lock manager 2025-04-02 14:39:40 -07:00
Ishaan Jaff
19914aad11 docs new deadlock fixing architecture 2025-04-02 13:24:53 -07:00
Krish Dholakia
0519c0c507 Add Google AI Studio /v1/files upload API support (#9645)
* test: fix import for test

* fix: fix bad error string

* docs: cleanup files docs

* fix(files/main.py): cleanup error string

* style: initial commit with a provider/config pattern for files api

google ai studio files api onboarding

* fix: test

* feat(gemini/files/transformation.py): support gemini files api response transformation

* fix(gemini/files/transformation.py): return file id as gemini uri

allows id to be passed in to chat completion request, just like openai

* feat(llm_http_handler.py): support async route for files api on llm_http_handler

* fix: fix linting errors

* fix: fix model info check

* fix: fix ruff errors

* fix: fix linting errors

* Revert "fix: fix linting errors"

This reverts commit 926a5a527f.

* fix: fix linting errors

* test: fix test

* test: fix tests
2025-04-02 08:56:58 -07:00
Krish Dholakia
d1abb9b68b fix(gemini/): add gemini/ route optional param mapping support (#9677)
Fixes https://github.com/BerriAI/litellm/issues/9654
2025-04-02 08:56:32 -07:00
Krish Dholakia
cb4199b441 fix(model_management_endpoints.py): fix allowing team admins to update team models (#9697)
* fix(model_management_endpoints.py): fix allowing team admins to update their models

* test(test_models.py): add e2e test to for team model flow

ensure team admin can always add / edit / delete team models
2025-04-01 22:28:15 -07:00
Krish Dholakia
5a3eab0247 Litellm user daily activity allow non admin usage (#9695)
* feat(internal_user_endpoints.py): allow non-admin to view their own usage via `/user/daily/activity` route

* fix(leftnav.tsx): allow users to view their own usage via new_usage.tsx

allows internal users to see their usage via new api

Handles 1m+ spend logs scenario

* fix(leftnav.tsx): allow all users to see new usage tab
2025-04-01 22:27:26 -07:00
Tomer Bin
e848caaa4d Virtual key based policies in Aim Guardrails (#9499)
* report key alias to aim

* send litellm version to aim

* Update docs

* blacken

* add docs

* Add info part about virtual keys specific guards

* sort guardrails alphabetically

* fix ruff
2025-04-01 21:57:23 -07:00
Ishaan Jaff
e11dd9a779 Merge pull request #9690 from BerriAI/litellm_fixes_for_deadlock_round_2
[Reliability Fixes] - Ensure no deadlocks occur when updating `DailyUserSpendTransaction`
2025-04-01 20:59:41 -07:00
Krrish Dholakia
d66cba15ae build: new ui build 2025-04-01 19:44:23 -07:00
Krish Dholakia
aa01fb9b34 UI - New Usage Tab fixes (#9696)
* fix(new_usage.tsx): enable smooth scrolling - remove double scroll bars

enables easier viewing of overflowing content

* fix(new_usage.tsx): fix ordering of daily spend

ensure always from earliest to latest date

* feat(internal_user_endpoints.py): return key alias on `/user/daily/activity`

Enables easier consumption on UI

* fix(new_usage.tsx): show key alias on usage tab

* feat(activity_metric.tsx): new activity panel - showing spend per model per day

allows debugging if models are not being tracked

* fix(top_key_view.tsx): use consistent param for selecting key
2025-04-01 19:36:00 -07:00
Ishaan Jaff
4e9bdc9e6e BaseUpdateQueue 2025-04-01 18:31:54 -07:00
Ishaan Jaff
be9e9e6db5 refactor file structure 2025-04-01 18:30:48 -07:00
Ishaan Jaff
69947a8bb4 add debug statement 2025-04-01 17:54:52 -07:00
Ishaan Jaff
f122cabd24 fix spend update queue 2025-04-01 17:49:28 -07:00
Ishaan Jaff
9270823d6c refactor daily spend updates to use new Queue DS 2025-04-01 17:45:19 -07:00
Ishaan Jaff
3d84fd97d9 Merge branch 'main' into litellm_fix_service_account_behavior 2025-04-01 12:04:28 -07:00
Ishaan Jaff
853e89f94e Merge pull request #9673 from BerriAI/litellm_qa_deadlock_fixes
[Reliability] - Ensure new Redis + DB architecture tracks spend accurately
2025-04-01 12:04:03 -07:00
Ishaan Jaff
23167ecde3 remove deprecated logic 2025-04-01 07:55:28 -07:00
Ishaan Jaff
f9a6264152 fix check_if_token_is_service_account 2025-04-01 07:36:35 -07:00