Commit graph

4490 commits

Author SHA1 Message Date
Krish Dholakia
c7f1d62991 fix(logging): add json formatting for uncaught exceptions (#9615) (#9619)
* fix(logging): add json formatting for uncaught exceptions (#9615)

* fix(_logging.py): cleanup logging to catch unhandled exceptions

* fix(_logging.py): avoid using 'print' '

---------

Co-authored-by: Henrique Cavarsan <hcavarsan@yahoo.com.br>
2025-03-28 15:16:15 -07:00
Krish Dholakia
7470cb5599 fix(proxy_server.py): get master key from environment, if not set in … (#9617)
* fix(proxy_server.py): get master key from environment, if not set in general settings or general settings not set at all

* test: mark flaky test

* test(test_proxy_server.py): mock prisma client

* ci: add new github workflow for testing just the mock tests

* fix: fix linting error

* ci(conftest.py): add conftest.py to isolate proxy tests

* build(pyproject.toml): add respx to dev dependencies

* build(pyproject.toml): add prisma to dev dependencies

* test: fix mock prompt management tests to use a mock anthropic key

* ci(test-litellm.yml): parallelize mock testing

make it run faster

* build(pyproject.toml): add hypercorn as dev dep

* build(pyproject.toml): separate proxy vs. core dev dependencies

make it easier for non-proxy contributors to run tests locally - e.g. no need to install hypercorn

* ci(test-litellm.yml): pin python version

* test(test_rerank.py): move test - cannot be mocked, requires aws credentials for e2e testing

* ci: add thank you message to ci

* test: add mock env var to test

* test: add autouse to tests

* test: test mock env vars for e2e tests
2025-03-28 15:16:15 -07:00
Nicholas Grabar
36458f5eb5 Add recursion depth to convert_anyof_null_to_nullable, constants.py. Fix recursive_detector.py raise error state 2025-03-28 13:11:19 -07:00
Krrish Dholakia
8542d26c99 fix: cleanup 2025-03-27 23:31:12 -07:00
Krish Dholakia
7c4944cb5b Connect UI to "LiteLLM_DailyUserSpend" spend table - enables usage tab to work at 1m+ spend logs (#9603)
* feat(spend_management_endpoints.py): expose new endpoint for querying user's usage at 1m+ spend logs

Allows user to view their spend at 1m+ spend logs

* build(schema.prisma): add api_requests to dailyuserspend table

* build(migration.sql): add migration file for new column to daily user spend table

* build(prisma_client.py): add logic for copying over migration folder, if deploy/migrations present in expected location

enables easier testing of prisma migration flow

* build(ui/): initial commit successfully using the dailyuserspend table on the UI

* refactor(internal_user_endpoints.py): refactor `/user/daily/activity` to give breakdowns by provider/model/key

* feat: feature parity (cost page) with existing 'usage' page

* build(ui/): add activity tab to new_usage.tsx

gets to feature parity on 'All Up' page of 'usage.tsx'

* fix(proxy/utils.py): count number of api requests in daily user spend table

allows us to see activity by model on new usage tab

* style(new_usage.tsx): fix y-axis to be in ascending order of date

* fix: fix linting errors

* fix: fix ruff check errors
2025-03-27 23:29:15 -07:00
Krrish Dholakia
37c3414095 fix(proxy_server.py): Fix https://github.com/BerriAI/litellm/issues/9576 2025-03-27 23:25:46 -07:00
Krish Dholakia
2a131e5284 Litellm new UI build (#9601)
* build: new ui build

* build: new ui build

* fix(proxy_server.py): only show user models their key can access on `/models`

* fix(model_management_endpoints.py): ensure team admin can add models

* test: update unit testing to reflect changes

* fix(model_dashboard.tsx): fix sizing on models page

* build: fix ui
2025-03-27 17:15:25 -07:00
Krish Dholakia
ed8c63b51e Allow team admins to add/update/delete models on UI + show api base and model id on request logs (#9572)
* feat(view_logs.tsx): show model id + api base in request logs

easier debugging

* fix(index.tsx): fix length of api base

easier viewing

* refactor(leftnav.tsx): show models tab to team admin

* feat(model_dashboard.tsx): add explainer for what the 'models' page is for team admin

helps them understand how they can use it

* feat(model_management_endpoints.py): restrict model add by team to just team admin

allow team admin to add models via non-team keys (e.g. ui token)

* test(test_add_update_models.py): update unit testing for new behaviour

* fix(model_dashboard.tsx): show user the models

* feat(proxy_server.py): add new query param 'user_models_only' to `/v2/model/info`

Allows user to retrieve just the models they've added

Used in UI to show internal users just the models they've added

* feat(model_dashboard.tsx): allow team admins to view their own models

* fix: allow ui user to fetch model cost map

* feat(add_model_tab.tsx): require team admins to specify team when onboarding models

* fix(_types.py): add `/v1/model/info` to info route

`/model/info` was already there

* fix(model_info_view.tsx): allow user to edit a model they created

* fix(model_management_endpoints.py): allow team admin to update team model

* feat(model_managament_endpoints.py): allow team admin to delete team models

* fix(model_management_endpoints.py): don't require team id to be set when adding a model

* fix(proxy_server.py): fix linting error

* fix: fix ui linting error

* fix(model_management_endpoints.py): ensure consistent auth checks on all model calls

* test: remove old test - function no longer exists in same form

* test: add updated mock testing
2025-03-27 12:06:31 -07:00
Krish Dholakia
d58fe5a9f9 Add OpenAI gpt-4o-transcribe support (#9517)
* refactor: introduce new transformation config for gpt-4o-transcribe models

* refactor: expose new transformation configs for audio transcription

* ci: fix config yml

* feat(openai/transcriptions): support provider config transformation on openai audio transcriptions

allows gpt-4o and whisper audio transformation to work as expected

* refactor: migrate fireworks ai + deepgram to new transform request pattern

* feat(openai/): working support for gpt-4o-audio-transcribe

* build(model_prices_and_context_window.json): add gpt-4o-transcribe to model cost map

* build(model_prices_and_context_window.json): specify what endpoints are supported for `/audio/transcriptions`

* fix(get_supported_openai_params.py): fix return

* refactor(deepgram/): migrate unit test to deepgram handler

* refactor: cleanup unused imports

* fix(get_supported_openai_params.py): fix linting error

* test: update test
2025-03-26 23:10:25 -07:00
Ishaan Jaff
155a53c426 fix order of _setup_prisma_client 2025-03-26 20:42:01 -07:00
Ishaan Jaff
e46c789bb7 fix _setup_prisma_client 2025-03-26 20:35:27 -07:00
Ishaan Jaff
6f3be086b5 refactor tests 2025-03-26 20:00:10 -07:00
Ishaan Jaff
d1ff7a5f45 allow proxy to startup on DB unavailable 2025-03-26 19:50:57 -07:00
Ishaan Jaff
3d4fc22eaf bug fix - allow pods to startup when DB is unavailable 2025-03-26 19:49:51 -07:00
Ishaan Jaff
6648e85c97 refactor PrismaDBExceptionHandler 2025-03-26 19:22:24 -07:00
Ishaan Jaff
cb44f8d64d Merge pull request #9533 from BerriAI/litellm_stability_fixes
[Reliability Fixes] - Gracefully handle exceptions when DB is having an outage
2025-03-26 18:57:38 -07:00
Krish Dholakia
eeab48ce5a Add Daily User Spend Aggregate view - allows UI Usage tab to work > 1m rows (#9538)
* ci: update github action

* build(schema.prisma): enable daily user spend table

allows storing aggregate view of user's daily spend

* build(schema.prisma): add new daily user spend table

* feat: working daily user spend tracking

maintains an aggregate view for easier querying in high traffic

* setup_google_dns

* ci: update ci yaml

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-03-26 16:36:36 -07:00
Ishaan Jaff
1274d27534 fix ProxyException 2025-03-26 15:53:33 -07:00
Ishaan Jaff
ba0d8a6d22 fix auth checks 2025-03-26 15:45:58 -07:00
Ishaan Jaff
427b549975 fix get_key_object 2025-03-26 15:41:40 -07:00
Ishaan Jaff
43c402b8b2 is_database_connection_error 2025-03-26 14:55:40 -07:00
Ishaan Jaff
f85c0dac4f Merge branch 'main' into litellm_stability_fixes 2025-03-26 14:41:43 -07:00
Ishaan Jaff
ad5d26f664 fix user_api_key_auth example config 2025-03-26 08:36:11 -07:00
Krish Dholakia
1ab5d49132 Support litellm.api_base for vertex_ai + gemini/ across completion, embedding, image_generation (#9516)
* test(tests): add unit testing for litellm_proxy integration

* fix(cost_calculator.py): fix tracking cost in sdk when calling proxy

* fix(main.py): respect litellm.api_base on `vertex_ai/` and `gemini/` routes

* fix(main.py): consistently support custom api base across gemini + vertexai on embedding + completion

* feat(vertex_ai/): test

* fix: fix linting error

* test: set api base as None before starting loadtest
2025-03-25 23:46:20 -07:00
Ishaan Jaff
fea5213517 add test config 2025-03-25 19:21:51 -07:00
Ishaan Jaff
6493a6422d fixes for auth checks 2025-03-25 15:44:13 -07:00
Ishaan Jaff
ea209661d5 fix ProxyErrorTypes 2025-03-25 14:40:11 -07:00
Ishaan Jaff
74e3600367 UserAPIKeyAuthExceptionHandler 2025-03-25 14:07:14 -07:00
Krish Dholakia
f09f4a6396 Merge pull request #9512 from BerriAI/litellm_dev_03_24_2025_p3
fix(invoke_handler.py): remove hard coded chunk on streaming usage
2025-03-24 21:21:36 -07:00
Ishaan Jaff
a61925448a fix import mcp router 2025-03-24 21:08:24 -07:00
Ishaan Jaff
dc899570ce fix code quality 2025-03-24 21:06:17 -07:00
Ishaan Jaff
a616375b3c fix mcp import 2025-03-24 21:02:54 -07:00
Ishaan Jaff
c6424d6246 Merge branch 'main' into litellm_exp_mcp_server 2025-03-24 19:03:56 -07:00
Ishaan Jaff
f600899ff7 fix linting errors 2025-03-24 18:48:43 -07:00
Krrish Dholakia
017a9b882a fix(invoke_handler.py): remove hard code 2025-03-24 17:58:26 -07:00
Krrish Dholakia
2f01caa919 fix(litellm_logging.py): always log the api base
Fixes issue where api base missing from spend logs due to refactor
2025-03-24 13:45:39 -07:00
Ishaan Jaff
d4b53d3fa9 fix mcp type imports 2025-03-24 11:01:29 -07:00
Ishaan Jaff
f133bb07d1 fix pydantic import error 2025-03-24 07:11:48 -07:00
Tyler Hutcherson
785f9714ca update redisvl dependency 2025-03-24 08:42:11 -04:00
Ishaan Jaff
d932206bfb Merge branch 'main' into litellm_exp_mcp_server 2025-03-22 18:51:25 -07:00
Krrish Dholakia
2e5146855e fix: fix linting error 2025-03-22 17:56:39 -07:00
Krrish Dholakia
c568952490 fix(router.py): fix get_model_list to return all wildcard models
enables viewing all wildcard models on `/model/info`
2025-03-22 15:39:23 -07:00
Krrish Dholakia
c45a61940d fix(llm_passthrough_endpoints.py): fix raising helpful debug error message 2025-03-22 15:06:16 -07:00
Krrish Dholakia
7bd9c5cd9e build: cleanup unused files 2025-03-22 12:54:11 -07:00
Krrish Dholakia
2d5c7f809d fix(llm_passthrough_endpoints.py): raise verbose error if credentials not found on proxy 2025-03-22 11:49:51 -07:00
Krrish Dholakia
851c993063 fix(vertex_ai/common_utils.py): fix handling constructed url with default vertex config 2025-03-22 11:32:01 -07:00
Krrish Dholakia
0145ebd343 feat(llm_passthrough_endpoints.py): base case passing for refactored vertex passthrough route 2025-03-22 11:06:52 -07:00
Krrish Dholakia
e99a22b9ac refactor(llm_passthrough_endpoints.py): refactor vertex passthrough to use common llm passthrough handler.py 2025-03-22 10:42:46 -07:00
Krrish Dholakia
12046a1a2e fix: fix linting error 2025-03-21 18:23:58 -07:00
Krrish Dholakia
4a88ba1bc8 fix: fix linting error 2025-03-21 18:08:00 -07:00