Commit graph

4546 commits

Author SHA1 Message Date
Ishaan Jaff
518723bfc7 docs mcp litellm 2025-03-29 21:59:58 -07:00
Ishaan Jaff
44ff6991b5 docs mcp tools 2025-03-29 20:42:14 -07:00
Ishaan Jaff
f5c0afcf96 Merge pull request #9642 from BerriAI/litellm_mcp_improvements_expose_sse_urls
[Feat] - MCP improvements, add support for using SSE MCP servers
2025-03-29 19:37:57 -07:00
Krish Dholakia
70f993d3d7 Add gemini audio input support + handle special tokens in sagemaker response (#9640)
* fix(internal_user_endpoints.py): cleanup unused variables on beta endpoint

no team/org split on daily user endpoint

* build(model_prices_and_context_window.json): gemini-2.0-flash supports audio input

* feat(gemini/transformation.py): support passing audio input to gemini

* test: fix test

* fix(gemini/transformation.py): support audio input as a url

enables passing google cloud bucket urls

* fix(gemini/transformation.py): support explicitly passing format of file

* fix(gemini/transformation.py): expand support for inferred file types from url

* fix(sagemaker/completion/transformation.py): fix special token error when counting sagemaker tokens

* test: fix import
2025-03-29 19:23:09 -07:00
Ishaan Jaff
198fb23d0c fix listing mcp tools 2025-03-29 18:40:58 -07:00
Ishaan Jaff
dde78a8ae9 fix order of imports 2025-03-29 18:32:06 -07:00
Ishaan Jaff
70e67c872b list_tool_rest_api 2025-03-29 17:55:37 -07:00
Ishaan Jaff
55ad7cbda0 fix import errors without mcp 2025-03-29 17:44:32 -07:00
Ishaan Jaff
6fa5de6353 fix showing list of MCP tools 2025-03-29 16:23:58 -07:00
Ishaan Jaff
759303687a render MCP tools on ui logs page 2025-03-29 16:01:34 -07:00
Ishaan Jaff
2dfd302a82 log MCP tool call metadata in SLP 2025-03-29 15:50:13 -07:00
Krish Dholakia
797a4bb1d5 install prisma migration files - connects litellm proxy to litellm's prisma migration files (#9637)
* build(README.md): initial commit adding a separate folder for additional proxy files. Meant to reduce size of core package

* build(litellm-proxy-extras/): new pip package for storing migration files

allows litellm proxy to use migration files, without adding them to core repo

* build(litellm-proxy-extras/): cleanup pyproject.toml

* build: move prisma migration files inside new proxy extras package

* build(run_migration.py): update script to write to correct folder

* build(proxy_cli.py): load in migration files from litellm-proxy-extras

Closes https://github.com/BerriAI/litellm/issues/9558

* build: add MIT license to litellm-proxy-extras

* test: update test

* fix: fix schema

* bump: version 0.1.0 → 0.1.1

* build(publish-proxy-extras.sh): add script for publishing new proxy-extras version

* build(liccheck.ini): add litellm-proxy-extras to authorized packages

* fix(litellm-proxy-extras/utils.py): move prisma migrate logic inside extra proxy pkg

easier since migrations folder already there

* build(pre-commit-config.yaml): add litellm_proxy_extras to ci tests

* docs(config_settings.md): document new env var

* build(pyproject.toml): bump relevant files when litellm-proxy-extras version changed

* build(pre-commit-config.yaml): run poetry check on litellm-proxy-extras as well
2025-03-29 15:27:09 -07:00
Ishaan Jaff
f1e1cdf730 working MCP tool call logging 2025-03-29 15:20:52 -07:00
Ishaan Jaff
db026d385c working MCP call tool method 2025-03-29 14:42:19 -07:00
Ishaan Jaff
7d00abc37f endpoints to list and call tools 2025-03-29 14:31:35 -07:00
Ishaan Jaff
6c7b42f575 REST API endpoint for MCP 2025-03-29 13:35:46 -07:00
Ishaan Jaff
790febde47 add testing mcp server 2025-03-29 12:52:46 -07:00
Ishaan Jaff
061f98c570 mcp server manager 2025-03-29 12:51:16 -07:00
Ishaan Jaff
c0d07987c4 use global_mcp_server_manager 2025-03-29 12:46:13 -07:00
Ishaan Jaff
ecc31e7899 init mcp client manager 2025-03-29 11:26:18 -07:00
Krish Dholakia
d7b294dd0a build(pyproject.toml): add new dev dependencies - for type checking (#9631)
* build(pyproject.toml): add new dev dependencies - for type checking

* build: reformat files to fit black

* ci: reformat to fit black

* ci(test-litellm.yml): make tests run clear

* build(pyproject.toml): add ruff

* fix: fix ruff checks

* build(mypy/): fix mypy linting errors

* fix(hashicorp_secret_manager.py): fix passing cert for tls auth

* build(mypy/): resolve all mypy errors

* test: update test

* fix: fix black formatting

* build(pre-commit-config.yaml): use poetry run black

* fix(proxy_server.py): fix linting error

* fix: fix ruff safe representation error
2025-03-29 11:02:13 -07:00
Ishaan Jaff
2f059f5028 MAX_REDIS_BUFFER_DEQUEUE_COUNT 2025-03-28 20:59:28 -07:00
Ishaan Jaff
64bc3f0c66 fix linting 2025-03-28 20:57:28 -07:00
Ishaan Jaff
0944640d6b Merge branch 'main' into litellm_use_redis_for_updates 2025-03-28 20:12:29 -07:00
Ishaan Jaff
93c20b8d2d fix - locking in memory leads to failing tests 2025-03-28 20:10:58 -07:00
Ishaan Jaff
fb98bd3a4a test local spend accuracy 2025-03-28 19:52:39 -07:00
Ishaan Jaff
96e4369160 prisma client in_memory_transaction_lock 2025-03-28 19:52:07 -07:00
Ishaan Jaff
96812a7da0 use asyncio lock for updating PrismaClient txs 2025-03-28 19:51:15 -07:00
Ishaan Jaff
7a63cbe8d0 redis update buffer queue 2025-03-28 17:24:54 -07:00
Ishaan Jaff
36f15f19af debugging for Redis TX management 2025-03-28 16:16:44 -07:00
Ishaan Jaff
b4e9683889 add docstring for DBSpendUpdateTransactions 2025-03-28 15:20:28 -07:00
Ishaan Jaff
cc9f47b1d2 fix loc of DBSpendUpdateTransactions 2025-03-28 15:17:01 -07:00
Krish Dholakia
c7f1d62991 fix(logging): add json formatting for uncaught exceptions (#9615) (#9619)
* fix(logging): add json formatting for uncaught exceptions (#9615)

* fix(_logging.py): cleanup logging to catch unhandled exceptions

* fix(_logging.py): avoid using 'print' '

---------

Co-authored-by: Henrique Cavarsan <hcavarsan@yahoo.com.br>
2025-03-28 15:16:15 -07:00
Krish Dholakia
7470cb5599 fix(proxy_server.py): get master key from environment, if not set in … (#9617)
* fix(proxy_server.py): get master key from environment, if not set in general settings or general settings not set at all

* test: mark flaky test

* test(test_proxy_server.py): mock prisma client

* ci: add new github workflow for testing just the mock tests

* fix: fix linting error

* ci(conftest.py): add conftest.py to isolate proxy tests

* build(pyproject.toml): add respx to dev dependencies

* build(pyproject.toml): add prisma to dev dependencies

* test: fix mock prompt management tests to use a mock anthropic key

* ci(test-litellm.yml): parallelize mock testing

make it run faster

* build(pyproject.toml): add hypercorn as dev dep

* build(pyproject.toml): separate proxy vs. core dev dependencies

make it easier for non-proxy contributors to run tests locally - e.g. no need to install hypercorn

* ci(test-litellm.yml): pin python version

* test(test_rerank.py): move test - cannot be mocked, requires aws credentials for e2e testing

* ci: add thank you message to ci

* test: add mock env var to test

* test: add autouse to tests

* test: test mock env vars for e2e tests
2025-03-28 15:16:15 -07:00
Ishaan Jaff
fa304a538d use helper methods _commit_spend_updates_to_db_without_redis_buffer 2025-03-28 15:12:19 -07:00
Ishaan Jaff
8dd2863d71 test pod lock manager 2025-03-28 13:31:45 -07:00
Nicholas Grabar
36458f5eb5 Add recursion depth to convert_anyof_null_to_nullable, constants.py. Fix recursive_detector.py raise error state 2025-03-28 13:11:19 -07:00
Ishaan Jaff
17d27fcd36 rename pod lock manager 2025-03-28 12:57:00 -07:00
Krrish Dholakia
8542d26c99 fix: cleanup 2025-03-27 23:31:12 -07:00
Ishaan Jaff
2f5d7b86d9 fix PodLockManager 2025-03-27 23:29:48 -07:00
Krish Dholakia
7c4944cb5b Connect UI to "LiteLLM_DailyUserSpend" spend table - enables usage tab to work at 1m+ spend logs (#9603)
* feat(spend_management_endpoints.py): expose new endpoint for querying user's usage at 1m+ spend logs

Allows user to view their spend at 1m+ spend logs

* build(schema.prisma): add api_requests to dailyuserspend table

* build(migration.sql): add migration file for new column to daily user spend table

* build(prisma_client.py): add logic for copying over migration folder, if deploy/migrations present in expected location

enables easier testing of prisma migration flow

* build(ui/): initial commit successfully using the dailyuserspend table on the UI

* refactor(internal_user_endpoints.py): refactor `/user/daily/activity` to give breakdowns by provider/model/key

* feat: feature parity (cost page) with existing 'usage' page

* build(ui/): add activity tab to new_usage.tsx

gets to feature parity on 'All Up' page of 'usage.tsx'

* fix(proxy/utils.py): count number of api requests in daily user spend table

allows us to see activity by model on new usage tab

* style(new_usage.tsx): fix y-axis to be in ascending order of date

* fix: fix linting errors

* fix: fix ruff check errors
2025-03-27 23:29:15 -07:00
Krrish Dholakia
37c3414095 fix(proxy_server.py): Fix https://github.com/BerriAI/litellm/issues/9576 2025-03-27 23:25:46 -07:00
Ishaan Jaff
0a5db39c89 working cron job updates 2025-03-27 23:13:01 -07:00
Ishaan Jaff
39cc75c36b handle commit updates to the DB 2025-03-27 22:55:36 -07:00
Ishaan Jaff
2a181a45f1 fix schema to work with lock acquisition 2025-03-27 22:54:46 -07:00
Ishaan Jaff
a1a038aa19 fix typo on codebase 2025-03-27 22:36:00 -07:00
Ishaan Jaff
3bd005f148 fix db spend update buffer 2025-03-27 22:34:15 -07:00
Ishaan Jaff
9df978b511 fix typo 2025-03-27 22:31:38 -07:00
Ishaan Jaff
081bd58ea5 update the correct set of txs 2025-03-27 21:54:12 -07:00
Ishaan Jaff
9efd1814bd use lock manager for update spend job 2025-03-27 21:43:02 -07:00