Commit graph

21459 commits

Author SHA1 Message Date
Ishaan Jaff
09e073d57b ui mcp tools 2025-03-29 17:04:32 -07:00
Ishaan Jaff
f2885bfb65 add code example 2025-03-29 16:48:38 -07:00
Ishaan Jaff
79e8bbbfd4 fix types on tools.py 2025-03-29 16:48:15 -07:00
Ishaan Jaff
815263f7bc rename transform_openai_tool_call_request_to_mcp_tool_call_request 2025-03-29 16:28:23 -07:00
Ishaan Jaff
7dd54112cb fix showing list of MCP tools 2025-03-29 16:23:58 -07:00
Ishaan Jaff
fe6c033427 render MCP tools on ui logs page 2025-03-29 16:01:34 -07:00
Ishaan Jaff
08a52f4389 log MCP tool call metadata in SLP 2025-03-29 15:50:13 -07:00
Krish Dholakia
1604f87663
install prisma migration files - connects litellm proxy to litellm's prisma migration files (#9637)
* build(README.md): initial commit adding a separate folder for additional proxy files. Meant to reduce size of core package

* build(litellm-proxy-extras/): new pip package for storing migration files

allows litellm proxy to use migration files, without adding them to core repo

* build(litellm-proxy-extras/): cleanup pyproject.toml

* build: move prisma migration files inside new proxy extras package

* build(run_migration.py): update script to write to correct folder

* build(proxy_cli.py): load in migration files from litellm-proxy-extras

Closes https://github.com/BerriAI/litellm/issues/9558

* build: add MIT license to litellm-proxy-extras

* test: update test

* fix: fix schema

* bump: version 0.1.0 → 0.1.1

* build(publish-proxy-extras.sh): add script for publishing new proxy-extras version

* build(liccheck.ini): add litellm-proxy-extras to authorized packages

* fix(litellm-proxy-extras/utils.py): move prisma migrate logic inside extra proxy pkg

easier since migrations folder already there

* build(pre-commit-config.yaml): add litellm_proxy_extras to ci tests

* docs(config_settings.md): document new env var

* build(pyproject.toml): bump relevant files when litellm-proxy-extras version changed

* build(pre-commit-config.yaml): run poetry check on litellm-proxy-extras as well
2025-03-29 15:27:09 -07:00
Ishaan Jaff
b7b9f9d9da working MCP tool call logging 2025-03-29 15:20:52 -07:00
Ishaan Jaff
a4a083041d working MCP call tool method 2025-03-29 14:42:19 -07:00
Ishaan Jaff
e0cff7595c endpoints to list and call tools 2025-03-29 14:31:35 -07:00
Ishaan Jaff
b381dde9ac basic UI rendering of MCP tools 2025-03-29 14:30:34 -07:00
Ishaan Jaff
1cf5cbab50 REST API endpoint for MCP 2025-03-29 13:35:46 -07:00
Ishaan Jaff
a1ec0dd0e2 add testing mcp server 2025-03-29 12:52:46 -07:00
Ishaan Jaff
6aa660b3f5 mcp server manager 2025-03-29 12:51:16 -07:00
Ishaan Jaff
ec283f7289 use global_mcp_server_manager 2025-03-29 12:46:13 -07:00
Krrish Dholakia
aa2489d74f build(.gitignore): update gitignore
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 15s
Helm unit test / unit-test (push) Successful in 46s
2025-03-29 11:37:00 -07:00
Ishaan Jaff
fe3623d775 init mcp client manager 2025-03-29 11:26:18 -07:00
Krish Dholakia
9b7ebb6a7d
build(pyproject.toml): add new dev dependencies - for type checking (#9631)
* build(pyproject.toml): add new dev dependencies - for type checking

* build: reformat files to fit black

* ci: reformat to fit black

* ci(test-litellm.yml): make tests run clear

* build(pyproject.toml): add ruff

* fix: fix ruff checks

* build(mypy/): fix mypy linting errors

* fix(hashicorp_secret_manager.py): fix passing cert for tls auth

* build(mypy/): resolve all mypy errors

* test: update test

* fix: fix black formatting

* build(pre-commit-config.yaml): use poetry run black

* fix(proxy_server.py): fix linting error

* fix: fix ruff safe representation error
2025-03-29 11:02:13 -07:00
Krrish Dholakia
95e5dfae5a build(ci_cd/): add migration script 2025-03-29 08:48:48 -07:00
Krrish Dholakia
217e8d7d44 test: make script to run clearer 2025-03-29 08:23:18 -07:00
Krrish Dholakia
2a0c7e7d20 bump: version 1.65.0 → 1.65.1
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 42s
Helm unit test / unit-test (push) Successful in 51s
2025-03-29 00:39:18 -07:00
Krrish Dholakia
2fcfabd66f docs(bedrock.md): clarify version on docs 2025-03-29 00:31:35 -07:00
Krrish Dholakia
cac60d5091 docs(bedrock.md): add latency optimized inference to docs 2025-03-29 00:31:06 -07:00
Krish Dholakia
5ac61a7572
Add bedrock latency optimized inference support (#9623)
* fix(converse_transformation.py): add performanceConfig param support on bedrock

Closes https://github.com/BerriAI/litellm/issues/7606

* fix(converse_transformation.py): refactor to use more flexible single getter for params which are separate config blocks

* test(test_main.py): add e2e mock test for bedrock performance config

* build(model_prices_and_context_window.json): add versioned multimodal embedding

* refactor(multimodal_embeddings/): migrate to config pattern

* feat(vertex_ai/multimodalembeddings): calculate usage for multimodal embedding calls

Enables cost calculation for multimodal embeddings

* feat(vertex_ai/multimodalembeddings): get usage object for embedding calls

ensures accurate cost tracking for vertexai multimodal embedding calls

* fix(embedding_handler.py): remove unused imports

* fix: fix linting errors

* fix: handle response api usage calculation

* test(test_vertex_ai_multimodal_embedding_transformation.py): update tests

* test: mark flaky test

* feat(vertex_ai/multimodal_embeddings/transformation.py): support text+image+video input

* docs(vertex.md): document sending text + image to vertex multimodal embeddings

* test: remove incorrect file

* fix(multimodal_embeddings/transformation.py): fix linting error

* style: remove unused import
2025-03-29 00:23:09 -07:00
Ishaan Jaff
0742e6afd6
Merge pull request #9608 from BerriAI/litellm_use_redis_for_updates
Some checks failed
Read Version from pyproject.toml / read-version (push) Successful in 16s
Helm unit test / unit-test (push) Successful in 19s
Publish Prisma Migrations / publish-migrations (push) Failing after 1m14s
[Reliability] - Reduce DB Deadlocks by storing spend updates in Redis and then committing to DB
2025-03-28 21:47:45 -07:00
Ishaan Jaff
7c93b19b00 add migration.sql 2025-03-28 21:47:26 -07:00
Ishaan Jaff
c95fb6c692 MAX_REDIS_BUFFER_DEQUEUE_COUNT 2025-03-28 20:59:28 -07:00
Ishaan Jaff
29d6968f58 fix linting 2025-03-28 20:57:28 -07:00
Ishaan Jaff
4fb9d27a35 use constants for redis buffer in DB 2025-03-28 20:55:43 -07:00
Krish Dholakia
4879e1ecf0
fix(openrouter/chat/transformation.py): raise informative message for openrouter key error (#9626)
Related Issue: https://github.com/Aider-AI/aider/issues/3550#issuecomment-2763052355
2025-03-28 20:24:28 -07:00
Ishaan Jaff
7e8a02099c Merge branch 'main' into litellm_use_redis_for_updates 2025-03-28 20:12:29 -07:00
Ishaan Jaff
69d5734685 fix - locking in memory leads to failing tests 2025-03-28 20:10:58 -07:00
Ishaan Jaff
ba550e2147 test local spend accuracy 2025-03-28 19:52:39 -07:00
Ishaan Jaff
183c33bb02 prisma client in_memory_transaction_lock 2025-03-28 19:52:07 -07:00
Ishaan Jaff
21bf15263c use asyncio lock for updating PrismaClient txs 2025-03-28 19:51:15 -07:00
Ishaan Jaff
eae2adcee0 redis cache add async push and pop methods 2025-03-28 19:49:38 -07:00
Ishaan Jaff
88458a6568 redis update buffer queue 2025-03-28 17:24:54 -07:00
Ishaan Jaff
e5e0ffa28a add async_rpush 2025-03-28 17:23:47 -07:00
Ishaan Jaff
cec5280e03 debugging for Redis TX management 2025-03-28 16:16:44 -07:00
NickGrab
220d4c07f4
Merge pull request #9625 from BerriAI/litellm_mar_28_vertex_fix
Add support to Vertex AI transformation for anyOf union type with null fields
2025-03-28 16:09:29 -07:00
Krish Dholakia
222898d727
Fix anthropic thinking + response_format (#9594)
* fix(anthropic/chat/transformation.py): Don't set tool choice on response_format conversion when thinking is enabled

Not allowed by Anthropic

Fixes https://github.com/BerriAI/litellm/issues/8901

* refactor: move test to base anthropic chat tests

ensures consistent behaviour across vertex/anthropic/bedrock

* fix(anthropic/chat/transformation.py): if thinking token is specified and max tokens is not - ensure max token to anthropic is higher than thinking tokens

* feat(converse_transformation.py): correctly handle thinking + response format on Bedrock Converse

Fixes https://github.com/BerriAI/litellm/issues/8901

* fix(converse_transformation.py): correctly handle adding max tokens

* test: handle service unavailable error
2025-03-28 15:57:40 -07:00
Ishaan Jaff
df136c5689 add docstring for DBSpendUpdateTransactions 2025-03-28 15:20:28 -07:00
Nicholas Grabar
09daeac188 Rebasing 2 2025-03-28 15:18:09 -07:00
Nicholas Grabar
06a45706b2 Rebase 3 2025-03-28 15:18:05 -07:00
Ishaan Jaff
7c73987066 fix loc of DBSpendUpdateTransactions 2025-03-28 15:17:01 -07:00
Krrish Dholakia
8c9ff23e19 test(test_caching_handler.py): move to in-memory cache - prevent redis flakiness from impacting ci/cd 2025-03-28 15:16:15 -07:00
Dbzman
9e4165ac75 fix: wrong indentation for ttlSecondsAfterFinished (#9611) 2025-03-28 15:16:15 -07:00
Krish Dholakia
1a441def03 fix(logging): add json formatting for uncaught exceptions (#9615) (#9619)
* fix(logging): add json formatting for uncaught exceptions (#9615)

* fix(_logging.py): cleanup logging to catch unhandled exceptions

* fix(_logging.py): avoid using 'print' '

---------

Co-authored-by: Henrique Cavarsan <hcavarsan@yahoo.com.br>
2025-03-28 15:16:15 -07:00
Krish Dholakia
205db622bf fix(proxy_server.py): get master key from environment, if not set in … (#9617)
* fix(proxy_server.py): get master key from environment, if not set in general settings or general settings not set at all

* test: mark flaky test

* test(test_proxy_server.py): mock prisma client

* ci: add new github workflow for testing just the mock tests

* fix: fix linting error

* ci(conftest.py): add conftest.py to isolate proxy tests

* build(pyproject.toml): add respx to dev dependencies

* build(pyproject.toml): add prisma to dev dependencies

* test: fix mock prompt management tests to use a mock anthropic key

* ci(test-litellm.yml): parallelize mock testing

make it run faster

* build(pyproject.toml): add hypercorn as dev dep

* build(pyproject.toml): separate proxy vs. core dev dependencies

make it easier for non-proxy contributors to run tests locally - e.g. no need to install hypercorn

* ci(test-litellm.yml): pin python version

* test(test_rerank.py): move test - cannot be mocked, requires aws credentials for e2e testing

* ci: add thank you message to ci

* test: add mock env var to test

* test: add autouse to tests

* test: test mock env vars for e2e tests
2025-03-28 15:16:15 -07:00