Commit graph

21445 commits

Author SHA1 Message Date
Ishaan Jaff
6aa660b3f5 mcp server manager 2025-03-29 12:51:16 -07:00
Ishaan Jaff
ec283f7289 use global_mcp_server_manager 2025-03-29 12:46:13 -07:00
Krrish Dholakia
aa2489d74f build(.gitignore): update gitignore
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 15s
Helm unit test / unit-test (push) Successful in 46s
2025-03-29 11:37:00 -07:00
Ishaan Jaff
fe3623d775 init mcp client manager 2025-03-29 11:26:18 -07:00
Krish Dholakia
9b7ebb6a7d
build(pyproject.toml): add new dev dependencies - for type checking (#9631)
* build(pyproject.toml): add new dev dependencies - for type checking

* build: reformat files to fit black

* ci: reformat to fit black

* ci(test-litellm.yml): make tests run clear

* build(pyproject.toml): add ruff

* fix: fix ruff checks

* build(mypy/): fix mypy linting errors

* fix(hashicorp_secret_manager.py): fix passing cert for tls auth

* build(mypy/): resolve all mypy errors

* test: update test

* fix: fix black formatting

* build(pre-commit-config.yaml): use poetry run black

* fix(proxy_server.py): fix linting error

* fix: fix ruff safe representation error
2025-03-29 11:02:13 -07:00
Krrish Dholakia
95e5dfae5a build(ci_cd/): add migration script 2025-03-29 08:48:48 -07:00
Krrish Dholakia
217e8d7d44 test: make script to run clearer 2025-03-29 08:23:18 -07:00
Krrish Dholakia
2a0c7e7d20 bump: version 1.65.0 → 1.65.1
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 42s
Helm unit test / unit-test (push) Successful in 51s
2025-03-29 00:39:18 -07:00
Krrish Dholakia
2fcfabd66f docs(bedrock.md): clarify version on docs 2025-03-29 00:31:35 -07:00
Krrish Dholakia
cac60d5091 docs(bedrock.md): add latency optimized inference to docs 2025-03-29 00:31:06 -07:00
Krish Dholakia
5ac61a7572
Add bedrock latency optimized inference support (#9623)
* fix(converse_transformation.py): add performanceConfig param support on bedrock

Closes https://github.com/BerriAI/litellm/issues/7606

* fix(converse_transformation.py): refactor to use more flexible single getter for params which are separate config blocks

* test(test_main.py): add e2e mock test for bedrock performance config

* build(model_prices_and_context_window.json): add versioned multimodal embedding

* refactor(multimodal_embeddings/): migrate to config pattern

* feat(vertex_ai/multimodalembeddings): calculate usage for multimodal embedding calls

Enables cost calculation for multimodal embeddings

* feat(vertex_ai/multimodalembeddings): get usage object for embedding calls

ensures accurate cost tracking for vertexai multimodal embedding calls

* fix(embedding_handler.py): remove unused imports

* fix: fix linting errors

* fix: handle response api usage calculation

* test(test_vertex_ai_multimodal_embedding_transformation.py): update tests

* test: mark flaky test

* feat(vertex_ai/multimodal_embeddings/transformation.py): support text+image+video input

* docs(vertex.md): document sending text + image to vertex multimodal embeddings

* test: remove incorrect file

* fix(multimodal_embeddings/transformation.py): fix linting error

* style: remove unused import
2025-03-29 00:23:09 -07:00
Ishaan Jaff
0742e6afd6
Merge pull request #9608 from BerriAI/litellm_use_redis_for_updates
Some checks failed
Read Version from pyproject.toml / read-version (push) Successful in 16s
Helm unit test / unit-test (push) Successful in 19s
Publish Prisma Migrations / publish-migrations (push) Failing after 1m14s
[Reliability] - Reduce DB Deadlocks by storing spend updates in Redis and then committing to DB
2025-03-28 21:47:45 -07:00
Ishaan Jaff
7c93b19b00 add migration.sql 2025-03-28 21:47:26 -07:00
Ishaan Jaff
c95fb6c692 MAX_REDIS_BUFFER_DEQUEUE_COUNT 2025-03-28 20:59:28 -07:00
Ishaan Jaff
29d6968f58 fix linting 2025-03-28 20:57:28 -07:00
Ishaan Jaff
4fb9d27a35 use constants for redis buffer in DB 2025-03-28 20:55:43 -07:00
Krish Dholakia
4879e1ecf0
fix(openrouter/chat/transformation.py): raise informative message for openrouter key error (#9626)
Related Issue: https://github.com/Aider-AI/aider/issues/3550#issuecomment-2763052355
2025-03-28 20:24:28 -07:00
Ishaan Jaff
7e8a02099c Merge branch 'main' into litellm_use_redis_for_updates 2025-03-28 20:12:29 -07:00
Ishaan Jaff
69d5734685 fix - locking in memory leads to failing tests 2025-03-28 20:10:58 -07:00
Ishaan Jaff
ba550e2147 test local spend accuracy 2025-03-28 19:52:39 -07:00
Ishaan Jaff
183c33bb02 prisma client in_memory_transaction_lock 2025-03-28 19:52:07 -07:00
Ishaan Jaff
21bf15263c use asyncio lock for updating PrismaClient txs 2025-03-28 19:51:15 -07:00
Ishaan Jaff
eae2adcee0 redis cache add async push and pop methods 2025-03-28 19:49:38 -07:00
Ishaan Jaff
88458a6568 redis update buffer queue 2025-03-28 17:24:54 -07:00
Ishaan Jaff
e5e0ffa28a add async_rpush 2025-03-28 17:23:47 -07:00
Ishaan Jaff
cec5280e03 debugging for Redis TX management 2025-03-28 16:16:44 -07:00
NickGrab
220d4c07f4
Merge pull request #9625 from BerriAI/litellm_mar_28_vertex_fix
Add support to Vertex AI transformation for anyOf union type with null fields
2025-03-28 16:09:29 -07:00
Krish Dholakia
222898d727
Fix anthropic thinking + response_format (#9594)
* fix(anthropic/chat/transformation.py): Don't set tool choice on response_format conversion when thinking is enabled

Not allowed by Anthropic

Fixes https://github.com/BerriAI/litellm/issues/8901

* refactor: move test to base anthropic chat tests

ensures consistent behaviour across vertex/anthropic/bedrock

* fix(anthropic/chat/transformation.py): if thinking token is specified and max tokens is not - ensure max token to anthropic is higher than thinking tokens

* feat(converse_transformation.py): correctly handle thinking + response format on Bedrock Converse

Fixes https://github.com/BerriAI/litellm/issues/8901

* fix(converse_transformation.py): correctly handle adding max tokens

* test: handle service unavailable error
2025-03-28 15:57:40 -07:00
Ishaan Jaff
df136c5689 add docstring for DBSpendUpdateTransactions 2025-03-28 15:20:28 -07:00
Nicholas Grabar
09daeac188 Rebasing 2 2025-03-28 15:18:09 -07:00
Nicholas Grabar
06a45706b2 Rebase 3 2025-03-28 15:18:05 -07:00
Ishaan Jaff
7c73987066 fix loc of DBSpendUpdateTransactions 2025-03-28 15:17:01 -07:00
Krrish Dholakia
8c9ff23e19 test(test_caching_handler.py): move to in-memory cache - prevent redis flakiness from impacting ci/cd 2025-03-28 15:16:15 -07:00
Dbzman
9e4165ac75 fix: wrong indentation for ttlSecondsAfterFinished (#9611) 2025-03-28 15:16:15 -07:00
Krish Dholakia
1a441def03 fix(logging): add json formatting for uncaught exceptions (#9615) (#9619)
* fix(logging): add json formatting for uncaught exceptions (#9615)

* fix(_logging.py): cleanup logging to catch unhandled exceptions

* fix(_logging.py): avoid using 'print' '

---------

Co-authored-by: Henrique Cavarsan <hcavarsan@yahoo.com.br>
2025-03-28 15:16:15 -07:00
Krish Dholakia
205db622bf fix(proxy_server.py): get master key from environment, if not set in … (#9617)
* fix(proxy_server.py): get master key from environment, if not set in general settings or general settings not set at all

* test: mark flaky test

* test(test_proxy_server.py): mock prisma client

* ci: add new github workflow for testing just the mock tests

* fix: fix linting error

* ci(conftest.py): add conftest.py to isolate proxy tests

* build(pyproject.toml): add respx to dev dependencies

* build(pyproject.toml): add prisma to dev dependencies

* test: fix mock prompt management tests to use a mock anthropic key

* ci(test-litellm.yml): parallelize mock testing

make it run faster

* build(pyproject.toml): add hypercorn as dev dep

* build(pyproject.toml): separate proxy vs. core dev dependencies

make it easier for non-proxy contributors to run tests locally - e.g. no need to install hypercorn

* ci(test-litellm.yml): pin python version

* test(test_rerank.py): move test - cannot be mocked, requires aws credentials for e2e testing

* ci: add thank you message to ci

* test: add mock env var to test

* test: add autouse to tests

* test: test mock env vars for e2e tests
2025-03-28 15:16:15 -07:00
Ishaan Jaff
3df839d878 use helper methods _commit_spend_updates_to_db_without_redis_buffer 2025-03-28 15:12:19 -07:00
Ishaan Jaff
193052ed70 test pod lock manager 2025-03-28 15:05:17 -07:00
Krrish Dholakia
28a9edb547 test(test_caching_handler.py): move to in-memory cache - prevent redis flakiness from impacting ci/cd
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 15s
Helm unit test / unit-test (push) Successful in 19s
2025-03-28 13:32:04 -07:00
Ishaan Jaff
1eaf847f8a test pod lock manager 2025-03-28 13:31:45 -07:00
Dbzman
7c85054bad
fix: wrong indentation for ttlSecondsAfterFinished (#9611) 2025-03-28 13:20:12 -07:00
Nicholas Grabar
1f2bbda11d Add recursion depth to convert_anyof_null_to_nullable, constants.py. Fix recursive_detector.py raise error state 2025-03-28 13:11:19 -07:00
Ishaan Jaff
021eedaf69 test pod lock manager 2025-03-28 12:59:16 -07:00
Ishaan Jaff
c53d172b06 rename pod lock manager 2025-03-28 12:57:00 -07:00
Krish Dholakia
f3ff7e2779
fix(logging): add json formatting for uncaught exceptions (#9615) (#9619)
* fix(logging): add json formatting for uncaught exceptions (#9615)

* fix(_logging.py): cleanup logging to catch unhandled exceptions

* fix(_logging.py): avoid using 'print' '

---------

Co-authored-by: Henrique Cavarsan <hcavarsan@yahoo.com.br>
2025-03-28 12:47:26 -07:00
Krish Dholakia
0865e52db3
fix(proxy_server.py): get master key from environment, if not set in … (#9617)
* fix(proxy_server.py): get master key from environment, if not set in general settings or general settings not set at all

* test: mark flaky test

* test(test_proxy_server.py): mock prisma client

* ci: add new github workflow for testing just the mock tests

* fix: fix linting error

* ci(conftest.py): add conftest.py to isolate proxy tests

* build(pyproject.toml): add respx to dev dependencies

* build(pyproject.toml): add prisma to dev dependencies

* test: fix mock prompt management tests to use a mock anthropic key

* ci(test-litellm.yml): parallelize mock testing

make it run faster

* build(pyproject.toml): add hypercorn as dev dep

* build(pyproject.toml): separate proxy vs. core dev dependencies

make it easier for non-proxy contributors to run tests locally - e.g. no need to install hypercorn

* ci(test-litellm.yml): pin python version

* test(test_rerank.py): move test - cannot be mocked, requires aws credentials for e2e testing

* ci: add thank you message to ci

* test: add mock env var to test

* test: add autouse to tests

* test: test mock env vars for e2e tests
2025-03-28 12:32:04 -07:00
Shixian Sheng
e729615e41
Merge branch 'BerriAI:main' into main 2025-03-28 14:38:28 -04:00
Ishaan Jaff
e7181395ff fix code quality check 2025-03-28 10:32:39 -07:00
Ishaan Jaff
462e9baffa
Merge pull request #9618 from NickGrab/litellm_8864-feature-vertex-anyOf-support
Add support to Vertex AI transformation for anyOf union type with null fields
2025-03-28 10:29:33 -07:00
NickGrab
b72fbdde74
Merge branch 'main' into litellm_8864-feature-vertex-anyOf-support 2025-03-28 10:25:04 -07:00