Commit graph

13364 commits

Author SHA1 Message Date
Ishaan Jaff
4fb9d27a35 use constants for redis buffer in DB 2025-03-28 20:55:43 -07:00
Krish Dholakia
4879e1ecf0
fix(openrouter/chat/transformation.py): raise informative message for openrouter key error (#9626)
Related Issue: https://github.com/Aider-AI/aider/issues/3550#issuecomment-2763052355
2025-03-28 20:24:28 -07:00
Ishaan Jaff
7e8a02099c Merge branch 'main' into litellm_use_redis_for_updates 2025-03-28 20:12:29 -07:00
Ishaan Jaff
69d5734685 fix - locking in memory leads to failing tests 2025-03-28 20:10:58 -07:00
Ishaan Jaff
ba550e2147 test local spend accuracy 2025-03-28 19:52:39 -07:00
Ishaan Jaff
183c33bb02 prisma client in_memory_transaction_lock 2025-03-28 19:52:07 -07:00
Ishaan Jaff
21bf15263c use asyncio lock for updating PrismaClient txs 2025-03-28 19:51:15 -07:00
Ishaan Jaff
eae2adcee0 redis cache add async push and pop methods 2025-03-28 19:49:38 -07:00
Ishaan Jaff
88458a6568 redis update buffer queue 2025-03-28 17:24:54 -07:00
Ishaan Jaff
e5e0ffa28a add async_rpush 2025-03-28 17:23:47 -07:00
Ishaan Jaff
cec5280e03 debugging for Redis TX management 2025-03-28 16:16:44 -07:00
NickGrab
220d4c07f4
Merge pull request #9625 from BerriAI/litellm_mar_28_vertex_fix
Add support to Vertex AI transformation for anyOf union type with null fields
2025-03-28 16:09:29 -07:00
Krish Dholakia
222898d727
Fix anthropic thinking + response_format (#9594)
* fix(anthropic/chat/transformation.py): Don't set tool choice on response_format conversion when thinking is enabled

Not allowed by Anthropic

Fixes https://github.com/BerriAI/litellm/issues/8901

* refactor: move test to base anthropic chat tests

ensures consistent behaviour across vertex/anthropic/bedrock

* fix(anthropic/chat/transformation.py): if thinking token is specified and max tokens is not - ensure max token to anthropic is higher than thinking tokens

* feat(converse_transformation.py): correctly handle thinking + response format on Bedrock Converse

Fixes https://github.com/BerriAI/litellm/issues/8901

* fix(converse_transformation.py): correctly handle adding max tokens

* test: handle service unavailable error
2025-03-28 15:57:40 -07:00
Ishaan Jaff
df136c5689 add docstring for DBSpendUpdateTransactions 2025-03-28 15:20:28 -07:00
Ishaan Jaff
7c73987066 fix loc of DBSpendUpdateTransactions 2025-03-28 15:17:01 -07:00
Krish Dholakia
1a441def03 fix(logging): add json formatting for uncaught exceptions (#9615) (#9619)
* fix(logging): add json formatting for uncaught exceptions (#9615)

* fix(_logging.py): cleanup logging to catch unhandled exceptions

* fix(_logging.py): avoid using 'print' '

---------

Co-authored-by: Henrique Cavarsan <hcavarsan@yahoo.com.br>
2025-03-28 15:16:15 -07:00
Krish Dholakia
205db622bf fix(proxy_server.py): get master key from environment, if not set in … (#9617)
* fix(proxy_server.py): get master key from environment, if not set in general settings or general settings not set at all

* test: mark flaky test

* test(test_proxy_server.py): mock prisma client

* ci: add new github workflow for testing just the mock tests

* fix: fix linting error

* ci(conftest.py): add conftest.py to isolate proxy tests

* build(pyproject.toml): add respx to dev dependencies

* build(pyproject.toml): add prisma to dev dependencies

* test: fix mock prompt management tests to use a mock anthropic key

* ci(test-litellm.yml): parallelize mock testing

make it run faster

* build(pyproject.toml): add hypercorn as dev dep

* build(pyproject.toml): separate proxy vs. core dev dependencies

make it easier for non-proxy contributors to run tests locally - e.g. no need to install hypercorn

* ci(test-litellm.yml): pin python version

* test(test_rerank.py): move test - cannot be mocked, requires aws credentials for e2e testing

* ci: add thank you message to ci

* test: add mock env var to test

* test: add autouse to tests

* test: test mock env vars for e2e tests
2025-03-28 15:16:15 -07:00
Ishaan Jaff
3df839d878 use helper methods _commit_spend_updates_to_db_without_redis_buffer 2025-03-28 15:12:19 -07:00
Ishaan Jaff
1eaf847f8a test pod lock manager 2025-03-28 13:31:45 -07:00
Nicholas Grabar
1f2bbda11d Add recursion depth to convert_anyof_null_to_nullable, constants.py. Fix recursive_detector.py raise error state 2025-03-28 13:11:19 -07:00
Ishaan Jaff
c53d172b06 rename pod lock manager 2025-03-28 12:57:00 -07:00
Krish Dholakia
f3ff7e2779
fix(logging): add json formatting for uncaught exceptions (#9615) (#9619)
* fix(logging): add json formatting for uncaught exceptions (#9615)

* fix(_logging.py): cleanup logging to catch unhandled exceptions

* fix(_logging.py): avoid using 'print' '

---------

Co-authored-by: Henrique Cavarsan <hcavarsan@yahoo.com.br>
2025-03-28 12:47:26 -07:00
Krish Dholakia
0865e52db3
fix(proxy_server.py): get master key from environment, if not set in … (#9617)
* fix(proxy_server.py): get master key from environment, if not set in general settings or general settings not set at all

* test: mark flaky test

* test(test_proxy_server.py): mock prisma client

* ci: add new github workflow for testing just the mock tests

* fix: fix linting error

* ci(conftest.py): add conftest.py to isolate proxy tests

* build(pyproject.toml): add respx to dev dependencies

* build(pyproject.toml): add prisma to dev dependencies

* test: fix mock prompt management tests to use a mock anthropic key

* ci(test-litellm.yml): parallelize mock testing

make it run faster

* build(pyproject.toml): add hypercorn as dev dep

* build(pyproject.toml): separate proxy vs. core dev dependencies

make it easier for non-proxy contributors to run tests locally - e.g. no need to install hypercorn

* ci(test-litellm.yml): pin python version

* test(test_rerank.py): move test - cannot be mocked, requires aws credentials for e2e testing

* ci: add thank you message to ci

* test: add mock env var to test

* test: add autouse to tests

* test: test mock env vars for e2e tests
2025-03-28 12:32:04 -07:00
Ishaan Jaff
e7181395ff fix code quality check 2025-03-28 10:32:39 -07:00
NickGrab
b72fbdde74
Merge branch 'main' into litellm_8864-feature-vertex-anyOf-support 2025-03-28 10:25:04 -07:00
Krrish Dholakia
69e28b92c6 fix: fix python38 linting error 2025-03-28 09:38:32 -07:00
Krrish Dholakia
34a1a48af1 fix(common_utils.py): fix linting error
Some checks failed
Read Version from pyproject.toml / read-version (push) Successful in 19s
Helm unit test / unit-test (push) Successful in 20s
Publish Prisma Migrations / publish-migrations (push) Failing after 1m15s
2025-03-27 23:31:58 -07:00
Krrish Dholakia
3226477464 fix: cleanup 2025-03-27 23:31:12 -07:00
Ishaan Jaff
0edd4aa8a7 fix PodLockManager 2025-03-27 23:29:48 -07:00
Krish Dholakia
cdcc8ea9b7
Connect UI to "LiteLLM_DailyUserSpend" spend table - enables usage tab to work at 1m+ spend logs (#9603)
* feat(spend_management_endpoints.py): expose new endpoint for querying user's usage at 1m+ spend logs

Allows user to view their spend at 1m+ spend logs

* build(schema.prisma): add api_requests to dailyuserspend table

* build(migration.sql): add migration file for new column to daily user spend table

* build(prisma_client.py): add logic for copying over migration folder, if deploy/migrations present in expected location

enables easier testing of prisma migration flow

* build(ui/): initial commit successfully using the dailyuserspend table on the UI

* refactor(internal_user_endpoints.py): refactor `/user/daily/activity` to give breakdowns by provider/model/key

* feat: feature parity (cost page) with existing 'usage' page

* build(ui/): add activity tab to new_usage.tsx

gets to feature parity on 'All Up' page of 'usage.tsx'

* fix(proxy/utils.py): count number of api requests in daily user spend table

allows us to see activity by model on new usage tab

* style(new_usage.tsx): fix y-axis to be in ascending order of date

* fix: fix linting errors

* fix: fix ruff check errors
2025-03-27 23:29:15 -07:00
Krrish Dholakia
b155f4f4a6 fix(proxy_server.py): Fix https://github.com/BerriAI/litellm/issues/9576 2025-03-27 23:25:46 -07:00
Ishaan Jaff
1ba73f491a working cron job updates 2025-03-27 23:13:01 -07:00
Ishaan Jaff
666cd8d4e3 handle commit updates to the DB 2025-03-27 22:55:36 -07:00
Ishaan Jaff
0cb6cb6c66 fix schema to work with lock acquisition 2025-03-27 22:54:46 -07:00
Krish Dholakia
ccbac691e5
Support discovering gemini, anthropic, xai models by calling their /v1/model endpoint (#9530)
* fix: initial commit for adding provider model discovery to gemini

* feat(gemini/): add model discovery for gemini/ route

* docs(set_keys.md): update docs to show you can check available gemini models as well

* feat(anthropic/): add model discovery for anthropic api key

* feat(xai/): add model discovery for XAI

enables checking what models an xai key can call

* ci: bump ci config yml

* fix(topaz/common_utils.py): fix linting error

* fix: fix linting error for python38
2025-03-27 22:50:48 -07:00
Ishaan Jaff
758182fc7f fix typo on codebase 2025-03-27 22:36:00 -07:00
Ishaan Jaff
fc46f6b861 fix db spend update buffer 2025-03-27 22:34:15 -07:00
Ishaan Jaff
1bfffadd05 fix typo 2025-03-27 22:31:38 -07:00
Krish Dholakia
340a63bace
fix(mistral_chat_transformation.py): add missing comma (#9606)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 17s
Helm unit test / unit-test (push) Successful in 17s
2025-03-27 22:16:21 -07:00
Ishaan Jaff
9139230531 update the correct set of txs 2025-03-27 21:54:12 -07:00
Ishaan Jaff
aa6c34cb97 use lock manager for update spend job 2025-03-27 21:43:02 -07:00
Ishaan Jaff
dd00429ade update schema 2025-03-27 21:36:22 -07:00
Ishaan Jaff
ea93a09b7a add model CronJob 2025-03-27 21:10:26 -07:00
Ishaan Jaff
e65b708fa2 get_all_update_transactions_from_redis 2025-03-27 19:15:30 -07:00
Krish Dholakia
b9d0f460e8
Revert "Support max_completion_tokens on Mistral (#9589)" (#9604)
This reverts commit fef5d23dd5.
2025-03-27 19:14:26 -07:00
Ishaan Jaff
963791bbb5 use redis update buffer class 2025-03-27 19:12:51 -07:00
Ishaan Jaff
ad72078167 basic structure for commit update txs to redis 2025-03-27 18:21:33 -07:00
Ishaan Jaff
894306141e refactor, use commit_update_transactions_to_db 2025-03-27 18:07:23 -07:00
Chris Mancuso
fef5d23dd5
Support max_completion_tokens on Mistral (#9589)
* Support max_completion_tokens on Mistral

* test fix
2025-03-27 17:27:19 -07:00
Krish Dholakia
fb83567a03
Litellm new UI build (#9601)
* build: new ui build

* build: new ui build

* fix(proxy_server.py): only show user models their key can access on `/models`

* fix(model_management_endpoints.py): ensure team admin can add models

* test: update unit testing to reflect changes

* fix(model_dashboard.tsx): fix sizing on models page

* build: fix ui
2025-03-27 17:15:25 -07:00