Commit graph

109 commits

Author SHA1 Message Date
Krish Dholakia
0d503ad8ad
Move daily user transaction logging outside of 'disable_spend_logs' flag - different tables (#9772)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 16s
Helm unit test / unit-test (push) Successful in 18s
* refactor(db_spend_update_writer.py): aggregate table is entirely different

* test(test_db_spend_update_writer.py): add unit test to ensure if disable_spend_logs is true daily user transactions is still logged

* test: fix test
2025-04-05 09:58:16 -07:00
Ishaan Jaff
df51d8bcfa Merge branch 'main' into litellm_metrics_pod_lock_manager 2025-04-04 21:11:39 -07:00
Ishaan Jaff
decb6649ec test_queue_flush_limit 2025-04-04 16:29:06 -07:00
Ishaan Jaff
e77a178a37 test_queue_size_reduction_with_large_volume 2025-04-04 16:21:29 -07:00
Ishaan Jaff
eb48cbdec6 aggregate_queue_updates 2025-04-04 15:54:07 -07:00
Ishaan Jaff
93068cb142 flush_all_updates_from_in_memory_queue 2025-04-04 15:34:56 -07:00
Ishaan Jaff
065477abb4 add _get_aggregated_spend_update_queue_item 2025-04-04 15:32:27 -07:00
Ishaan Jaff
9abaefea62 add logic for max size in memory queue 2025-04-04 15:31:40 -07:00
Ishaan Jaff
363fb0c46f add MAX_SIZE_IN_MEMORY_QUEUE 2025-04-04 15:31:09 -07:00
Ishaan Jaff
80fb4ece97 prom emit size of DB TX queues for observability 2025-04-02 18:39:29 -07:00
Ishaan Jaff
73bbd0a446 emit lock acquired and released events 2025-04-02 17:40:25 -07:00
Ishaan Jaff
a64631edfb test pod lock manager 2025-04-02 14:39:40 -07:00
Ishaan Jaff
44bc8974c8 BaseUpdateQueue 2025-04-01 18:31:54 -07:00
Ishaan Jaff
8dc792139e refactor file structure 2025-04-01 18:30:48 -07:00
Ishaan Jaff
07fc5a72f1 add debug statement 2025-04-01 17:54:52 -07:00
Ishaan Jaff
827ad38e7f fix spend update queue 2025-04-01 17:49:28 -07:00
Ishaan Jaff
4c14550721 refactor daily spend updates to use new Queue DS 2025-04-01 17:45:19 -07:00
Ishaan Jaff
d0a7e44a6e fix linting 2025-03-31 21:33:05 -07:00
Ishaan Jaff
115946d402 unit testing for SpendUpdateQueue 2025-03-31 21:25:24 -07:00
Ishaan Jaff
f7ddc583f0 linting fix get_aggregated_db_spend_update_transactions 2025-03-31 21:15:21 -07:00
Ishaan Jaff
5fa5c1154e add SpendUpdateQueue 2025-03-31 19:42:20 -07:00
Ishaan Jaff
811f488ca3 get_aggregated_db_spend_update_transactions 2025-03-31 19:31:53 -07:00
Ishaan Jaff
71e772dd4a use typed data structure for queue 2025-03-31 19:28:17 -07:00
Ishaan Jaff
4959512170 fix update_end_user_spend 2025-03-31 19:05:58 -07:00
Ishaan Jaff
73d6af5572 fix docstring 2025-03-31 19:03:02 -07:00
Ishaan Jaff
3e16a51ca6 fix update_database helper on db_spend_update_writer 2025-03-31 19:01:00 -07:00
Ishaan Jaff
bcd49204f6 use spend_update_queue for RedisUpdateBuffer 2025-03-31 18:40:52 -07:00
Ishaan Jaff
efe6d375e9 add new SpendUpdateQueue 2025-03-31 18:40:03 -07:00
Krish Dholakia
1604f87663
install prisma migration files - connects litellm proxy to litellm's prisma migration files (#9637)
* build(README.md): initial commit adding a separate folder for additional proxy files. Meant to reduce size of core package

* build(litellm-proxy-extras/): new pip package for storing migration files

allows litellm proxy to use migration files, without adding them to core repo

* build(litellm-proxy-extras/): cleanup pyproject.toml

* build: move prisma migration files inside new proxy extras package

* build(run_migration.py): update script to write to correct folder

* build(proxy_cli.py): load in migration files from litellm-proxy-extras

Closes https://github.com/BerriAI/litellm/issues/9558

* build: add MIT license to litellm-proxy-extras

* test: update test

* fix: fix schema

* bump: version 0.1.0 → 0.1.1

* build(publish-proxy-extras.sh): add script for publishing new proxy-extras version

* build(liccheck.ini): add litellm-proxy-extras to authorized packages

* fix(litellm-proxy-extras/utils.py): move prisma migrate logic inside extra proxy pkg

easier since migrations folder already there

* build(pre-commit-config.yaml): add litellm_proxy_extras to ci tests

* docs(config_settings.md): document new env var

* build(pyproject.toml): bump relevant files when litellm-proxy-extras version changed

* build(pre-commit-config.yaml): run poetry check on litellm-proxy-extras as well
2025-03-29 15:27:09 -07:00
Krish Dholakia
9b7ebb6a7d
build(pyproject.toml): add new dev dependencies - for type checking (#9631)
* build(pyproject.toml): add new dev dependencies - for type checking

* build: reformat files to fit black

* ci: reformat to fit black

* ci(test-litellm.yml): make tests run clear

* build(pyproject.toml): add ruff

* fix: fix ruff checks

* build(mypy/): fix mypy linting errors

* fix(hashicorp_secret_manager.py): fix passing cert for tls auth

* build(mypy/): resolve all mypy errors

* test: update test

* fix: fix black formatting

* build(pre-commit-config.yaml): use poetry run black

* fix(proxy_server.py): fix linting error

* fix: fix ruff safe representation error
2025-03-29 11:02:13 -07:00
Ishaan Jaff
c95fb6c692 MAX_REDIS_BUFFER_DEQUEUE_COUNT 2025-03-28 20:59:28 -07:00
Ishaan Jaff
29d6968f58 fix linting 2025-03-28 20:57:28 -07:00
Ishaan Jaff
7e8a02099c Merge branch 'main' into litellm_use_redis_for_updates 2025-03-28 20:12:29 -07:00
Ishaan Jaff
69d5734685 fix - locking in memory leads to failing tests 2025-03-28 20:10:58 -07:00
Ishaan Jaff
183c33bb02 prisma client in_memory_transaction_lock 2025-03-28 19:52:07 -07:00
Ishaan Jaff
21bf15263c use asyncio lock for updating PrismaClient txs 2025-03-28 19:51:15 -07:00
Ishaan Jaff
88458a6568 redis update buffer queue 2025-03-28 17:24:54 -07:00
Ishaan Jaff
cec5280e03 debugging for Redis TX management 2025-03-28 16:16:44 -07:00
Ishaan Jaff
7c73987066 fix loc of DBSpendUpdateTransactions 2025-03-28 15:17:01 -07:00
Ishaan Jaff
3df839d878 use helper methods _commit_spend_updates_to_db_without_redis_buffer 2025-03-28 15:12:19 -07:00
Ishaan Jaff
1eaf847f8a test pod lock manager 2025-03-28 13:31:45 -07:00
Ishaan Jaff
c53d172b06 rename pod lock manager 2025-03-28 12:57:00 -07:00
Ishaan Jaff
0edd4aa8a7 fix PodLockManager 2025-03-27 23:29:48 -07:00
Krish Dholakia
cdcc8ea9b7
Connect UI to "LiteLLM_DailyUserSpend" spend table - enables usage tab to work at 1m+ spend logs (#9603)
* feat(spend_management_endpoints.py): expose new endpoint for querying user's usage at 1m+ spend logs

Allows user to view their spend at 1m+ spend logs

* build(schema.prisma): add api_requests to dailyuserspend table

* build(migration.sql): add migration file for new column to daily user spend table

* build(prisma_client.py): add logic for copying over migration folder, if deploy/migrations present in expected location

enables easier testing of prisma migration flow

* build(ui/): initial commit successfully using the dailyuserspend table on the UI

* refactor(internal_user_endpoints.py): refactor `/user/daily/activity` to give breakdowns by provider/model/key

* feat: feature parity (cost page) with existing 'usage' page

* build(ui/): add activity tab to new_usage.tsx

gets to feature parity on 'All Up' page of 'usage.tsx'

* fix(proxy/utils.py): count number of api requests in daily user spend table

allows us to see activity by model on new usage tab

* style(new_usage.tsx): fix y-axis to be in ascending order of date

* fix: fix linting errors

* fix: fix ruff check errors
2025-03-27 23:29:15 -07:00
Ishaan Jaff
1ba73f491a working cron job updates 2025-03-27 23:13:01 -07:00
Ishaan Jaff
666cd8d4e3 handle commit updates to the DB 2025-03-27 22:55:36 -07:00
Ishaan Jaff
0cb6cb6c66 fix schema to work with lock acquisition 2025-03-27 22:54:46 -07:00
Ishaan Jaff
758182fc7f fix typo on codebase 2025-03-27 22:36:00 -07:00
Ishaan Jaff
fc46f6b861 fix db spend update buffer 2025-03-27 22:34:15 -07:00
Ishaan Jaff
9139230531 update the correct set of txs 2025-03-27 21:54:12 -07:00