Commit graph

32 commits

Author SHA1 Message Date
Krish Dholakia
3f5c8ae000 Add aggregate spend by tag (#10071)
* feat: initial commit adding daily tag spend table to db

* feat(db_spend_update_writer.py): correctly log tag spend transactions

* build(schema.prisma): add new tag table to root

* build: add new migration file
2025-04-16 12:26:21 -07:00
Ishaan Jaff
3aed97e63e [Feat - Cost Tracking improvement] Track prompt caching metrics in DailyUserSpendTransactions (#10029)
* stash changes

* emit cache read/write tokens to daily spend update

* emit cache read/write tokens on daily activity

* update types.ts

* docs prompt caching

* undo ui change

* fix activity metrics

* fix prompt caching metrics

* fix typed dict fields

* fix get_aggregated_daily_spend_update_transactions

* fix aggregating cache tokens

* test_cache_token_fields_aggregation

* daily_transaction

* add cache_creation_input_tokens and cache_read_input_tokens to LiteLLM_DailyUserSpend

* test_daily_spend_update_queue.py
2025-04-15 21:40:57 -07:00
Krish Dholakia
08053e8f12 Add aggregate team based usage logging (#10039)
* feat(schema.prisma): initial commit adding aggregate table for team spend

allows team spend to be visible at 1m+ logs

* feat(db_spend_update_writer.py): support logging aggregate team spend

allows usage dashboard to work at 1m+ logs

* feat(litellm-proxy-extras/): add new migration file

* fix(db_spend_update_writer.py): fix return type

* build: bump requirements

* fix: fix ruff error
2025-04-15 20:58:48 -07:00
Krish Dholakia
c5c7c117af Support CRUD endpoints for Managed Files (#9924)
* fix(openai.py): ensure openai file object shows up on logs

* fix(managed_files.py): return unified file id as b64 str

allows retrieve file id to work as expected

* fix(managed_files.py): apply decoded file id transformation

* fix: add unit test for file id + decode logic

* fix: initial commit for litellm_proxy support with CRUD Endpoints

* fix(managed_files.py): support retrieve file operation

* fix(managed_files.py): support for DELETE endpoint for files

* fix(managed_files.py): retrieve file content support

supports retrieve file content api from openai

* fix: fix linting error

* test: update tests

* fix: fix linting error

* fix(files/main.py): pass litellm params to azure route

* test: fix test
2025-04-11 21:48:27 -07:00
Ishaan Jaff
4c85e13226 [Feat] Emit Key, Team Budget metrics on a cron job schedule (#9528)
* _initialize_remaining_budget_metrics

* initialize_budget_metrics_cron_job

* initialize_budget_metrics_cron_job

* initialize_budget_metrics_cron_job

* test_initialize_budget_metrics_cron_job

* LITELLM_PROXY_ADMIN_NAME

* fix code qa checks

* test_initialize_budget_metrics_cron_job

* test_initialize_budget_metrics_cron_job

* pod lock manager allow dynamic cron job ID

* fix pod lock manager

* require cronjobid for PodLockManager

* fix DB_SPEND_UPDATE_JOB_NAME acquire / release lock

* add comment on prometheus logger

* add debug statements for emitting key, team budget metrics

* test_pod_lock_manager.py

* test_initialize_budget_metrics_cron_job

* initialize_budget_metrics_cron_job

* initialize_remaining_budget_metrics

* remove outdated test
2025-04-10 16:59:14 -07:00
Krish Dholakia
36cdb79409 Move daily user transaction logging outside of 'disable_spend_logs' flag - different tables (#9772)
* refactor(db_spend_update_writer.py): aggregate table is entirely different

* test(test_db_spend_update_writer.py): add unit test to ensure if disable_spend_logs is true daily user transactions is still logged

* test: fix test
2025-04-05 09:58:16 -07:00
Ishaan Jaff
be9e9e6db5 refactor file structure 2025-04-01 18:30:48 -07:00
Ishaan Jaff
f122cabd24 fix spend update queue 2025-04-01 17:49:28 -07:00
Ishaan Jaff
9270823d6c refactor daily spend updates to use new Queue DS 2025-04-01 17:45:19 -07:00
Ishaan Jaff
b845b8f863 use typed data structure for queue 2025-03-31 19:28:17 -07:00
Ishaan Jaff
5b706c049a fix update_end_user_spend 2025-03-31 19:05:58 -07:00
Ishaan Jaff
fc72a61627 fix update_database helper on db_spend_update_writer 2025-03-31 19:01:00 -07:00
Ishaan Jaff
93c20b8d2d fix - locking in memory leads to failing tests 2025-03-28 20:10:58 -07:00
Ishaan Jaff
96e4369160 prisma client in_memory_transaction_lock 2025-03-28 19:52:07 -07:00
Ishaan Jaff
7a63cbe8d0 redis update buffer queue 2025-03-28 17:24:54 -07:00
Ishaan Jaff
36f15f19af debugging for Redis TX management 2025-03-28 16:16:44 -07:00
Ishaan Jaff
cc9f47b1d2 fix loc of DBSpendUpdateTransactions 2025-03-28 15:17:01 -07:00
Ishaan Jaff
fa304a538d use helper methods _commit_spend_updates_to_db_without_redis_buffer 2025-03-28 15:12:19 -07:00
Ishaan Jaff
17d27fcd36 rename pod lock manager 2025-03-28 12:57:00 -07:00
Ishaan Jaff
39cc75c36b handle commit updates to the DB 2025-03-27 22:55:36 -07:00
Ishaan Jaff
a1a038aa19 fix typo on codebase 2025-03-27 22:36:00 -07:00
Ishaan Jaff
3bd005f148 fix db spend update buffer 2025-03-27 22:34:15 -07:00
Ishaan Jaff
081bd58ea5 update the correct set of txs 2025-03-27 21:54:12 -07:00
Ishaan Jaff
9efd1814bd use lock manager for update spend job 2025-03-27 21:43:02 -07:00
Ishaan Jaff
7c4a73ba39 use redis update buffer class 2025-03-27 19:12:51 -07:00
Ishaan Jaff
d03b087e25 basic structure for commit update txs to redis 2025-03-27 18:21:33 -07:00
Ishaan Jaff
33f877e8b0 refactor, use commit_update_transactions_to_db 2025-03-27 18:07:23 -07:00
Ishaan Jaff
7f24714576 use DBSpendUpdateWriter common function for 2025-03-27 17:04:50 -07:00
Ishaan Jaff
a43c7e9104 use simple static methods for updating spend 2025-03-27 16:57:12 -07:00
Ishaan Jaff
3b5fd92f75 fix DBSpendUpdateWriter 2025-03-27 16:44:46 -07:00
Ishaan Jaff
2c9d1cd4d4 DBSpendUpdateWriter 2025-03-27 16:43:18 -07:00
Ishaan Jaff
c367a10c1c use DBSpendUpdateWriter class for managing DB spend updates 2025-03-27 16:31:23 -07:00