Ishaan Jaff
bd88263b29
[Feat - Cost Tracking improvement] Track prompt caching metrics in DailyUserSpendTransactions ( #10029 )
...
* stash changes
* emit cache read/write tokens to daily spend update
* emit cache read/write tokens on daily activity
* update types.ts
* docs prompt caching
* undo ui change
* fix activity metrics
* fix prompt caching metrics
* fix typed dict fields
* fix get_aggregated_daily_spend_update_transactions
* fix aggregating cache tokens
* test_cache_token_fields_aggregation
* daily_transaction
* add cache_creation_input_tokens and cache_read_input_tokens to LiteLLM_DailyUserSpend
* test_daily_spend_update_queue.py
2025-04-15 21:40:57 -07:00
Krish Dholakia
9b77559ccf
Add aggregate team based usage logging ( #10039 )
...
* feat(schema.prisma): initial commit adding aggregate table for team spend
allows team spend to be visible at 1m+ logs
* feat(db_spend_update_writer.py): support logging aggregate team spend
allows usage dashboard to work at 1m+ logs
* feat(litellm-proxy-extras/): add new migration file
* fix(db_spend_update_writer.py): fix return type
* build: bump requirements
* fix: fix ruff error
2025-04-15 20:58:48 -07:00
Krish Dholakia
3ca82c22b6
Support CRUD endpoints for Managed Files ( #9924 )
...
* fix(openai.py): ensure openai file object shows up on logs
* fix(managed_files.py): return unified file id as b64 str
allows retrieve file id to work as expected
* fix(managed_files.py): apply decoded file id transformation
* fix: add unit test for file id + decode logic
* fix: initial commit for litellm_proxy support with CRUD Endpoints
* fix(managed_files.py): support retrieve file operation
* fix(managed_files.py): support for DELETE endpoint for files
* fix(managed_files.py): retrieve file content support
supports retrieve file content api from openai
* fix: fix linting error
* test: update tests
* fix: fix linting error
* fix(files/main.py): pass litellm params to azure route
* test: fix test
2025-04-11 21:48:27 -07:00
Ishaan Jaff
94a553dbb2
[Feat] Emit Key, Team Budget metrics on a cron job schedule ( #9528 )
...
* _initialize_remaining_budget_metrics
* initialize_budget_metrics_cron_job
* initialize_budget_metrics_cron_job
* initialize_budget_metrics_cron_job
* test_initialize_budget_metrics_cron_job
* LITELLM_PROXY_ADMIN_NAME
* fix code qa checks
* test_initialize_budget_metrics_cron_job
* test_initialize_budget_metrics_cron_job
* pod lock manager allow dynamic cron job ID
* fix pod lock manager
* require cronjobid for PodLockManager
* fix DB_SPEND_UPDATE_JOB_NAME acquire / release lock
* add comment on prometheus logger
* add debug statements for emitting key, team budget metrics
* test_pod_lock_manager.py
* test_initialize_budget_metrics_cron_job
* initialize_budget_metrics_cron_job
* initialize_remaining_budget_metrics
* remove outdated test
2025-04-10 16:59:14 -07:00
Krish Dholakia
0d503ad8ad
Move daily user transaction logging outside of 'disable_spend_logs' flag - different tables ( #9772 )
...
Read Version from pyproject.toml / read-version (push) Successful in 16s
Helm unit test / unit-test (push) Successful in 18s
* refactor(db_spend_update_writer.py): aggregate table is entirely different
* test(test_db_spend_update_writer.py): add unit test to ensure if disable_spend_logs is true daily user transactions is still logged
* test: fix test
2025-04-05 09:58:16 -07:00
Ishaan Jaff
8dc792139e
refactor file structure
2025-04-01 18:30:48 -07:00
Ishaan Jaff
827ad38e7f
fix spend update queue
2025-04-01 17:49:28 -07:00
Ishaan Jaff
4c14550721
refactor daily spend updates to use new Queue DS
2025-04-01 17:45:19 -07:00
Ishaan Jaff
71e772dd4a
use typed data structure for queue
2025-03-31 19:28:17 -07:00
Ishaan Jaff
4959512170
fix update_end_user_spend
2025-03-31 19:05:58 -07:00
Ishaan Jaff
3e16a51ca6
fix update_database helper on db_spend_update_writer
2025-03-31 19:01:00 -07:00
Ishaan Jaff
69d5734685
fix - locking in memory leads to failing tests
2025-03-28 20:10:58 -07:00
Ishaan Jaff
183c33bb02
prisma client in_memory_transaction_lock
2025-03-28 19:52:07 -07:00
Ishaan Jaff
88458a6568
redis update buffer queue
2025-03-28 17:24:54 -07:00
Ishaan Jaff
cec5280e03
debugging for Redis TX management
2025-03-28 16:16:44 -07:00
Ishaan Jaff
7c73987066
fix loc of DBSpendUpdateTransactions
2025-03-28 15:17:01 -07:00
Ishaan Jaff
3df839d878
use helper methods _commit_spend_updates_to_db_without_redis_buffer
2025-03-28 15:12:19 -07:00
Ishaan Jaff
c53d172b06
rename pod lock manager
2025-03-28 12:57:00 -07:00
Ishaan Jaff
666cd8d4e3
handle commit updates to the DB
2025-03-27 22:55:36 -07:00
Ishaan Jaff
758182fc7f
fix typo on codebase
2025-03-27 22:36:00 -07:00
Ishaan Jaff
fc46f6b861
fix db spend update buffer
2025-03-27 22:34:15 -07:00
Ishaan Jaff
9139230531
update the correct set of txs
2025-03-27 21:54:12 -07:00
Ishaan Jaff
aa6c34cb97
use lock manager for update spend job
2025-03-27 21:43:02 -07:00
Ishaan Jaff
963791bbb5
use redis update buffer class
2025-03-27 19:12:51 -07:00
Ishaan Jaff
ad72078167
basic structure for commit update txs to redis
2025-03-27 18:21:33 -07:00
Ishaan Jaff
894306141e
refactor, use commit_update_transactions_to_db
2025-03-27 18:07:23 -07:00
Ishaan Jaff
b721b2b4ac
use DBSpendUpdateWriter common function for
2025-03-27 17:04:50 -07:00
Ishaan Jaff
403f2ef68d
use simple static methods for updating spend
2025-03-27 16:57:12 -07:00
Ishaan Jaff
7995fd7c98
fix DBSpendUpdateWriter
2025-03-27 16:44:46 -07:00
Ishaan Jaff
a0fd508de4
DBSpendUpdateWriter
2025-03-27 16:43:18 -07:00
Ishaan Jaff
21e3b764f5
use DBSpendUpdateWriter class for managing DB spend updates
2025-03-27 16:31:23 -07:00
