Ishaan Jaff
3aed97e63e
[Feat - Cost Tracking improvement] Track prompt caching metrics in DailyUserSpendTransactions ( #10029 )
...
* stash changes
* emit cache read/write tokens to daily spend update
* emit cache read/write tokens on daily activity
* update types.ts
* docs prompt caching
* undo ui change
* fix activity metrics
* fix prompt caching metrics
* fix typed dict fields
* fix get_aggregated_daily_spend_update_transactions
* fix aggregating cache tokens
* test_cache_token_fields_aggregation
* daily_transaction
* add cache_creation_input_tokens and cache_read_input_tokens to LiteLLM_DailyUserSpend
* test_daily_spend_update_queue.py
2025-04-15 21:40:57 -07:00
Krish Dholakia
08053e8f12
Add aggregate team based usage logging ( #10039 )
...
* feat(schema.prisma): initial commit adding aggregate table for team spend
allows team spend to be visible at 1m+ logs
* feat(db_spend_update_writer.py): support logging aggregate team spend
allows usage dashboard to work at 1m+ logs
* feat(litellm-proxy-extras/): add new migration file
* fix(db_spend_update_writer.py): fix return type
* build: bump requirements
* fix: fix ruff error
2025-04-15 20:58:48 -07:00
Krish Dholakia
c5c7c117af
Support CRUD endpoints for Managed Files ( #9924 )
...
* fix(openai.py): ensure openai file object shows up on logs
* fix(managed_files.py): return unified file id as b64 str
allows retrieve file id to work as expected
* fix(managed_files.py): apply decoded file id transformation
* fix: add unit test for file id + decode logic
* fix: initial commit for litellm_proxy support with CRUD Endpoints
* fix(managed_files.py): support retrieve file operation
* fix(managed_files.py): support for DELETE endpoint for files
* fix(managed_files.py): retrieve file content support
supports retrieve file content api from openai
* fix: fix linting error
* test: update tests
* fix: fix linting error
* fix(files/main.py): pass litellm params to azure route
* test: fix test
2025-04-11 21:48:27 -07:00
Ishaan Jaff
4c85e13226
[Feat] Emit Key, Team Budget metrics on a cron job schedule ( #9528 )
...
* _initialize_remaining_budget_metrics
* initialize_budget_metrics_cron_job
* initialize_budget_metrics_cron_job
* initialize_budget_metrics_cron_job
* test_initialize_budget_metrics_cron_job
* LITELLM_PROXY_ADMIN_NAME
* fix code qa checks
* test_initialize_budget_metrics_cron_job
* test_initialize_budget_metrics_cron_job
* pod lock manager allow dynamic cron job ID
* fix pod lock manager
* require cronjobid for PodLockManager
* fix DB_SPEND_UPDATE_JOB_NAME acquire / release lock
* add comment on prometheus logger
* add debug statements for emitting key, team budget metrics
* test_pod_lock_manager.py
* test_initialize_budget_metrics_cron_job
* initialize_budget_metrics_cron_job
* initialize_remaining_budget_metrics
* remove outdated test
2025-04-10 16:59:14 -07:00
Krish Dholakia
36cdb79409
Move daily user transaction logging outside of 'disable_spend_logs' flag - different tables ( #9772 )
...
* refactor(db_spend_update_writer.py): aggregate table is entirely different
* test(test_db_spend_update_writer.py): add unit test to ensure if disable_spend_logs is true daily user transactions is still logged
* test: fix test
2025-04-05 09:58:16 -07:00
Ishaan Jaff
be9e9e6db5
refactor file structure
2025-04-01 18:30:48 -07:00
Ishaan Jaff
f122cabd24
fix spend update queue
2025-04-01 17:49:28 -07:00
Ishaan Jaff
9270823d6c
refactor daily spend updates to use new Queue DS
2025-04-01 17:45:19 -07:00
Ishaan Jaff
b845b8f863
use typed data structure for queue
2025-03-31 19:28:17 -07:00
Ishaan Jaff
5b706c049a
fix update_end_user_spend
2025-03-31 19:05:58 -07:00
Ishaan Jaff
fc72a61627
fix update_database helper on db_spend_update_writer
2025-03-31 19:01:00 -07:00
Ishaan Jaff
93c20b8d2d
fix - locking in memory leads to failing tests
2025-03-28 20:10:58 -07:00
Ishaan Jaff
96e4369160
prisma client in_memory_transaction_lock
2025-03-28 19:52:07 -07:00
Ishaan Jaff
7a63cbe8d0
redis update buffer queue
2025-03-28 17:24:54 -07:00
Ishaan Jaff
36f15f19af
debugging for Redis TX management
2025-03-28 16:16:44 -07:00
Ishaan Jaff
cc9f47b1d2
fix loc of DBSpendUpdateTransactions
2025-03-28 15:17:01 -07:00
Ishaan Jaff
fa304a538d
use helper methods _commit_spend_updates_to_db_without_redis_buffer
2025-03-28 15:12:19 -07:00
Ishaan Jaff
17d27fcd36
rename pod lock manager
2025-03-28 12:57:00 -07:00
Ishaan Jaff
39cc75c36b
handle commit updates to the DB
2025-03-27 22:55:36 -07:00
Ishaan Jaff
a1a038aa19
fix typo on codebase
2025-03-27 22:36:00 -07:00
Ishaan Jaff
3bd005f148
fix db spend update buffer
2025-03-27 22:34:15 -07:00
Ishaan Jaff
081bd58ea5
update the correct set of txs
2025-03-27 21:54:12 -07:00
Ishaan Jaff
9efd1814bd
use lock manager for update spend job
2025-03-27 21:43:02 -07:00
Ishaan Jaff
7c4a73ba39
use redis update buffer class
2025-03-27 19:12:51 -07:00
Ishaan Jaff
d03b087e25
basic structure for commit update txs to redis
2025-03-27 18:21:33 -07:00
Ishaan Jaff
33f877e8b0
refactor, use commit_update_transactions_to_db
2025-03-27 18:07:23 -07:00
Ishaan Jaff
7f24714576
use DBSpendUpdateWriter common function for
2025-03-27 17:04:50 -07:00
Ishaan Jaff
a43c7e9104
use simple static methods for updating spend
2025-03-27 16:57:12 -07:00
Ishaan Jaff
3b5fd92f75
fix DBSpendUpdateWriter
2025-03-27 16:44:46 -07:00
Ishaan Jaff
2c9d1cd4d4
DBSpendUpdateWriter
2025-03-27 16:43:18 -07:00
Ishaan Jaff
c367a10c1c
use DBSpendUpdateWriter class for managing DB spend updates
2025-03-27 16:31:23 -07:00
