Commit graph

4515 commits

Author SHA1 Message Date
Ishaan Jaff
93c20b8d2d fix - locking in memory leads to failing tests 2025-03-28 20:10:58 -07:00
Ishaan Jaff
fb98bd3a4a test local spend accuracy 2025-03-28 19:52:39 -07:00
Ishaan Jaff
96e4369160 prisma client in_memory_transaction_lock 2025-03-28 19:52:07 -07:00
Ishaan Jaff
96812a7da0 use asyncio lock for updating PrismaClient txs 2025-03-28 19:51:15 -07:00
Ishaan Jaff
7a63cbe8d0 redis update buffer queue 2025-03-28 17:24:54 -07:00
Ishaan Jaff
36f15f19af debugging for Redis TX management 2025-03-28 16:16:44 -07:00
Ishaan Jaff
b4e9683889 add docstring for DBSpendUpdateTransactions 2025-03-28 15:20:28 -07:00
Ishaan Jaff
cc9f47b1d2 fix loc of DBSpendUpdateTransactions 2025-03-28 15:17:01 -07:00
Ishaan Jaff
fa304a538d use helper methods _commit_spend_updates_to_db_without_redis_buffer 2025-03-28 15:12:19 -07:00
Ishaan Jaff
8dd2863d71 test pod lock manager 2025-03-28 13:31:45 -07:00
Ishaan Jaff
17d27fcd36 rename pod lock manager 2025-03-28 12:57:00 -07:00
Ishaan Jaff
2f5d7b86d9 fix PodLockManager 2025-03-27 23:29:48 -07:00
Ishaan Jaff
0a5db39c89 working cron job updates 2025-03-27 23:13:01 -07:00
Ishaan Jaff
39cc75c36b handle commit updates to the DB 2025-03-27 22:55:36 -07:00
Ishaan Jaff
2a181a45f1 fix schema to work with lock acquisition 2025-03-27 22:54:46 -07:00
Ishaan Jaff
a1a038aa19 fix typo on codebase 2025-03-27 22:36:00 -07:00
Ishaan Jaff
3bd005f148 fix db spend update buffer 2025-03-27 22:34:15 -07:00
Ishaan Jaff
9df978b511 fix typo 2025-03-27 22:31:38 -07:00
Ishaan Jaff
081bd58ea5 update the correct set of txs 2025-03-27 21:54:12 -07:00
Ishaan Jaff
9efd1814bd use lock manager for update spend job 2025-03-27 21:43:02 -07:00
Ishaan Jaff
d0ddcb5d63 update schema 2025-03-27 21:36:22 -07:00
Ishaan Jaff
69ce2e9f55 add model CronJob 2025-03-27 21:10:26 -07:00
Ishaan Jaff
c0dc4fd519 get_all_update_transactions_from_redis 2025-03-27 19:15:30 -07:00
Ishaan Jaff
7c4a73ba39 use redis update buffer class 2025-03-27 19:12:51 -07:00
Ishaan Jaff
d03b087e25 basic structure for commit update txs to redis 2025-03-27 18:21:33 -07:00
Ishaan Jaff
33f877e8b0 refactor, use commit_update_transactions_to_db 2025-03-27 18:07:23 -07:00
Ishaan Jaff
7f24714576 use DBSpendUpdateWriter common function for 2025-03-27 17:04:50 -07:00
Ishaan Jaff
a43c7e9104 use simple static methods for updating spend 2025-03-27 16:57:12 -07:00
Ishaan Jaff
3b5fd92f75 fix DBSpendUpdateWriter 2025-03-27 16:44:46 -07:00
Ishaan Jaff
2c9d1cd4d4 DBSpendUpdateWriter 2025-03-27 16:43:18 -07:00
Ishaan Jaff
eed0612884 fix get_credentials job 2025-03-27 16:34:06 -07:00
Ishaan Jaff
c367a10c1c use DBSpendUpdateWriter class for managing DB spend updates 2025-03-27 16:31:23 -07:00
Krish Dholakia
ed8c63b51e Allow team admins to add/update/delete models on UI + show api base and model id on request logs (#9572)
* feat(view_logs.tsx): show model id + api base in request logs

easier debugging

* fix(index.tsx): fix length of api base

easier viewing

* refactor(leftnav.tsx): show models tab to team admin

* feat(model_dashboard.tsx): add explainer for what the 'models' page is for team admin

helps them understand how they can use it

* feat(model_management_endpoints.py): restrict model add by team to just team admin

allow team admin to add models via non-team keys (e.g. ui token)

* test(test_add_update_models.py): update unit testing for new behaviour

* fix(model_dashboard.tsx): show user the models

* feat(proxy_server.py): add new query param 'user_models_only' to `/v2/model/info`

Allows user to retrieve just the models they've added

Used in UI to show internal users just the models they've added

* feat(model_dashboard.tsx): allow team admins to view their own models

* fix: allow ui user to fetch model cost map

* feat(add_model_tab.tsx): require team admins to specify team when onboarding models

* fix(_types.py): add `/v1/model/info` to info route

`/model/info` was already there

* fix(model_info_view.tsx): allow user to edit a model they created

* fix(model_management_endpoints.py): allow team admin to update team model

* feat(model_managament_endpoints.py): allow team admin to delete team models

* fix(model_management_endpoints.py): don't require team id to be set when adding a model

* fix(proxy_server.py): fix linting error

* fix: fix ui linting error

* fix(model_management_endpoints.py): ensure consistent auth checks on all model calls

* test: remove old test - function no longer exists in same form

* test: add updated mock testing
2025-03-27 12:06:31 -07:00
Krish Dholakia
d58fe5a9f9 Add OpenAI gpt-4o-transcribe support (#9517)
* refactor: introduce new transformation config for gpt-4o-transcribe models

* refactor: expose new transformation configs for audio transcription

* ci: fix config yml

* feat(openai/transcriptions): support provider config transformation on openai audio transcriptions

allows gpt-4o and whisper audio transformation to work as expected

* refactor: migrate fireworks ai + deepgram to new transform request pattern

* feat(openai/): working support for gpt-4o-audio-transcribe

* build(model_prices_and_context_window.json): add gpt-4o-transcribe to model cost map

* build(model_prices_and_context_window.json): specify what endpoints are supported for `/audio/transcriptions`

* fix(get_supported_openai_params.py): fix return

* refactor(deepgram/): migrate unit test to deepgram handler

* refactor: cleanup unused imports

* fix(get_supported_openai_params.py): fix linting error

* test: update test
2025-03-26 23:10:25 -07:00
Ishaan Jaff
155a53c426 fix order of _setup_prisma_client 2025-03-26 20:42:01 -07:00
Ishaan Jaff
e46c789bb7 fix _setup_prisma_client 2025-03-26 20:35:27 -07:00
Ishaan Jaff
6f3be086b5 refactor tests 2025-03-26 20:00:10 -07:00
Ishaan Jaff
d1ff7a5f45 allow proxy to startup on DB unavailable 2025-03-26 19:50:57 -07:00
Ishaan Jaff
3d4fc22eaf bug fix - allow pods to startup when DB is unavailable 2025-03-26 19:49:51 -07:00
Ishaan Jaff
6648e85c97 refactor PrismaDBExceptionHandler 2025-03-26 19:22:24 -07:00
Ishaan Jaff
cb44f8d64d Merge pull request #9533 from BerriAI/litellm_stability_fixes
[Reliability Fixes] - Gracefully handle exceptions when DB is having an outage
2025-03-26 18:57:38 -07:00
Krish Dholakia
eeab48ce5a Add Daily User Spend Aggregate view - allows UI Usage tab to work > 1m rows (#9538)
* ci: update github action

* build(schema.prisma): enable daily user spend table

allows storing aggregate view of user's daily spend

* build(schema.prisma): add new daily user spend table

* feat: working daily user spend tracking

maintains an aggregate view for easier querying in high traffic

* setup_google_dns

* ci: update ci yaml

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-03-26 16:36:36 -07:00
Ishaan Jaff
1274d27534 fix ProxyException 2025-03-26 15:53:33 -07:00
Ishaan Jaff
ba0d8a6d22 fix auth checks 2025-03-26 15:45:58 -07:00
Ishaan Jaff
427b549975 fix get_key_object 2025-03-26 15:41:40 -07:00
Ishaan Jaff
43c402b8b2 is_database_connection_error 2025-03-26 14:55:40 -07:00
Ishaan Jaff
f85c0dac4f Merge branch 'main' into litellm_stability_fixes 2025-03-26 14:41:43 -07:00
Ishaan Jaff
ad5d26f664 fix user_api_key_auth example config 2025-03-26 08:36:11 -07:00
Krish Dholakia
1ab5d49132 Support litellm.api_base for vertex_ai + gemini/ across completion, embedding, image_generation (#9516)
* test(tests): add unit testing for litellm_proxy integration

* fix(cost_calculator.py): fix tracking cost in sdk when calling proxy

* fix(main.py): respect litellm.api_base on `vertex_ai/` and `gemini/` routes

* fix(main.py): consistently support custom api base across gemini + vertexai on embedding + completion

* feat(vertex_ai/): test

* fix: fix linting error

* test: set api base as None before starting loadtest
2025-03-25 23:46:20 -07:00
Ishaan Jaff
fea5213517 add test config 2025-03-25 19:21:51 -07:00