Ishaan Jaff
f7129e5e59
fix _apply_openai_param_overrides
2025-04-01 21:17:59 -07:00
Ishaan Jaff
ac3399238e
Merge pull request #9690 from BerriAI/litellm_fixes_for_deadlock_round_2
...
[Reliability Fixes] - Ensure no deadlocks occur when updating `DailyUserSpendTransaction`
2025-04-01 20:59:41 -07:00
Ishaan Jaff
9acda77b75
add allowed_openai_params
2025-04-01 19:54:35 -07:00
Ishaan Jaff
50aa34a4a0
allowed_openai_params as a litellm param
2025-04-01 19:50:31 -07:00
Krrish Dholakia
0cfc3e822c
build: new ui build
2025-04-01 19:44:23 -07:00
Krish Dholakia
60863bb134
UI - New Usage Tab fixes ( #9696 )
...
* fix(new_usage.tsx): enable smooth scrolling - remove double scroll bars
enables easier viewing of overflowing content
* fix(new_usage.tsx): fix ordering of daily spend
ensure always from earliest to latest date
* feat(internal_user_endpoints.py): return key alias on `/user/daily/activity`
Enables easier consumption on UI
* fix(new_usage.tsx): show key alias on usage tab
* feat(activity_metric.tsx): new activity panel - showing spend per model per day
allows debugging if models are not being tracked
* fix(top_key_view.tsx): use consistent param for selecting key
2025-04-01 19:36:00 -07:00
Ishaan Jaff
c454dbec30
get_supported_openai_params for o-1 series models
2025-04-01 19:03:50 -07:00
Ishaan Jaff
44bc8974c8
BaseUpdateQueue
2025-04-01 18:31:54 -07:00
Ishaan Jaff
8dc792139e
refactor file structure
2025-04-01 18:30:48 -07:00
Ishaan Jaff
07fc5a72f1
add debug statement
2025-04-01 17:54:52 -07:00
Ishaan Jaff
827ad38e7f
fix spend update queue
2025-04-01 17:49:28 -07:00
Ishaan Jaff
4c14550721
refactor daily spend updates to use new Queue DS
2025-04-01 17:45:19 -07:00
Ishaan Jaff
4ddca7a79c
Merge branch 'main' into litellm_fix_service_account_behavior
2025-04-01 12:04:28 -07:00
Ishaan Jaff
61b609f320
Merge pull request #9673 from BerriAI/litellm_qa_deadlock_fixes
...
[Reliability] - Ensure new Redis + DB architecture tracks spend accurately
2025-04-01 12:04:03 -07:00
Krrish Dholakia
40a792472b
build(enterprise.md): add why enterprise to docs
2025-04-01 11:27:03 -07:00
Ishaan Jaff
01bd418e0b
remove deprecated logic
2025-04-01 07:55:28 -07:00
Ishaan Jaff
f8ce30763e
fix check_if_token_is_service_account
2025-04-01 07:36:35 -07:00
Krish Dholakia
62ad84fb64
UI (new_usage.tsx): Report 'total_tokens' + report success/failure calls ( #9675 )
...
* feat(internal_user_endpoints.py): return 'total_tokens' in `/user/daily/analytics`
* test(test_internal_user_endpoints.py): add unit test to assert spend metrics and dailyspend metadata always report the same fields
* build(schema.prisma): record success + failure calls to daily user table
allows understanding why model requests might exceed provider requests (e.g. user hit rate limit error)
* fix(internal_user_endpoints.py): report success / failure requests in API
* fix(proxy/utils.py): default to success
status can be missing or none at times for successful requests
* feat(new_usage.tsx): show success/failure calls on UI
* style(new_usage.tsx): ui cleanup
* fix: fix linting error
* fix: fix linting error
* feat(litellm-proxy-extras/): add new migration files
2025-03-31 22:48:43 -07:00
Krish Dholakia
f2a7edaddc
fix(proxy_server.py): Fix "Circular reference detected" error when max_parallel_requests = 0 ( #9671 )
...
* fix(proxy_server.py): remove non-functional parent backoff/retry on /chat/completion
Causes circular reference error
* fix(http_parsing_utils.py): safely return parsed body - don't allow mutation of cached request body by client functions
Root cause fix for circular reference error
* Revert "fix: Anthropic prompt caching on GCP Vertex AI (#9605 )" (#9670 )
This reverts commit a8673246dc
2025-03-31 22:06:02 -07:00
Ishaan Jaff
d0a7e44a6e
fix linting
2025-03-31 21:33:05 -07:00
Krish Dholakia
ef88792ad3
Revert "Revert "Correct Databricks llama3.3-70b endpoint and add databricks c…" ( #9676 )
...
This reverts commit 1e26c594cb
2025-03-31 21:30:50 -07:00
Krish Dholakia
722f3ff0e6
fix(cost_calculator.py): allows checking received + sent model name when checking for cost calculation ( #9669 )
...
Fixes issue introduced by dfb838eaff (r154667517)
2025-03-31 21:29:48 -07:00
Ishaan Jaff
115946d402
unit testing for SpendUpdateQueue
2025-03-31 21:25:24 -07:00
Krish Dholakia
5ad2fbcba6
Openrouter streaming fixes + Anthropic 'file' message support ( #9667 )
...
* fix(openrouter/transformation.py): Handle error in openrouter stream
Fixes https://github.com/Aider-AI/aider/issues/3550
* test(test_openrouter_chat_transformation.py): add unit tests
* feat(anthropic/chat/transformation.py): add openai 'file' message content type support
Closes https://github.com/BerriAI/litellm/issues/9463
* fix(factory.py): add bedrock converse support for openai 'file' message content type
Closes https://github.com/BerriAI/litellm/issues/9463
2025-03-31 21:22:59 -07:00
Ishaan Jaff
f7ddc583f0
linting fix get_aggregated_db_spend_update_transactions
2025-03-31 21:15:21 -07:00
Ishaan Jaff
bc5cc51b9d
Merge pull request #9567 from BerriAI/litellm_anthropic_messages_improvements
...
[Refactor] - Expose litellm.messages.acreate() and litellm.messages.create() to make LLM API calls in Anthropic API spec
2025-03-31 20:50:30 -07:00
Ishaan Jaff
5fa5c1154e
add SpendUpdateQueue
2025-03-31 19:42:20 -07:00
Ishaan Jaff
6d4a6a8402
add spend tracking config.yaml
2025-03-31 19:42:00 -07:00
Ishaan Jaff
aa8261af89
test fixes
2025-03-31 19:33:10 -07:00
Ishaan Jaff
811f488ca3
get_aggregated_db_spend_update_transactions
2025-03-31 19:31:53 -07:00
Ishaan Jaff
71e772dd4a
use typed data structure for queue
2025-03-31 19:28:17 -07:00
Ishaan Jaff
4959512170
fix update_end_user_spend
2025-03-31 19:05:58 -07:00
Ishaan Jaff
73d6af5572
fix docstring
2025-03-31 19:03:02 -07:00
Ishaan Jaff
95e674d1e9
test spend accuracy
2025-03-31 19:01:17 -07:00
Ishaan Jaff
3e16a51ca6
fix update_database helper on db_spend_update_writer
2025-03-31 19:01:00 -07:00
Ishaan Jaff
bcd49204f6
use spend_update_queue for RedisUpdateBuffer
2025-03-31 18:40:52 -07:00
Ishaan Jaff
efe6d375e9
add new SpendUpdateQueue
2025-03-31 18:40:03 -07:00
Krish Dholakia
46b3dbde8f
Revert "fix: Anthropic prompt caching on GCP Vertex AI ( #9605 )" ( #9670 )
...
This reverts commit a8673246dc
2025-03-31 17:13:55 -07:00
Krish Dholakia
1e26c594cb
Revert "Correct Databricks llama3.3-70b endpoint and add databricks claude 3.…" ( #9668 )
...
This reverts commit dfb838eaff
2025-03-31 17:02:21 -07:00
Ishaan Jaff
d91bf3cc91
docs anthropic messages interface
2025-03-31 16:53:39 -07:00
Ishaan Jaff
ceab3cfa7c
fix mypy lint errors
2025-03-31 16:52:39 -07:00
Ishaan Jaff
0d48652012
Merge pull request #9515 from BerriAI/litellm_sagemaker_fix_stream
...
[Bug fix] - Sagemaker endpoint with inference component streaming
2025-03-31 16:18:10 -07:00
Ishaan Jaff
3d0d5e1473
run black formatting
2025-03-31 16:12:31 -07:00
Anton Abilov
dfb838eaff
Correct Databricks llama3.3-70b endpoint and add databricks claude 3.7 sonnet endpoint ( #9661 )
2025-03-31 15:22:12 -07:00
Ishaan Jaff
cfab8eec5b
fix code quality check
2025-03-31 14:44:12 -07:00
Ishaan Jaff
bd39a395f1
use new anthropic interface
2025-03-31 14:31:09 -07:00
Ishaan Jaff
01d85d5fb7
Merge branch 'main' into litellm_anthropic_messages_improvements
2025-03-31 14:22:56 -07:00
Ishaan Jaff
83ba96b8c6
Merge branch 'main' into litellm_sagemaker_fix_stream
2025-03-31 14:22:20 -07:00
Sam
a8673246dc
fix: Anthropic prompt caching on GCP Vertex AI ( #9605 )
...
* fix: Anthropic prompt caching on GCP Vertex AI
* test(vertex): anthropic prompt caching
2025-03-29 23:40:34 -07:00
jmarshall-medallia
da953c74a1
Add password to sync sentinel client ( #9622 )
2025-03-29 23:38:54 -07:00
