Ishaan Jaff
9e7c67805b
get_supported_openai_params
2025-04-02 06:52:07 -07:00
Krish Dholakia
6c69ad4c89
fix(model_management_endpoints.py): fix allowing team admins to update team models ( #9697 )
...
Read Version from pyproject.toml / read-version (push) Successful in 17s
Helm unit test / unit-test (push) Successful in 22s
* fix(model_management_endpoints.py): fix allowing team admins to update their models
* test(test_models.py): add e2e test to for team model flow
ensure team admin can always add / edit / delete team models
2025-04-01 22:28:15 -07:00
Krish Dholakia
3d0313b15b
Litellm user daily activity allow non admin usage ( #9695 )
...
* feat(internal_user_endpoints.py): allow non-admin to view their own usage via `/user/daily/activity` route
* fix(leftnav.tsx): allow users to view their own usage via new_usage.tsx
allows internal users to see their usage via new api
Handles 1m+ spend logs scenario
* fix(leftnav.tsx): allow all users to see new usage tab
2025-04-01 22:27:26 -07:00
Krish Dholakia
23051d89dd
fix(streaming_handler.py): fix completion start time tracking ( #9688 )
...
* fix(streaming_handler.py): fix completion start time tracking
Fixes https://github.com/BerriAI/litellm/issues/9210
* feat(anthropic/chat/transformation.py): map openai 'reasoning_effort' to anthropic 'thinking' param
Fixes https://github.com/BerriAI/litellm/issues/9022
* feat: map 'reasoning_effort' to 'thinking' param across bedrock + vertex
Closes https://github.com/BerriAI/litellm/issues/9022#issuecomment-2705260808
2025-04-01 22:00:56 -07:00
Tomer Bin
0690f7a3cb
Virtual key based policies in Aim Guardrails ( #9499 )
...
* report key alias to aim
* send litellm version to aim
* Update docs
* blacken
* add docs
* Add info part about virtual keys specific guards
* sort guardrails alphabetically
* fix ruff
2025-04-01 21:57:23 -07:00
Ishaan Jaff
4080fe54d5
clean up o series
2025-04-01 21:21:41 -07:00
Ishaan Jaff
5f286fe147
fix _check_valid_arg
2025-04-01 21:20:31 -07:00
Ishaan Jaff
f7129e5e59
fix _apply_openai_param_overrides
2025-04-01 21:17:59 -07:00
Ishaan Jaff
ac3399238e
Merge pull request #9690 from BerriAI/litellm_fixes_for_deadlock_round_2
...
[Reliability Fixes] - Ensure no deadlocks occur when updating `DailyUserSpendTransaction`
2025-04-01 20:59:41 -07:00
Ishaan Jaff
9acda77b75
add allowed_openai_params
2025-04-01 19:54:35 -07:00
Ishaan Jaff
50aa34a4a0
allowed_openai_params as a litellm param
2025-04-01 19:50:31 -07:00
Krrish Dholakia
0cfc3e822c
build: new ui build
2025-04-01 19:44:23 -07:00
Krish Dholakia
60863bb134
UI - New Usage Tab fixes ( #9696 )
...
* fix(new_usage.tsx): enable smooth scrolling - remove double scroll bars
enables easier viewing of overflowing content
* fix(new_usage.tsx): fix ordering of daily spend
ensure always from earliest to latest date
* feat(internal_user_endpoints.py): return key alias on `/user/daily/activity`
Enables easier consumption on UI
* fix(new_usage.tsx): show key alias on usage tab
* feat(activity_metric.tsx): new activity panel - showing spend per model per day
allows debugging if models are not being tracked
* fix(top_key_view.tsx): use consistent param for selecting key
2025-04-01 19:36:00 -07:00
Ishaan Jaff
c454dbec30
get_supported_openai_params for o-1 series models
2025-04-01 19:03:50 -07:00
Ishaan Jaff
44bc8974c8
BaseUpdateQueue
2025-04-01 18:31:54 -07:00
Ishaan Jaff
8dc792139e
refactor file structure
2025-04-01 18:30:48 -07:00
Ishaan Jaff
07fc5a72f1
add debug statement
2025-04-01 17:54:52 -07:00
Ishaan Jaff
827ad38e7f
fix spend update queue
2025-04-01 17:49:28 -07:00
Ishaan Jaff
4c14550721
refactor daily spend updates to use new Queue DS
2025-04-01 17:45:19 -07:00
Ishaan Jaff
4ddca7a79c
Merge branch 'main' into litellm_fix_service_account_behavior
2025-04-01 12:04:28 -07:00
Ishaan Jaff
61b609f320
Merge pull request #9673 from BerriAI/litellm_qa_deadlock_fixes
...
[Reliability] - Ensure new Redis + DB architecture tracks spend accurately
2025-04-01 12:04:03 -07:00
Krrish Dholakia
40a792472b
build(enterprise.md): add why enterprise to docs
2025-04-01 11:27:03 -07:00
Ishaan Jaff
01bd418e0b
remove deprecated logic
2025-04-01 07:55:28 -07:00
Ishaan Jaff
f8ce30763e
fix check_if_token_is_service_account
2025-04-01 07:36:35 -07:00
Krish Dholakia
62ad84fb64
UI (new_usage.tsx): Report 'total_tokens' + report success/failure calls ( #9675 )
...
* feat(internal_user_endpoints.py): return 'total_tokens' in `/user/daily/analytics`
* test(test_internal_user_endpoints.py): add unit test to assert spend metrics and dailyspend metadata always report the same fields
* build(schema.prisma): record success + failure calls to daily user table
allows understanding why model requests might exceed provider requests (e.g. user hit rate limit error)
* fix(internal_user_endpoints.py): report success / failure requests in API
* fix(proxy/utils.py): default to success
status can be missing or none at times for successful requests
* feat(new_usage.tsx): show success/failure calls on UI
* style(new_usage.tsx): ui cleanup
* fix: fix linting error
* fix: fix linting error
* feat(litellm-proxy-extras/): add new migration files
2025-03-31 22:48:43 -07:00
Krish Dholakia
f2a7edaddc
fix(proxy_server.py): Fix "Circular reference detected" error when max_parallel_requests = 0 ( #9671 )
...
* fix(proxy_server.py): remove non-functional parent backoff/retry on /chat/completion
Causes circular reference error
* fix(http_parsing_utils.py): safely return parsed body - don't allow mutation of cached request body by client functions
Root cause fix for circular reference error
* Revert "fix: Anthropic prompt caching on GCP Vertex AI (#9605 )" (#9670 )
This reverts commit a8673246dc
2025-03-31 22:06:02 -07:00
Ishaan Jaff
d0a7e44a6e
fix linting
2025-03-31 21:33:05 -07:00
Krish Dholakia
ef88792ad3
Revert "Revert "Correct Databricks llama3.3-70b endpoint and add databricks c…" ( #9676 )
...
This reverts commit 1e26c594cb
2025-03-31 21:30:50 -07:00
Krish Dholakia
722f3ff0e6
fix(cost_calculator.py): allows checking received + sent model name when checking for cost calculation ( #9669 )
...
Fixes issue introduced by dfb838eaff (r154667517)
2025-03-31 21:29:48 -07:00
Ishaan Jaff
115946d402
unit testing for SpendUpdateQueue
2025-03-31 21:25:24 -07:00
Krish Dholakia
5ad2fbcba6
Openrouter streaming fixes + Anthropic 'file' message support ( #9667 )
...
* fix(openrouter/transformation.py): Handle error in openrouter stream
Fixes https://github.com/Aider-AI/aider/issues/3550
* test(test_openrouter_chat_transformation.py): add unit tests
* feat(anthropic/chat/transformation.py): add openai 'file' message content type support
Closes https://github.com/BerriAI/litellm/issues/9463
* fix(factory.py): add bedrock converse support for openai 'file' message content type
Closes https://github.com/BerriAI/litellm/issues/9463
2025-03-31 21:22:59 -07:00
Ishaan Jaff
f7ddc583f0
linting fix get_aggregated_db_spend_update_transactions
2025-03-31 21:15:21 -07:00
Ishaan Jaff
bc5cc51b9d
Merge pull request #9567 from BerriAI/litellm_anthropic_messages_improvements
...
[Refactor] - Expose litellm.messages.acreate() and litellm.messages.create() to make LLM API calls in Anthropic API spec
2025-03-31 20:50:30 -07:00
Ishaan Jaff
5fa5c1154e
add SpendUpdateQueue
2025-03-31 19:42:20 -07:00
Ishaan Jaff
6d4a6a8402
add spend tracking config.yaml
2025-03-31 19:42:00 -07:00
Ishaan Jaff
aa8261af89
test fixes
2025-03-31 19:33:10 -07:00
Ishaan Jaff
811f488ca3
get_aggregated_db_spend_update_transactions
2025-03-31 19:31:53 -07:00
Ishaan Jaff
71e772dd4a
use typed data structure for queue
2025-03-31 19:28:17 -07:00
Ishaan Jaff
4959512170
fix update_end_user_spend
2025-03-31 19:05:58 -07:00
Ishaan Jaff
73d6af5572
fix docstring
2025-03-31 19:03:02 -07:00
Ishaan Jaff
95e674d1e9
test spend accuracy
2025-03-31 19:01:17 -07:00
Ishaan Jaff
3e16a51ca6
fix update_database helper on db_spend_update_writer
2025-03-31 19:01:00 -07:00
Ishaan Jaff
bcd49204f6
use spend_update_queue for RedisUpdateBuffer
2025-03-31 18:40:52 -07:00
Ishaan Jaff
efe6d375e9
add new SpendUpdateQueue
2025-03-31 18:40:03 -07:00
Krish Dholakia
46b3dbde8f
Revert "fix: Anthropic prompt caching on GCP Vertex AI ( #9605 )" ( #9670 )
...
This reverts commit a8673246dc
2025-03-31 17:13:55 -07:00
Krish Dholakia
1e26c594cb
Revert "Correct Databricks llama3.3-70b endpoint and add databricks claude 3.…" ( #9668 )
...
This reverts commit dfb838eaff
2025-03-31 17:02:21 -07:00
Ishaan Jaff
d91bf3cc91
docs anthropic messages interface
2025-03-31 16:53:39 -07:00
Ishaan Jaff
ceab3cfa7c
fix mypy lint errors
2025-03-31 16:52:39 -07:00
Ishaan Jaff
0d48652012
Merge pull request #9515 from BerriAI/litellm_sagemaker_fix_stream
...
[Bug fix] - Sagemaker endpoint with inference component streaming
2025-03-31 16:18:10 -07:00
Ishaan Jaff
3d0d5e1473
run black formatting
2025-03-31 16:12:31 -07:00
