Ishaan Jaff
b35c5f9d30
Merge branch 'main' into litellm_contributors_apr_22
2025-04-22 15:47:26 -07:00
Ishaan Jaff
44264ab6d6
fix failing agent ops test
2025-04-22 14:39:50 -07:00
Ishaan Jaff
6328018c05
fixes importlib
2025-04-22 14:18:34 -07:00
Krish Dholakia
66680c421d
Add global filtering to Users tab ( #10195 )
...
* style(internal_user_endpoints.py): add response model to `/user/list` endpoint
make sure we maintain consistent response spec
* fix(key_management_endpoints.py): return 'created_at' and 'updated_at' on `/key/generate`
Show 'created_at' on UI when key created
* test(test_keys.py): add e2e test to ensure created at is always returned
* fix(view_users.tsx): support global search by user email
allows easier search
* test(search_users.spec.ts): add e2e test ensure user search works on admin ui
* fix(view_users.tsx): support filtering user by role and user id
More powerful filtering on internal users table
* fix(view_users.tsx): allow filtering users by team
* style(view_users.tsx): cleanup ui to show filters in consistent style
* refactor(view_users.tsx): cleanup to just use 1 variable for the data
* fix(view_users.tsx): cleanup use effect hooks
* fix(internal_user_endpoints.py): fix check to pass testing
* test: update tests
* test: update tests
* Revert "test: update tests"
This reverts commit 6553eeb232
2025-04-22 13:59:43 -07:00
Nathan Brake
08e5bdf542
Update utils.py ( #10201 )
2025-04-22 13:54:46 -07:00
Dwij
b2955a2bdd
Add AgentOps Integration to LiteLLM ( #9685 )
...
* feat(sidebars): add new item for agentops integration in Logging & Observability category
* Update agentops_integration.md to enhance title formatting and remove redundant section
* Enhance AgentOps integration in documentation and codebase by removing LiteLLMCallbackHandler references, adding environment variable configurations, and updating logging initialization for AgentOps support.
* Update AgentOps integration documentation to include instructions for obtaining API keys and clarify environment variable setup.
* Add unit tests for AgentOps integration and improve error handling in token fetching
* Add unit tests for AgentOps configuration and token fetching functionality
* Corrected agentops test directory
* Linting fix
* chore: add OpenTelemetry dependencies to pyproject.toml
* chore: update OpenTelemetry dependencies and add new packages in pyproject.toml and poetry.lock
2025-04-22 10:29:01 -07:00
Ishaan Jaff
ebfff975d4
docs responses routing
Read Version from pyproject.toml / read-version (push) Successful in 19s
Helm unit test / unit-test (push) Successful in 52s
2025-04-21 23:05:53 -07:00
Krish Dholakia
a7db0df043
Gemini-2.5-flash improvements ( #10198 )
...
* fix(vertex_and_google_ai_studio_gemini.py): allow thinking budget = 0
Fixes https://github.com/BerriAI/litellm/issues/10121
* fix(vertex_and_google_ai_studio_gemini.py): handle nuance in counting exclusive vs. inclusive tokens
Addresses https://github.com/BerriAI/litellm/pull/10141#discussion_r2052272035
2025-04-21 22:48:00 -07:00
Ishaan Jaff
d1fb051d25
bump: version 1.67.0 → 1.67.1
2025-04-21 22:43:13 -07:00
Ishaan Jaff
7cb95bcc96
[Bug Fix] caching does not account for thinking or reasoning_effort config ( #10140 )
...
* _get_litellm_supported_chat_completion_kwargs
* test caching with thinking
2025-04-21 22:39:40 -07:00
Ishaan Jaff
104e4cb1bc
[Feat] Add infinity embedding support (contributor pr) ( #10196 )
...
* Feature - infinity support for #8764 (#10009 )
* Added support for infinity embeddings
* Added test cases
* Fixed tests and api base
* Updated docs and tests
* Removed unused import
* Updated signature
* Added support for infinity embeddings
* Added test cases
* Fixed tests and api base
* Updated docs and tests
* Removed unused import
* Updated signature
* Updated validate params
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* fix InfinityEmbeddingConfig
---------
Co-authored-by: Prathamesh Saraf <pratamesh1867@gmail.com>
2025-04-21 20:01:29 -07:00
Ishaan Jaff
0c2f705417
[Feat] Add Responses API - Routing Affinity logic for sessions ( #10193 )
...
* test for test_responses_api_routing_with_previous_response_id
* test_responses_api_routing_with_previous_response_id
* add ResponsesApiDeploymentCheck
* ResponsesApiDeploymentCheck
* ResponsesApiDeploymentCheck
* fix ResponsesApiDeploymentCheck
* test_responses_api_routing_with_previous_response_id
* ResponsesApiDeploymentCheck
* test_responses_api_deployment_check.py
* docs routing affinity
* simplify ResponsesApiDeploymentCheck
* test response id
* fix code quality check
2025-04-21 20:00:27 -07:00
Ishaan Jaff
4eac0f64f3
[Feat] Pass through endpoints - ensure PassthroughStandardLoggingPayload is logged and contains method, url, request/response body ( #10194 )
...
* ensure passthrough_logging_payload is filled in kwargs
* test_assistants_passthrough_logging
* test_assistants_passthrough_logging
* test_assistants_passthrough_logging
* test_threads_passthrough_logging
* test _init_kwargs_for_pass_through_endpoint
* _init_kwargs_for_pass_through_endpoint
2025-04-21 19:46:22 -07:00
Krrish Dholakia
4a50cf10fb
build: update ui build
Read Version from pyproject.toml / read-version (push) Successful in 23s
Helm unit test / unit-test (push) Successful in 25s
2025-04-21 16:26:36 -07:00
Krish Dholakia
89131d8ed3
Remove user_id from url ( #10192 )
...
* fix(user_dashboard.tsx): initial commit using user id from jwt instead of url
* fix(proxy_server.py): remove user id from url
fixes security issue around sharing url's
* fix(user_dashboard.tsx): handle user id being null
2025-04-21 16:22:57 -07:00
Krrish Dholakia
a34778dda6
build(ui/): update ui build
...
supports new non-user id in url flow
2025-04-21 16:22:28 -07:00
Krish Dholakia
0c3b7bb37d
fix(router.py): handle edge case where user sets 'model_group' inside… ( #10191 )
...
* fix(router.py): handle edge case where user sets 'model_group' inside 'model_info'
* fix(key_management_endpoints.py): security fix - return hashed token in 'token' field
Ensures when creating a key on UI - only hashed token shown
* test(test_key_management_endpoints.py): add unit test
* test: update test
2025-04-21 16:17:45 -07:00
Nilanjan De
03245c732a
Fix: Potential SQLi in spend_management_endpoints.py ( #9878 )
...
* fix: Potential SQLi in spend_management_endpoints.py
* fix tests
* test: add tests for global spend keys endpoint
* chore: update error message
* chore: lint
* chore: rename test
2025-04-21 14:29:38 -07:00
Li Yang
10257426a2
fix(bedrock): wrong system prompt transformation ( #10120 )
...
Read Version from pyproject.toml / read-version (push) Successful in 16s
Helm unit test / unit-test (push) Successful in 25s
* fix(bedrock): wrong system transformation
* chore: add one more test case
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-04-21 08:48:14 -07:00
Marty Sullivan
0b63c7a2eb
Model pricing updates for Azure & VertexAI ( #10178 )
Read Version from pyproject.toml / read-version (push) Successful in 23s
Helm unit test / unit-test (push) Successful in 27s
2025-04-20 11:33:45 -07:00
Krrish Dholakia
1ff7625984
docs: cleanup
2025-04-20 09:26:05 -07:00
Krrish Dholakia
aa55103486
docs: cleanup doc
Read Version from pyproject.toml / read-version (push) Successful in 19s
Helm unit test / unit-test (push) Successful in 26s
2025-04-20 09:20:47 -07:00
Krrish Dholakia
1d9b58688b
docs(sidebars.js): place scim doc in correct place
2025-04-20 09:20:10 -07:00
Krish Dholakia
ce828408da
fix(proxy_server.py): pass llm router to get complete model list ( #10176 )
...
Read Version from pyproject.toml / read-version (push) Successful in 21s
Helm unit test / unit-test (push) Successful in 27s
allows model auth to work
2025-04-19 22:27:49 -07:00
Krish Dholakia
e0a613f88a
fix(common_daily_activity.py): support empty entity id field ( #10175 )
...
* fix(common_daily_activity.py): support empty entity id field
allows returning empty response when user is not admin and does not belong to any team
* test(test_common_daily_activity.py): add unit testing
2025-04-19 22:20:28 -07:00
Ishaan Jaff
72f6bd3972
fix azure foundry phi error
2025-04-19 22:10:18 -07:00
Ishaan Jaff
36bcb3de4e
fix models appearing under test key page
2025-04-19 21:37:08 -07:00
Krrish Dholakia
bb13ac45c8
docs(index.md): cleanup
2025-04-19 19:16:10 -07:00
Ishaan Jaff
1be36be72e
Litellm docs SCIM ( #10174 )
...
* docs scim
* docs SCIM stash
* docs litellm SCIM
* docs fix
* docs scim with LiteLLM
2025-04-19 18:29:09 -07:00
Krish Dholakia
55a17730fb
fix(transformation.py): pass back in gemini thinking content to api ( #10173 )
...
Ensures thinking content always returned
2025-04-19 18:03:05 -07:00
Krish Dholakia
bbfcb1ac7e
Litellm release notes 04 19 2025 ( #10169 )
...
* docs(index.md): initial draft release notes
* docs: note all pending docs
* build(model_prices_and_context_window.json): add o3, gpt-4.1, o4-mini pricing
* docs(vllm.md): update vllm doc to show file message type support
* docs(mistral.md): add mistral passthrough route doc
* docs(gemini.md): add gemini thinking to docs
* docs(vertex.md): add thinking/reasoning content for gemini models to docs
* docs(index.md): more links
* docs(index.md): add more links, images
* docs(index.md): cleanup highlights
2025-04-19 17:26:30 -07:00
Ishaan Jaff
daf024bad1
Supported Responses API Parameters
2025-04-19 17:14:53 -07:00
Ishaan Jaff
f39d917886
[Docs] Responses API ( #10172 )
...
* docs litellm responses api
* doc fix
* docs responses API
* add get_supported_openai_params for LiteLLMCompletionResponsesConfig
* add Supported Responses API Parameters
2025-04-19 17:10:45 -07:00
Ishaan Jaff
7c3df984da
can_user_call_model ( #10170 )
Read Version from pyproject.toml / read-version (push) Successful in 51s
Helm unit test / unit-test (push) Successful in 51s
2025-04-19 16:46:00 -07:00
Ishaan Jaff
431b230f07
[UI] Bug Fix, team model selector ( #10171 )
...
* fix tooltip
* bug fix fix team model selector
2025-04-19 16:31:38 -07:00
Ishaan Jaff
6206649219
bump: version 1.66.3 → 1.67.0
2025-04-19 14:41:16 -07:00
Ishaan Jaff
653570824a
Bug Fix - Responses API, Loosen restrictions on allowed environments for computer use tool ( #10168 )
...
* loosen allowed types on ComputerToolParam
* test_basic_computer_use_preview_tool_call
2025-04-19 14:40:32 -07:00
Ishaan Jaff
c80e984d7e
ui new build
2025-04-19 14:19:33 -07:00
Ishaan Jaff
b0024bb229
[Bug Fix] Spend Tracking Bug Fix, don't modify in memory default litellm params ( #10167 )
...
* _update_kwargs_with_default_litellm_params
* test_update_kwargs_does_not_mutate_defaults_and_merges_metadata
2025-04-19 14:13:59 -07:00
Ishaan Jaff
0717369ae6
[Feat] Expose Responses API on LiteLLM UI Test Key Page ( #10166 )
...
* add /responses API on UI
* add makeOpenAIResponsesRequest
* add makeOpenAIResponsesRequest
* fix add responses API on UI
* fix endpoint selector
* responses API render chunks on litellm chat ui
* fixes to streaming iterator
* fix render responses completed events
* fixes for MockResponsesAPIStreamingIterator
* transform_responses_api_request_to_chat_completion_request
* fix for responses API
* test_basic_openai_responses_api_streaming
* fix base responses api tests
2025-04-19 13:18:54 -07:00
Krish Dholakia
03b5399f86
test(utils.py): handle scenario where text tokens + reasoning tokens … ( #10165 )
...
* test(utils.py): handle scenario where text tokens + reasoning tokens set, but reasoning tokens not charged separately
Addresses https://github.com/BerriAI/litellm/pull/10141#discussion_r2051555332
* fix(vertex_and_google_ai_studio.py): only set content if non-empty str
2025-04-19 12:32:38 -07:00
Ong Khai Wei
99db1b7690
to get API key from environment viarble of WATSONX_APIKEY ( #10131 )
2025-04-19 11:25:14 -07:00
Classic298
173ef01ef6
Update model_prices_and_context_window_backup.json ( #10122 )
...
* Update model_prices_and_context_window_backup.json
* Update model_prices_and_context_window_backup.json
2025-04-19 11:24:52 -07:00
Krish Dholakia
5c929317cd
fix(triton/completion/transformation.py): remove bad_words / stop wor… ( #10163 )
...
* fix(triton/completion/transformation.py): remove bad_words / stop words from triton call
parameter 'bad_words' has invalid type. It should be either 'int', 'bool', or 'string'.
* fix(proxy_track_cost_callback.py): add debug logging for track cost callback error
2025-04-19 11:23:37 -07:00
Krish Dholakia
f08a4e3c06
Support 'file' message type for VLLM video url's + Anthropic redacted message thinking support ( #10129 )
...
* feat(hosted_vllm/chat/transformation.py): support calling vllm video url with openai 'file' message type
allows switching between gemini/vllm easily
* [WIP] redacted thinking tests (#9044 )
* WIP: redacted thinking tests
* test: add test for redacted thinking in assistant message
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* fix(anthropic/chat/transformation.py): support redacted thinking block on anthropic completion
Fixes https://github.com/BerriAI/litellm/issues/9058
* fix(anthropic/chat/handler.py): transform anthropic redacted messages on streaming
Fixes https://github.com/BerriAI/litellm/issues/9058
* fix(bedrock/): support redacted text on streaming + non-streaming
Fixes https://github.com/BerriAI/litellm/issues/9058
* feat(litellm_proxy/chat/transformation.py): support 'reasoning_effort' param for proxy
allows using reasoning effort with thinking models on proxy
* test: update tests
* fix(utils.py): fix linting error
* fix: fix linting errors
* fix: fix linting errors
* fix: fix linting error
* fix: fix linting errors
* fix(anthropic/chat/transformation.py): fix returning citations in chat completion
---------
Co-authored-by: Johann Miller <22018973+johannkm@users.noreply.github.com>
2025-04-19 11:16:37 -07:00
Ishaan Jaff
3c463f6715
test fix - output_cost_per_reasoning_token was added to model cost map
2025-04-19 10:02:25 -07:00
Krish Dholakia
2508ca71cb
Handle fireworks ai tool calling response ( #10130 )
...
* feat(fireworks_ai/chat): handle tool calling with fireworks ai correctly
Fixes https://github.com/BerriAI/litellm/issues/7209
* fix(utils.py): handle none type in message
* fix: fix model name in test
* fix(utils.py): fix validate check for openai messages
* fix: fix model returned
* fix(main.py): fix text completion routing
* test: update testing
* test: skip test - cohere having RBAC issues
2025-04-19 09:37:45 -07:00
Krrish Dholakia
b4f2b3dad1
test: update test to be more robust to usage updates
2025-04-19 09:26:26 -07:00
Ishaan Jaff
8ae2653280
fix calculated cache key for tests
2025-04-19 09:25:11 -07:00
Ishaan Jaff
97d7a5e78e
fix deployment name
2025-04-19 09:23:22 -07:00
