Commit graph

1676 commits

Author SHA1 Message Date
Ishaan Jaff
73a4a4b838 test_basic_openai_responses_get_endpoint 2025-04-23 14:43:29 -07:00
Ishaan Jaff
e82cb8d8bd test_basic_openai_responses_get_endpoint 2025-04-23 07:10:05 -07:00
Prathamesh Saraf
c832b0fded
Added get responses API (#10234) 2025-04-23 07:07:00 -07:00
Krrish Dholakia
f5996b2f6b test: update test to skip 'gemini-pro' - model deprecated
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 19s
Helm unit test / unit-test (push) Successful in 32s
2025-04-23 00:01:02 -07:00
Krish Dholakia
217681eb5e
Litellm dev 04 22 2025 p1 (#10206)
* fix(openai.py): initial commit adding generic event type for openai responses api streaming

Ensures handling for undocumented event types - e.g. "response.reasoning_summary_part.added"

* fix(transformation.py): handle unknown openai response type

* fix(datadog_llm_observability.py): handle dict[str, any] -> dict[str, str] conversion

Fixes https://github.com/BerriAI/litellm/issues/9494

* test: add more unit testing

* test: add unit test

* fix(common_utils.py): fix message with content list

* test: update testing
2025-04-22 23:58:43 -07:00
Krish Dholakia
f670ebeb2f
Users page - new user info pane (#10213)
* feat(user_info_view.tsx): be able to click in and see all teams user is part of

makes it easy to see which teams a user belongs to

* test(ui/): add unit testing for user info view

* fix(user_info_view.tsx): fix linting errors

* fix(login.ts): fix login

* fix: fix linting error
2025-04-22 21:55:47 -07:00
Ishaan Jaff
96e31d205c
feat: Added Missing Attributes For Arize & Phoenix Integration (#10043) (#10215)
* feat: Added Missing Attributes For Arize & Phoenix Integration

* chore: Added noqa for PLR0915 to suppress warning

* chore: Moved Contributor Test to Correct Location

* chore: Removed Redundant Fallback

Co-authored-by: Ali Saleh <saleh.a@turing.com>
2025-04-22 21:34:51 -07:00
Krish Dholakia
5f98d4d7de
UI - Users page - Enable global sorting (allows finding users with highest spend) (#10211)
* fix(view_users.tsx): add time tracking logic to debounce search - prevent new queries from being overwritten by previous ones

* fix(internal_user_endpoints.py): add sort functionality to user list endpoint

* feat(internal_user_endpoints.py): support sort by on `/user/list`

* fix(view_users.tsx): enable global sorting

allows finding user with highest spend

* feat(view_users.tsx): support filtering by sso user id

* test(search_users.spec.ts): add tests to ensure filtering works

* test: add more unit testing
2025-04-22 19:59:53 -07:00
Ishaan Jaff
0dba2886f0 fix test 2025-04-22 18:37:56 -07:00
Ishaan Jaff
868cdd0226
[Feat] Add Support for DELETE /v1/responses/{response_id} on OpenAI, Azure OpenAI (#10205)
* add transform_delete_response_api_request to base responses config

* add transform_delete_response_api_request

* add delete_response_api_handler

* fixes for deleting responses, response API

* add adelete_responses

* add async test_basic_openai_responses_delete_endpoint

* test_basic_openai_responses_delete_endpoint

* working delete for streaming on responses API

* fixes azure transformation

* TestAnthropicResponsesAPITest

* fix code check

* fix linting

* fixes for get_complete_url

* test_basic_openai_responses_streaming_delete_endpoint

* streaming fixes
2025-04-22 18:27:03 -07:00
Ishaan Jaff
44264ab6d6 fix failing agent ops test 2025-04-22 14:39:50 -07:00
Krish Dholakia
66680c421d
Add global filtering to Users tab (#10195)
* style(internal_user_endpoints.py): add response model to `/user/list` endpoint

make sure we maintain consistent response spec

* fix(key_management_endpoints.py): return 'created_at' and 'updated_at' on `/key/generate`

Show 'created_at' on UI when key created

* test(test_keys.py): add e2e test to ensure created at is always returned

* fix(view_users.tsx): support global search by user email

allows easier search

* test(search_users.spec.ts): add e2e test ensure user search works on admin ui

* fix(view_users.tsx): support filtering user by role and user id

More powerful filtering on internal users table

* fix(view_users.tsx): allow filtering users by team

* style(view_users.tsx): cleanup ui to show filters in consistent style

* refactor(view_users.tsx): cleanup to just use 1 variable for the data

* fix(view_users.tsx): cleanup use effect hooks

* fix(internal_user_endpoints.py): fix check to pass testing

* test: update tests

* test: update tests

* Revert "test: update tests"

This reverts commit 6553eeb232.

* fix(view_userts.tsx): add back in 'previous' and 'next' tabs for pagination
2025-04-22 13:59:43 -07:00
Dwij
b2955a2bdd
Add AgentOps Integration to LiteLLM (#9685)
* feat(sidebars): add new item for agentops integration in Logging & Observability category

* Update agentops_integration.md to enhance title formatting and remove redundant section

* Enhance AgentOps integration in documentation and codebase by removing LiteLLMCallbackHandler references, adding environment variable configurations, and updating logging initialization for AgentOps support.

* Update AgentOps integration documentation to include instructions for obtaining API keys and clarify environment variable setup.

* Add unit tests for AgentOps integration and improve error handling in token fetching

* Add unit tests for AgentOps configuration and token fetching functionality

* Corrected agentops test directory

* Linting fix

* chore: add OpenTelemetry dependencies to pyproject.toml

* chore: update OpenTelemetry dependencies and add new packages in pyproject.toml and poetry.lock
2025-04-22 10:29:01 -07:00
Krish Dholakia
a7db0df043
Gemini-2.5-flash improvements (#10198)
* fix(vertex_and_google_ai_studio_gemini.py): allow thinking budget = 0

Fixes https://github.com/BerriAI/litellm/issues/10121

* fix(vertex_and_google_ai_studio_gemini.py): handle nuance in counting exclusive vs. inclusive tokens

Addresses https://github.com/BerriAI/litellm/pull/10141#discussion_r2052272035
2025-04-21 22:48:00 -07:00
Ishaan Jaff
7cb95bcc96
[Bug Fix] caching does not account for thinking or reasoning_effort config (#10140)
* _get_litellm_supported_chat_completion_kwargs

* test caching with thinking
2025-04-21 22:39:40 -07:00
Ishaan Jaff
104e4cb1bc
[Feat] Add infinity embedding support (contributor pr) (#10196)
* Feature - infinity support for #8764 (#10009)

* Added support for infinity embeddings

* Added test cases

* Fixed tests and api base

* Updated docs and tests

* Removed unused import

* Updated signature

* Added support for infinity embeddings

* Added test cases

* Fixed tests and api base

* Updated docs and tests

* Removed unused import

* Updated signature

* Updated validate params

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

* fix InfinityEmbeddingConfig

---------

Co-authored-by: Prathamesh Saraf <pratamesh1867@gmail.com>
2025-04-21 20:01:29 -07:00
Ishaan Jaff
0c2f705417
[Feat] Add Responses API - Routing Affinity logic for sessions (#10193)
* test for test_responses_api_routing_with_previous_response_id

* test_responses_api_routing_with_previous_response_id

* add ResponsesApiDeploymentCheck

* ResponsesApiDeploymentCheck

* ResponsesApiDeploymentCheck

* fix ResponsesApiDeploymentCheck

* test_responses_api_routing_with_previous_response_id

* ResponsesApiDeploymentCheck

* test_responses_api_deployment_check.py

* docs routing affinity

* simplify ResponsesApiDeploymentCheck

* test response id

* fix code quality check
2025-04-21 20:00:27 -07:00
Ishaan Jaff
4eac0f64f3
[Feat] Pass through endpoints - ensure PassthroughStandardLoggingPayload is logged and contains method, url, request/response body (#10194)
* ensure passthrough_logging_payload is filled in kwargs

* test_assistants_passthrough_logging

* test_assistants_passthrough_logging

* test_assistants_passthrough_logging

* test_threads_passthrough_logging

* test _init_kwargs_for_pass_through_endpoint

* _init_kwargs_for_pass_through_endpoint
2025-04-21 19:46:22 -07:00
Krish Dholakia
89131d8ed3
Remove user_id from url (#10192)
* fix(user_dashboard.tsx): initial commit using user id from jwt instead of url

* fix(proxy_server.py): remove user id from url

fixes security issue around sharing url's

* fix(user_dashboard.tsx): handle user id being null
2025-04-21 16:22:57 -07:00
Krish Dholakia
0c3b7bb37d
fix(router.py): handle edge case where user sets 'model_group' inside… (#10191)
* fix(router.py): handle edge case where user sets 'model_group' inside 'model_info'

* fix(key_management_endpoints.py): security fix - return hashed token in 'token' field

Ensures when creating a key on UI - only hashed token shown

* test(test_key_management_endpoints.py): add unit test

* test: update test
2025-04-21 16:17:45 -07:00
Nilanjan De
03245c732a
Fix: Potential SQLi in spend_management_endpoints.py (#9878)
* fix: Potential SQLi in spend_management_endpoints.py

* fix tests

* test: add tests for global spend keys endpoint

* chore: update error message

* chore: lint

* chore: rename test
2025-04-21 14:29:38 -07:00
Li Yang
10257426a2
fix(bedrock): wrong system prompt transformation (#10120)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 16s
Helm unit test / unit-test (push) Successful in 25s
* fix(bedrock): wrong system transformation

* chore: add one more test case

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-04-21 08:48:14 -07:00
Krish Dholakia
e0a613f88a
fix(common_daily_activity.py): support empty entity id field (#10175)
* fix(common_daily_activity.py): support empty entity id field

allows returning empty response when user is not admin and does not belong to any team

* test(test_common_daily_activity.py): add unit testing
2025-04-19 22:20:28 -07:00
Krish Dholakia
55a17730fb
fix(transformation.py): pass back in gemini thinking content to api (#10173)
Ensures thinking content always returned
2025-04-19 18:03:05 -07:00
Ishaan Jaff
653570824a
Bug Fix - Responses API, Loosen restrictions on allowed environments for computer use tool (#10168)
* loosen allowed types on ComputerToolParam

* test_basic_computer_use_preview_tool_call
2025-04-19 14:40:32 -07:00
Ishaan Jaff
b0024bb229
[Bug Fix] Spend Tracking Bug Fix, don't modify in memory default litellm params (#10167)
* _update_kwargs_with_default_litellm_params

* test_update_kwargs_does_not_mutate_defaults_and_merges_metadata
2025-04-19 14:13:59 -07:00
Ishaan Jaff
0717369ae6
[Feat] Expose Responses API on LiteLLM UI Test Key Page (#10166)
* add /responses API on UI

* add makeOpenAIResponsesRequest

* add makeOpenAIResponsesRequest

* fix add responses API on UI

* fix endpoint selector

* responses API render chunks on litellm chat ui

* fixes to streaming iterator

* fix render responses completed events

* fixes for MockResponsesAPIStreamingIterator

* transform_responses_api_request_to_chat_completion_request

* fix for responses API

* test_basic_openai_responses_api_streaming

* fix base responses api tests
2025-04-19 13:18:54 -07:00
Krish Dholakia
03b5399f86
test(utils.py): handle scenario where text tokens + reasoning tokens … (#10165)
* test(utils.py): handle scenario where text tokens + reasoning tokens set, but reasoning tokens not charged separately

Addresses https://github.com/BerriAI/litellm/pull/10141#discussion_r2051555332

* fix(vertex_and_google_ai_studio.py): only set content if non-empty str
2025-04-19 12:32:38 -07:00
Krish Dholakia
5c929317cd
fix(triton/completion/transformation.py): remove bad_words / stop wor… (#10163)
* fix(triton/completion/transformation.py): remove bad_words / stop words from triton call

parameter 'bad_words' has invalid type. It should be either 'int', 'bool', or 'string'.

* fix(proxy_track_cost_callback.py): add debug logging for track cost callback error
2025-04-19 11:23:37 -07:00
Krish Dholakia
f08a4e3c06
Support 'file' message type for VLLM video url's + Anthropic redacted message thinking support (#10129)
* feat(hosted_vllm/chat/transformation.py): support calling vllm video url with openai 'file' message type

allows switching between gemini/vllm easily

* [WIP] redacted thinking tests (#9044)

* WIP: redacted thinking tests

* test: add test for redacted thinking in assistant message

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* fix(anthropic/chat/transformation.py): support redacted thinking block on anthropic completion

Fixes https://github.com/BerriAI/litellm/issues/9058

* fix(anthropic/chat/handler.py): transform anthropic redacted messages on streaming

Fixes https://github.com/BerriAI/litellm/issues/9058

* fix(bedrock/): support redacted text on streaming + non-streaming

Fixes https://github.com/BerriAI/litellm/issues/9058

* feat(litellm_proxy/chat/transformation.py): support 'reasoning_effort' param for proxy

allows using reasoning effort with thinking models on proxy

* test: update tests

* fix(utils.py): fix linting error

* fix: fix linting errors

* fix: fix linting errors

* fix: fix linting error

* fix: fix linting errors

* fix(anthropic/chat/transformation.py): fix returning citations in chat completion

---------

Co-authored-by: Johann Miller <22018973+johannkm@users.noreply.github.com>
2025-04-19 11:16:37 -07:00
Ishaan Jaff
3c463f6715 test fix - output_cost_per_reasoning_token was added to model cost map 2025-04-19 10:02:25 -07:00
Krish Dholakia
2508ca71cb
Handle fireworks ai tool calling response (#10130)
* feat(fireworks_ai/chat): handle tool calling with fireworks ai correctly

Fixes https://github.com/BerriAI/litellm/issues/7209

* fix(utils.py): handle none type in message

* fix: fix model name in test

* fix(utils.py): fix validate check for openai messages

* fix: fix model returned

* fix(main.py): fix text completion routing

* test: update testing

* test: skip test - cohere having RBAC issues
2025-04-19 09:37:45 -07:00
Krrish Dholakia
b4f2b3dad1 test: update test to be more robust to usage updates 2025-04-19 09:26:26 -07:00
Ishaan Jaff
8ae2653280 fix calculated cache key for tests 2025-04-19 09:25:11 -07:00
Krish Dholakia
36308a31be
Gemini-2.5-flash - support reasoning cost calc + return reasoning content (#10141)
* build(model_prices_and_context_window.json): add vertex ai gemini-2.5-flash pricing

* build(model_prices_and_context_window.json): add gemini reasoning token pricing

* fix(vertex_and_google_ai_studio_gemini.py): support counting thinking tokens for gemini

allows accurate cost calc

* fix(utils.py): add reasoning token cost calc to generic cost calc

ensures gemini-2.5-flash cost calculation is accurate

* build(model_prices_and_context_window.json): mark gemini-2.5-flash as 'supports_reasoning'

* feat(gemini/): support 'thinking' + 'reasoning_effort' params + new unit tests

allow controlling thinking effort for gemini-2.5-flash models

* test: update unit testing

* feat(vertex_and_google_ai_studio_gemini.py): return reasoning content if given in gemini response

* test: update model name

* fix: fix ruff check

* test(test_spend_management_endpoints.py): update tests to be less sensitive to new keys / updates to usage object

* fix(vertex_and_google_ai_studio_gemini.py): fix translation
2025-04-19 09:20:52 -07:00
Krrish Dholakia
d726e0f34c test: update testing imports 2025-04-19 09:13:16 -07:00
Ishaan Jaff
0a35c208d7 test assistants fixes 2025-04-19 08:09:45 -07:00
Ishaan Jaff
a62805f98f fixes for assistans API tests 2025-04-19 07:59:53 -07:00
Ishaan Jaff
5bf76f0bb1 test fixes for azure assistants 2025-04-19 07:36:40 -07:00
Ishaan Jaff
b9756bf006 test_completion_azure 2025-04-19 07:24:11 -07:00
Krrish Dholakia
652e1b7f0f test: update test 2025-04-18 20:36:15 -07:00
Ishaan Jaff
3d5022bd79
[Feat] Support for all litellm providers on Responses API (works with Codex) - Anthropic, Bedrock API, VertexAI, Ollama (#10132)
* transform request

* basic handler for LiteLLMCompletionTransformationHandler

* complete transform litellm to responses api

* fixes to test

* fix stream=True

* fix streaming iterator

* fixes for transformation

* fixes for anthropic codex support

* fix pass response_api_optional_params

* test anthropic responses api tools

* update responses types

* working codex with litellm

* add session handler

* fixes streaming iterator

* fix handler

* add litellm codex example

* fix code quality

* test fix

* docs litellm codex

* litellm codexdoc

* docs openai codex with litellm

* docs litellm openai codex

* litellm codex

* linting fixes for transforming responses API

* fix import error

* fix responses api test

* add sync iterator support for responses api
2025-04-18 19:53:59 -07:00
Krrish Dholakia
3e87ec4f16 test: replace removed fireworks ai models
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 19s
Helm unit test / unit-test (push) Successful in 24s
2025-04-18 14:23:16 -07:00
Krish Dholakia
1ea046cc61
test: update tests to new deployment model (#10142)
* test: update tests to new deployment model

* test: update model name

* test: skip cohere rbac issue test

* test: update test - replace gpt-4o model
2025-04-18 14:22:12 -07:00
Krrish Dholakia
415abfc222 test: update test 2025-04-18 13:13:58 -07:00
Krrish Dholakia
f7dd688035 test: handle cohere rbac issue (verified happens on calling azure directly)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 15s
Helm unit test / unit-test (push) Successful in 23s
2025-04-18 08:42:12 -07:00
Ishaan Jaff
d3e04eac7f
[Feat] Unified Responses API - Add Azure Responses API support (#10116)
* initial commit for azure responses api support

* update get complete url

* fixes for responses API

* working azure responses API

* working responses API

* test suite for responses API

* azure responses API test suite

* fix test with complete url

* fix test refactor

* test fix metadata checks

* fix code quality check
2025-04-17 16:47:59 -07:00
Ishaan Jaff
257e78ffb5 test fix vertex_ai/mistral-large@2407 2025-04-16 21:52:52 -07:00
Krish Dholakia
c73a6a8d1e
Add new /vertex_ai/discovery route - enables calling AgentBuilder API routes (#10084)
* feat(llm_passthrough_endpoints.py): expose new `/vertex_ai/discovery/` endpoint

Allows calling vertex ai discovery endpoints via passthrough

 For agentbuilder api calls

* refactor(llm_passthrough_endpoints.py): use common _base_vertex_proxy_route

Prevents duplicate code

* feat(llm_passthrough_endpoints.py): add vertex endpoint specific passthrough handlers
2025-04-16 21:45:51 -07:00
Ishaan Jaff
198922b26f test fixes for vertex mistral, this model was deprecated on vertex 2025-04-16 20:51:45 -07:00