* fix(anthropic_claude3_transformation.py): fix amazon anthropic claude 3 tool calling transformation on invoke route
move to using anthropic config as base
* fix(utils.py): expose anthropic config via providerconfigmanager
* fix(llm_http_handler.py): support json mode on async completion calls
* fix(invoke_handler/make_call): support json mode for anthropic called via bedrock invoke
* fix(anthropic/): handle 'response_format: {"type": "text"}` + migrate amazon claude 3 invoke config to inherit from anthropic config
Prevents error when passing in 'response_format: {"type": "text"}
* test: fix test
* fix(utils.py): fix base invoke provider check
* fix(anthropic_claude3_transformation.py): don't pass 'stream' param
* fix: fix linting errors
* fix(converse_transformation.py): handle response_format type=text for converse
* fix(http_parsing_utils.py): orjson can throw errors on some emoji's in text, default to json.loads
* fix(sagemaker/handler.py): support passing model id on async streaming
* fix(litellm_pre_call_utils.py): Fixes https://github.com/BerriAI/litellm/issues/7237
* refactor(internal_user_endpoints.py): refactor `/user/list` to accept 'user_ids' and use prisma for db calls
enables bulk search from UI
* fix(internal_user_endpoints.py): fix linting errors
* fix(all_keys_table.tsx): show user email on create key table
make it easier for admin to know which key is associated to which user
* docs(internal_user_endpoints.py): improve docstring
* fix: sync schema with main
* fix(columns.tsx): display SSO ID on Internal User Table
make it easy to identify what the SSO ID for a user is
* fix(columns.tsx): add tooltip to header
help user understand what SSO ID means
* style: add more tooltips in the management flows
make it easier to understand what you're seeing
* style(all_keys_table.tsx): replace 'Not Set' with '-'
reduces words on table
* fix(internal_user_endpoints.py): fix user ids check
* test: fix test
* fix(internal_user_endpoints.py): maintain returning key count in `/user/list`
* fix(user_api_key_auth.py): Fixes https://github.com/BerriAI/litellm/issues/8780
security fix - enforce model access checks on azure routes
* test(test_user_api_key_auth.py): add unit testing
* test(test_openai_endpoints.py): add e2e test to ensure azure routes also run through model validation checks
* fix(proxy/_types.py): return created_by and updated_by on /key/list
enables better trail of who made a key
* fix(all_keys_table.tsx): add created by to key table
allows easier tracking of who generated the key
* fix(key_management_endpoints.py): track 'created_by' and 'updated_by' fields
enable easier tracking of who created proxy keys
* fix(filtercomponent): always show apply filters button
fix hiding behavior
* style(create_key_button.tsx): style improvements on create key modal
remove the numbering
* feat(filter.tsx): allow searching team by team alias
* style(filter.tsx): style improvements to ensure dropdown + custom value works as expected
* style(filter.tsx): add explicit button allowing reset filters
* fix(filter.tsx): fix linting error
* feat(all_keys_table.tsx): show team alias on keys table
* style(all_keys_table.tsx): enforce length constraints on table
make it easier to see all columns
* fix track custom llm provider on pass through routes
* fix use correct provider for google ai studio
* fix tracking custom llm provider on pass through route
* ui fix get provider logo
* update tests to track custom llm provider
* test_anthropic_streaming_with_headers
* Potential fix for code scanning alert no. 2263: Incomplete URL substring sanitization
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* feat(bedrock/converse/transformation.py): support claude-3-7-sonnet reasoning_Content transformation
Closes https://github.com/BerriAI/litellm/issues/8777
* fix(bedrock/): support returning `reasoning_content` on streaming for claude-3-7
Resolves https://github.com/BerriAI/litellm/issues/8777
* feat(bedrock/): unify converse reasoning content blocks for consistency across anthropic and bedrock
* fix(anthropic/chat/transformation.py): handle deepseek-style 'reasoning_content' extraction within transformation.py
simpler logic
* feat(bedrock/): fix streaming to return blocks in consistent format
* fix: fix linting error
* test: fix test
* feat(factory.py): fix bedrock thinking block translation on tool calling
allows passing the thinking blocks back to bedrock for tool calling
* fix(types/utils.py): don't exclude provider_specific_fields on model dump
ensures consistent responses
* fix: fix linting errors
* fix(convert_dict_to_response.py): pass reasoning_content on root
* fix: test
* fix(streaming_handler.py): add helper util for setting model id
* fix(streaming_handler.py): fix setting model id on model response stream chunk
* fix(streaming_handler.py): fix linting error
* fix(streaming_handler.py): fix linting error
* fix(types/utils.py): add provider_specific_fields to model stream response
* fix(streaming_handler.py): copy provider specific fields and add them to the root of the streaming response
* fix(streaming_handler.py): fix check
* fix: fix test
* fix(types/utils.py): ensure messages content is always openai compatible
* fix(types/utils.py): fix delta object to always be openai compatible
only introduce new params if variable exists
* test: fix bedrock nova tests
* test: skip flaky test
* test: skip flaky test in ci/cd
* test_openai_assistants_e2e_operations
* test openai assistants pass through
* fix GET request on pass through handler
* _make_non_streaming_http_request
* _is_assistants_api_request
* test_openai_assistants_e2e_operations
* test_openai_assistants_e2e_operations
* openai_proxy_route
* docs openai pass through
* docs openai pass through
* docs openai pass through
* test pass through handler
* Potential fix for code scanning alert no. 2240: Incomplete URL substring sanitization
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* fix running litellm on windows
* fix importing litellm
* _init_hypercorn_server
* linting fix
* TestProxyInitializationHelpers
* ci/cd run again
* ci/cd run again
* allow bulk adding internal users
* allow sorting users by created at
* cleanup
* clean up user table
* show total num users
* show per user error when bulk adding users
* fix - don't allow creating duplicate internal users in DB
* ui flow fix for bulk adding users
* allow adding user in multiple teams with models
* correctly extract info
* working invitation link
* fix fill in table after bulk add
* fix the results from creating new users in bulkd
* bulk invite users
* fix view user flow
* fix ui type errors
* fix type errors
* fix type errors
* fix(proxy/_types.py): fixes issue where internal user able to escalate their role with ui key
Fixes https://github.com/BerriAI/litellm/issues/8029
* style: cleanup
* test: handle bedrock instability
* fix(amazon_deepseek_transformation.py): remove </think> from stream output - cleanup user facing stream
* fix(key_managenet_endpoints.py): return `/key/list` sorted by created_at
makes it easier to see created key
* style: cleanup team table
* feat(key_edit_view.tsx): support setting model specific tpm/rpm limits on keys
* feat(create_key_button.tsx): initial commit using openapi.json to ensure all values via api are supported on ui for `/key/generate`
Closes https://github.com/BerriAI/litellm/issues/7763
* style(create_key_button.tsx): put openapi settings inside 'advanced setting' accordion
* fix(check_openapi_schema.tsx): style improvements for advanced settings
* style(create_key_button.tsx): add tooltip explaining what the settings mean
* fix(team_info.tsx): render metadata field on team update
allow updating a team's metadata
* fix(networking.tsx): add 'metadata' field to create team form
* refactor: cleanup dead codeblock
* fix(organization_endpoints.py): fix metadata param support on `/organization/new`
* feat(organization_endpoints.py): support updating metadata for organization on api + ui
* test: mark flaky test
* fix(team_endpoints.py): allow team member to view team info
* test: handle model overloaded in tool calling test
* test: handle internal server error
* refactor get model info for team models
* allow adding a model to a team when creating team specific model
* ui update selected Team on Team Dropdown
* test_team_model_association
* testing for team specific models
* test_get_team_specific_model
* test: skip on internal server error
* remove model alias card on teams page
* linting fix _get_team_specific_model
* fix DeploymentTypedDict
* fix linting error
* fix code quality
* fix model info checks
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
* fix(team_endpoints.py): cleanup user <-> team association on team delete
Fixes issue where user table still listed team membership post delete
* test(test_team.py): update e2e test - ensure user/team membership is deleted on team delete
* fix(base_invoke_transformation.py): fix deepseek r1 transformation
remove deepseek name from model url
* test(test_completion.py): assert model route not in url
* feat(base_invoke_transformation.py): infer region name from model arn
prevent errors due to different region name in env var vs. model arn, respect if explicitly set in call though
* test: fix test
* test: skip on internal server error
