* Adding VertexAI Claude 3.7 Sonnet (#8774)
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
* build(model_prices_and_context_window.json): add anthropic 3-7 models on vertex ai and bedrock
* Support video_url (#8743)
* Support video_url
Support VLMs that works with video.
Example implemenation in vllm: https://github.com/vllm-project/vllm/pull/10020
* llms openai.py: Add ChatCompletionVideoObject
Add data structures to support `video_url` in chat completion
* test test_completion.py: add test for video_url
* Arize Phoenix - ensure correct endpoint/protocol are used; and default to phoenix cloud (#8750)
* minor fixes to default to http and to ensure that the correct endpoint is used
* Update test_arize_phoenix.py
* prioritize http over grpc
---------
Co-authored-by: Emerson Gomes <emerson.gomes@gmail.com>
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: Pang Wu <104795337+pang-wu@users.noreply.github.com>
Co-authored-by: Nate Mar <67926244+nate-mar@users.noreply.github.com>
* refactor(teams.tsx): refactor to display all teams, across all orgs
removes org switcher from navbar, simplifies viewing/creating teams on UI
* fix(key_list.tsx): show user keys across all orgs
make it easy to see flat list of keys across orgs on key table
* style(all_keys_table.tsx): cleanup keys table
* fix(user_dashboard.tsx): remove overflow-hidden in dashboard component
* fix(teams.tsx): move org id placement in create team flow
* fix(teams.tsx): support model selection on create team based on selected org
* feat(view_key_table.tsx): move to using a filter component on keys page
allows filtering keys by org and team
* fix(filter.tsx): handle reset filter
* fix: fix linting error
* (Feat) - return `x-litellm-attempted-fallbacks` in responses from litellm proxy (#8558)
* add_fallback_headers_to_response
* test x-litellm-attempted-fallbacks
* unit test attempted fallbacks
* fix add_fallback_headers_to_response
* docs document response headers
* fix file name
* test fix use mock endpoints for e2e files and ft tests
* Revert "test fix use mock endpoints for e2e files and ft tests"
This reverts commit c921d8dd81.
* cleanup_azure_files
* Add remaining org CRUD endpoints + support deleting orgs on UI (#8561)
* feat(organization_endpoints.py): expose new `/organization/delete` endpoint. Cascade org deletion to member, teams and keys
Ensures any org deletion is handled correctly
* test(test_organizations.py): add simple test to ensure org deletion works
* feat(organization_endpoints.py): expose /organization/update endpoint, and define response models for org delete + update
* fix(organizations.tsx): support org delete on UI + move org/delete endpoint to use DELETE
* feat(organization_endpoints.py): support `/organization/member_update` endpoint
Allow admin to update member's role within org
* feat(organization_endpoints.py): support deleting member from org
* test(test_organizations.py): add e2e test to ensure org member flow works
* fix(organization_endpoints.py): fix code qa check
* fix(schema.prisma): don't introduce ondelete:cascade - breaking change
* docs(organization_endpoints.py): document missing params
* refactor(organization_view.tsx): initial commit creating a generic update member component shared between org and team member classes
* feat(organization_view.tsx): support updating org member role on UI
* feat(organization_view.tsx): allow proxy admin to delete members from org
* Enable update/delete org members on UI (#8560)
* feat(organization_endpoints.py): expose new `/organization/delete` endpoint. Cascade org deletion to member, teams and keys
Ensures any org deletion is handled correctly
* test(test_organizations.py): add simple test to ensure org deletion works
* feat(organization_endpoints.py): expose /organization/update endpoint, and define response models for org delete + update
* fix(organizations.tsx): support org delete on UI + move org/delete endpoint to use DELETE
* feat(organization_endpoints.py): support `/organization/member_update` endpoint
Allow admin to update member's role within org
* feat(organization_endpoints.py): support deleting member from org
* test(test_organizations.py): add e2e test to ensure org member flow works
* fix(organization_endpoints.py): fix code qa check
* fix(schema.prisma): don't introduce ondelete:cascade - breaking change
* docs(organization_endpoints.py): document missing params
* (Bug Fix) - Add Regenerate Key on Virtual Keys Tab (#8567)
* add regenerate key to ui
* ui fix key info
* (Bug Fix + Better Observability) - BudgetResetJob: (#8562)
* use class ResetBudgetJob
* refactor reset budget job
* update reset_budget job
* refactor reset budget job
* fix LiteLLM_UserTable
* refactor reset budget job
* add telemetry for reset budget job
* dd - log service success/failure on DD
* add detailed reset budget reset info on DD
* initialize_scheduled_background_jobs
* refactor reset budget job
* trigger service failure hook when fails to reset a budget for team, key, user
* fix resetBudgetJob
* unit testing for ResetBudgetJob
* test_duration_in_seconds_basic
* testing for triggering service logging
* fix logs on test teams fail
* remove unused imports
* fix import duration in s
* duration_in_seconds
* (Patch/bug fix) - UI, filter out litellm ui session tokens on Virtual Keys Page (#8568)
* fix key list endpoint
* _get_condition_to_filter_out_ui_session_tokens
* duration_in_seconds
* test_list_key_helper_team_filtering
* bump: version 1.61.4 → 1.61.5
* ui fix tsx linting
* ui new build
* test_list_key_helper_team_filtering
* ui new build
* test_openai_fine_tuning
* test_openai_fine_tuning
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* use class ResetBudgetJob
* refactor reset budget job
* update reset_budget job
* refactor reset budget job
* fix LiteLLM_UserTable
* refactor reset budget job
* add telemetry for reset budget job
* dd - log service success/failure on DD
* add detailed reset budget reset info on DD
* initialize_scheduled_background_jobs
* refactor reset budget job
* trigger service failure hook when fails to reset a budget for team, key, user
* fix resetBudgetJob
* unit testing for ResetBudgetJob
* test_duration_in_seconds_basic
* testing for triggering service logging
* fix logs on test teams fail
* remove unused imports
* fix import duration in s
* duration_in_seconds
* fix(model_checks.py): update returning known model from wildcard to filter based on given model prefix
ensures wildcard route - `vertex_ai/gemini-*` just returns known vertex_ai/gemini- models
* test(test_proxy_utils.py): add unit testing for new 'get_known_models_from_wildcard' helper
* test(test_models.py): add e2e testing for `/model_group/info` endpoint
* feat(prometheus.py): support tracking total requests by user_email on prometheus
adds initial support for tracking total requests by user_email
* test(test_prometheus.py): add testing to ensure user email is always tracked
* test: update testing for new prometheus metric
* test(test_prometheus_unit_tests.py): add user email to total proxy metric
* test: update tests
* test: fix spend tests
* test: fix test
* fix(pagerduty.py): fix linting error
* refactor _get_langfuse_input_output_content
* test_langfuse_logging_completion_with_malformed_llm_response
* fix _get_langfuse_input_output_content
* fixes for langfuse linting
* unit testing for get chat/text content for langfuse
* fix _should_raise_content_policy_error
* fix(prometheus.py): fix setting key budget metrics
ensures custom metadata works with key budget metric
this is a patch. root cause pr is written in a separate branch
* test: fix test
* test(base_llm_unit_tests.py): add test to ensure drop params is respected
* fix(types/prometheus.py): use typing_extensions for python3.8 compatibility
* build: add cherry picked commits
* fix(base_utils.py): supported nested json schema passed in for anthropic calls
* refactor(base_utils.py): refactor ref parsing to prevent infinite loop
* test(test_openai_endpoints.py): refactor anthropic test to use bedrock
* fix(langfuse_prompt_management.py): add unit test for sync langfuse calls
Resolves https://github.com/BerriAI/litellm/issues/7938#issuecomment-2613293757
* fix(bedrock/converse_handler.py): fix bedrock region name on async calls
* fix(utils.py): fix split model handling
Fixes bedrock cost calculation when region name is given
* feat(_health_endpoints.py): support health checking datadog integration
Closes https://github.com/BerriAI/litellm/issues/7921
* Fix: Problem with langfuse_tags when using litellm proxy with langfuse integration (#7801)
* Refactor: Create method get_langfuse_tags
* Fix: Exception Handling
* fix base aws llm
* fix auth with aws role
* test aws base llm
* fix base aws llm init
* run ci/cd again
* fix get_credentials
* ci/cd run again
* _auth_with_aws_role
* feat(langfuse.py): log the used prompt when prompt management used
* test: fix test
* docs(self_serve.md): add doc on restricting personal key creation on ui
* feat(s3.py): support s3 logging with team alias prefixes (if available)
New preview feature
* fix(main.py): remove old if block - simplify to just await if coroutine returned
fixes lm_studio async embedding error
* fix(langfuse.py): handle get prompt check
* fix(vertex_ai/gemini/transformation.py): handle 'http://' in gemini process url
* refactor(router.py): refactor '_prompt_management_factory' to use logging obj get_chat_completion logic
deduplicates code
* fix(litellm_logging.py): update 'get_chat_completion_prompt' to update logging object messages
* docs(prompt_management.md): update prompt management to be in beta
given feedback - this still needs to be revised (e.g. passing in user message, not ignoring)
* refactor(prompt_management_base.py): introduce base class for prompt management
allows consistent behaviour across prompt management integrations
* feat(prompt_management_base.py): support adding client message to template message + refactor langfuse prompt management to use prompt management base
* fix(litellm_logging.py): log prompt id + prompt variables to langfuse if set
allows tracking what prompt was used for what purpose
* feat(litellm_logging.py): log prompt management metadata in standard logging payload + use in langfuse
allows logging prompt id / prompt variables to langfuse
* test: fix test
* fix(router.py): cleanup unused imports
* fix: fix linting error
* fix: fix trace param typing
* fix: fix linting errors
* fix: fix code qa check
