* build(pyproject.toml): add new dev dependencies - for type checking
* build: reformat files to fit black
* ci: reformat to fit black
* ci(test-litellm.yml): make tests run clear
* build(pyproject.toml): add ruff
* fix: fix ruff checks
* build(mypy/): fix mypy linting errors
* fix(hashicorp_secret_manager.py): fix passing cert for tls auth
* build(mypy/): resolve all mypy errors
* test: update test
* fix: fix black formatting
* build(pre-commit-config.yaml): use poetry run black
* fix(proxy_server.py): fix linting error
* fix: fix ruff safe representation error
* refactor get model info for team models
* allow adding a model to a team when creating team specific model
* ui update selected Team on Team Dropdown
* test_team_model_association
* testing for team specific models
* test_get_team_specific_model
* test: skip on internal server error
* remove model alias card on teams page
* linting fix _get_team_specific_model
* fix DeploymentTypedDict
* fix linting error
* fix code quality
* fix model info checks
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
* feat(litellm_pre_call_utils.py): support `x-litellm-tags` request header
allow tag based routing + spend tracking via request headers
* docs(request_headers.md): document new `x-litellm-tags` for tag based routing and spend tracking
* docs(tag_routing.md): add to docs
* fix(utils.py): only pass str values for openai metadata param
* fix(utils.py): drop non-str values for metadata param to openai
preview-feature, otel span was being sent in
* ui - use common team dropdown component
* re-use team component
* rename org field on add model
* handle add model submit
* working view model_id and team_id on root models page
* cleaner
* show all fields
* working model info view
* working team info selector
* clean up team id
* new component for model dashboard
* ui show table with dropdown
* make public model names like email
* revert changes to litellm model name
* fix litellm model name
* ui fix public model
* fix mappings
* fix conditional text input
* fix message
* ui fix bulk add models
* _add_team_model_to_db
* move model mgmt helper funcs
* test_add_team_model_to_db
* ui - display model team model name
* fix add model tab
* fix remove redundant info tab on models page
* dont pass model mappings all the way through
* fix jarring model name when adding team models
* fix edit model button
* delete button on model info
* ui fix model dashboard
* fix DeploymentTypedDict
* _is_model_access_group_for_wildcard_route
* test _get_public_model_name
* ui fix viewing public model name
* fix linting error
* fix linting errors
* fix selectedModel logic
* feat(lowest_tpm_rpm_v2.py): fix redis cache check to use >= instead of >
makes it consistent
* test(test_custom_guardrails.py): add more unit testing on default on guardrails
ensure it runs if user sent guardrail list is empty
* docs(quick_start.md): clarify default on guardrails run even if user guardrails list contains other guardrails
* refactor(litellm_logging.py): refactor no-log to helper util
allows for more consistent behavior
* feat(litellm_logging.py): add event hook to verbose logs
* fix(litellm_logging.py): add unit testing to ensure `litellm.disable_no_log_param` is respected
* docs(logging.md): document how to disable 'no-log' param
* test: fix test to handle feb
* test: cleanup old bedrock model
* fix: fix router check
* fix(model_dashboard.tsx): support setting model_info params - e.g. mode on ui
Closes https://github.com/BerriAI/litellm/issues/5270
* fix(lowest_tpm_rpm_v2.py): deployment rpm over limit check
fixes selection error when getting potential deployments below known tpm/rpm limit
Fixes https://github.com/BerriAI/litellm/issues/7395
* fix(test_tpm_rpm_routing_v2.py): add unit test for https://github.com/BerriAI/litellm/issues/7395
* fix(lowest_tpm_rpm_v2.py): fix tpm key name in dict post rpm update
* test: rename test to run earlier
* test: skip flaky test
* feat(proxy/utils.py): get associated litellm budget from db in combined_view for key
allows user to create rate limit tiers and associate those to keys
* feat(proxy/_types.py): update the value of key-level tpm/rpm/model max budget metrics with the associated budget table values if set
allows rate limit tiers to be easily applied to keys
* docs(rate_limit_tiers.md): add doc on setting rate limit / budget tiers
make feature discoverable
* feat(key_management_endpoints.py): return litellm_budget_table value in key generate
make it easy for user to know associated budget on key creation
* fix(key_management_endpoints.py): document 'budget_id' param in `/key/generate`
* docs(key_management_endpoints.py): document budget_id usage
* refactor(budget_management_endpoints.py): refactor budget endpoints into separate file - makes it easier to run documentation testing against it
* docs(test_api_docs.py): add budget endpoints to ci/cd doc test + add missing param info to docs
* fix(customer_endpoints.py): use new pydantic obj name
* docs(user_management_heirarchy.md): add simple doc explaining teams/keys/org/users on litellm
* Litellm dev 12 26 2024 p2 (#7432)
* (Feat) Add logging for `POST v1/fine_tuning/jobs` (#7426)
* init commit ft jobs logging
* add ft logging
* add logging for FineTuningJob
* simple FT Job create test
* (docs) - show all supported Azure OpenAI endpoints in overview (#7428)
* azure batches
* update doc
* docs azure endpoints
* docs endpoints on azure
* docs azure batches api
* docs azure batches api
* fix(key_management_endpoints.py): fix key update to actually work
* test(test_key_management.py): add e2e test asserting ui key update call works
* fix: proxy/_types - fix linting erros
* test: update test
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* fix: test
* fix(parallel_request_limiter.py): enforce tpm/rpm limits on key from tiers
* fix: fix linting errors
* test: fix test
* fix: remove unused import
* test: update test
* docs(customer_endpoints.py): document new model_max_budget param
* test: specify unique key alias
* docs(budget_management_endpoints.py): document new model_max_budget param
* test: fix test
* test: fix tests
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* feat(guardrails_endpoint.py): new `/guardrails/list` endpoint
Allow users to view what the available guardrails are
* docs: document new `/guardrails/list` endpoint
* docs(enterprise.md): update docs
* fix(openai/transcription/handler.py): support cost tracking on vtt + srt formats
* fix(openai/transcriptions/handler.py): default to 'verbose_json' response format if 'text' or 'json' response_format received. ensures 'duration' param is received for all audio transcription requests
* fix: fix linting errors
* fix: remove unused import
* fix test_deployment_budget_limits_e2e_test
* refactor async_log_success_event to track spend for provider + deployment
* fix format
* rename class to RouterBudgetLimiting
* rename func
* rename types used for budgets
* add new types for deployment budgets
* add budget limits for deployments
* fix checking budgets set for provider
* update file names
* fix linting error
* _track_provider_remaining_budget_prometheus
* async_filter_deployments
* fix model list passed to router
* update error
* test_deployment_budgets_e2e_test_expect_to_fail
* fix test case
* run deployment budget limits
* use 1 file for duration_in_seconds
* add to readme.md
* re use duration_in_seconds
* fix importing _extract_from_regex, get_last_day_of_month
* fix import
* update provider budget routing
* fix - remove dup test
* add support for using in multi instance environments
* test_in_memory_redis_sync_e2e
* test_in_memory_redis_sync_e2e
* fix test_in_memory_redis_sync_e2e
* fix code quality check
* fix test provider budgets
* working provider budget tests
* add fixture for provider budget routing
* fix router testing for provider budgets
* add comments on provider budget routing
* use RedisPipelineIncrementOperation
* add redis async_increment_pipeline
* use redis async_increment_pipeline
* use lower value for testing
* use redis async_increment_pipeline
* use consistent key name for increment op
* add handling for budget windows
* fix typing async_increment_pipeline
* fix set attr
* add clear doc strings
* unit testing for provider budgets
* test_redis_increment_pipeline
* fix router strat
* use async set / get cache in router_strategy
* add coverage for router strategy
* fix imports
* fix batch_get_cache
* use async methods for least busy
* fix least busy use async methods
* fix test_dual_cache_increment
* test async_get_available_deployment when routing_strategy="least-busy"
* docs(exception_mapping.md): add missing exception types
Fixes https://github.com/Aider-AI/aider/issues/2120#issuecomment-2438971183
* fix(main.py): register custom model pricing with specific key
Ensure custom model pricing is registered to the specific model+provider key combination
* test: make testing more robust for custom pricing
* fix(redis_cache.py): instrument otel logging for sync redis calls
ensures complete coverage for all redis cache calls
* refactor: pass parent_otel_span for redis caching calls in router
allows for more observability into what calls are causing latency issues
* test: update tests with new params
* refactor: ensure e2e otel tracing for router
* refactor(router.py): add more otel tracing acrosss router
catch all latency issues for router requests
* fix: fix linting error
* fix(router.py): fix linting error
* fix: fix test
* test: fix tests
* fix(dual_cache.py): pass ttl to redis cache
* fix: fix param
* add flake 8 check
* split up litellm _acompletion
* fix get model client
* refactor use commong func to add metadata to kwargs
* use common func to get timeout
* re-use helper to _get_async_model_client
* use _handle_mock_testing_rate_limit_error
* fix docstring for _handle_mock_testing_rate_limit_error
* fix function_with_retries
* use helper for mock testing fallbacks
* router - use 1 func for simple_shuffle
* add doc string for simple_shuffle
* use 1 function for filtering cooldown deployments
* fix use common helper to _get_fallback_model_group_from_fallbacks
