* build(pyproject.toml): add new dev dependencies - for type checking
* build: reformat files to fit black
* ci: reformat to fit black
* ci(test-litellm.yml): make tests run clear
* build(pyproject.toml): add ruff
* fix: fix ruff checks
* build(mypy/): fix mypy linting errors
* fix(hashicorp_secret_manager.py): fix passing cert for tls auth
* build(mypy/): resolve all mypy errors
* test: update test
* fix: fix black formatting
* build(pre-commit-config.yaml): use poetry run black
* fix(proxy_server.py): fix linting error
* fix: fix ruff safe representation error
* fix: remove aws params from bedrock embedding request body (#8618)
* fix: remove aws params from bedrock embedding request body
* fix-7548: handle aws params in base class
* test: load request data from mock call
* (Infra/DB) - Allow running older litellm version when out of sync with current state of DB (#8695)
* fix check migration
* clean up should_update_prisma_schema
* update test
* db_migration_disable_update_check
* Check container logs for expected message
* db_migration_disable_update_check
* test_check_migration_out_of_sync
* test_should_update_prisma_schema
* db_migration_disable_update_check
* pip install aiohttp
* ui new build
* delete deprecated code test
* bump: version 1.61.12 → 1.61.13
* Add cost tracking for rerank via bedrock (#8691)
* feat(bedrock/rerank): infer model region if model given as arn
* test: add unit testing to ensure bedrock region name inferred from arn on rerank
* feat(bedrock/rerank/transformation.py): include search units for bedrock rerank result
Resolves https://github.com/BerriAI/litellm/issues/7258#issuecomment-2671557137
* test(test_bedrock_completion.py): add testing for bedrock cohere rerank
* feat(cost_calculator.py): refactor rerank cost tracking to support bedrock cost tracking
* build(model_prices_and_context_window.json): add amazon.rerank model to model cost map
* fix(cost_calculator.py): bedrock/common_utils.py
get base model from model w/ arn -> handles rerank model
* build(model_prices_and_context_window.json): add bedrock cohere rerank pricing
* feat(bedrock/rerank): migrate bedrock config to basererank config
* Revert "feat(bedrock/rerank): migrate bedrock config to basererank config"
This reverts commit 84fae1f167.
* test: add testing to ensure large doc / queries are correctly counted
* Revert "test: add testing to ensure large doc / queries are correctly counted"
This reverts commit 4337f1657e.
* fix(migrate-jina-ai-to-rerank-config): enables cost tracking
* refactor(jina_ai/): finish migrating jina ai to base rerank config
enables cost tracking
* fix(jina_ai/rerank): e2e jina ai rerank cost tracking
* fix: cleanup dead code
* fix: fix python3.8 compatibility error
* test: fix test
* test: add e2e testing for azure ai rerank
* fix: fix linting error
* test: mark cohere as flaky
* add bedrock llama vision support + cohere / infinity rerank - 'return_documents' support (#8684)
* build(model_prices_and_context_window.json): mark bedrock llama as supporting vision based on docs
* Add price for Cerebras llama3.3-70b (#8676)
* docs(readme.md): fix contributing docs
point people to new mock directory testing structure s/o @vibhavbhat
* build: update contributing readme
* docs(readme.md): improve docs
* docs(readme.md): cleanup readme on tests/
* docs(README.md): cleanup doc
* feat(infinity/): support returning documents when return_documents=True
* test(test_rerank.py): add e2e testing for cohere rerank
* fix: fix linting errors
* fix(together_ai/): fix together ai transformation
* fix: fix linting error
* fix: fix linting errors
* fix: fix linting errors
* test: mark cohere as flaky
* build: fix model supports check
* test: fix test
* test: mark flaky test
* fix: fix test
* test: fix test
---------
Co-authored-by: Yury Koleda <fut.wrk@gmail.com>
* test: fix test
* fix: remove unused import
* bump: version 1.61.13 → 1.61.14
* Correct spelling in user_management_heirarchy.md (#8716)
Fixing irritating typo -- page and image names would also need to be updated
* (Feat) - UI, Allow sorting models by Created_At and all other columns on the UI (#8725)
* order models by created at
* use existing table component on models page
* sorting for created at
* ui clean up models page
* remove provider filter
* fix columns sorting
* decent switching
* ui fix models page
* (UI) Edit Model flow improvements (#8729)
* order models by created at
* use existing table component on models page
* sorting for created at
* ui clean up models page
* remove provider filter
* fix columns sorting
* decent switching
* ui fix models page
* show edit / delete button on root of table
* clean up columns
* working edit model flow
* decent working model edit page
* fix edit model
* show created at and created by
* ui easy model edit flow
* clean up columns
* ui clean up updated at
* fix model datatable
* ui new build
* bump: version 1.61.14 → 1.61.15
* Support arize phoenix on litellm proxy (#7756) (#8715)
* Update opentelemetry.py
wip
* Update test_opentelemetry_unit_tests.py
* fix a few paths and tests
* fix path
* Update litellm_logging.py
* accidentally removed code
* Add type for protocol
* Add and update tests
* minor changes
* update and add additional arize phoenix test
* update existing test
* address feedback
* use standard_logging_object
* address feedback
Co-authored-by: Nate Mar <67926244+nate-mar@users.noreply.github.com>
* fix(amazon_deepseek_transformation.py): remove </think> from stream o… (#8717)
* fix(amazon_deepseek_transformation.py): remove </think> from stream output - cleanup user facing stream
* fix(key_managenet_endpoints.py): return `/key/list` sorted by created_at
makes it easier to see created key
* style: cleanup team table
* feat(key_edit_view.tsx): support setting model specific tpm/rpm limits on keys
* Add cohere v2/rerank support (#8421) (#8605)
* Add cohere v2/rerank support (#8421)
* Support v2 endpoint cohere rerank
* Add tests and docs
* Make v1 default if old params used
* Update docs
* Update docs pt 2
* Update tests
* Add e2e test
* Clean up code
* Use inheritence for new config
* Fix linting issues (#8608)
* Fix cohere v2 failing test + linting (#8672)
* Fix test and unused imports
* Fix tests
* fix: fix linting errors
* test: handle tgai instability
* fix: skip service unavailable err
* test: print logs for unstable test
* test: skip unreliable tests
---------
Co-authored-by: vibhavbhat <vibhavb00@gmail.com>
* fix(proxy/_types.py): fixes issue where internal user able to escalat… (#8740)
* fix(proxy/_types.py): fixes issue where internal user able to escalate their role with ui key
Fixes https://github.com/BerriAI/litellm/issues/8029
* style: cleanup
* test: handle bedrock instability
---------
Co-authored-by: Madhukar Holla <mholla8@gmail.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Yury Koleda <fut.wrk@gmail.com>
Co-authored-by: Oskar Austegard <oskar@austegard.com>
Co-authored-by: Nate Mar <67926244+nate-mar@users.noreply.github.com>
Co-authored-by: vibhavbhat <vibhavb00@gmail.com>
* test: add new test image embedding to base llm unit tests
Addresses https://github.com/BerriAI/litellm/issues/6515
* fix(bedrock/embed/multimodal-embeddings): strip data prefix from image urls for bedrock multimodal embeddings
Fix https://github.com/BerriAI/litellm/issues/6515
* feat: initial commit for fireworks ai audio transcription support
Relevant issue: https://github.com/BerriAI/litellm/issues/7134
* test: initial fireworks ai test
* feat(fireworks_ai/): implemented fireworks ai audio transcription config
* fix(utils.py): register fireworks ai audio transcription config, in config manager
* fix(utils.py): add fireworks ai param translation to 'get_optional_params_transcription'
* refactor(fireworks_ai/): define text completion route with model name handling
moves model name handling to specific fireworks routes, as required by their api
* refactor(fireworks_ai/chat): define transform_Request - allows fixing model if accounts/ is missing
* fix: fix linting errors
* fix: fix linting errors
* fix: fix linting errors
* fix: fix linting errors
* fix(handler.py): fix linting errors
* fix(main.py): fix tgai text completion route
* refactor(together_ai/completion): refactors together ai text completion route to just use provider transform request
* refactor: move test_fine_tuning_api out of local_testing
reduces local testing ci/cd time
* fix(utils.py): support passing dynamic api base to validate_environment
Returns True if just api base is required and api base is passed
* fix(litellm_pre_call_utils.py): feature flag sending client headers to llm api
Fixes https://github.com/BerriAI/litellm/issues/6410
* fix(anthropic/chat/transformation.py): return correct error message
* fix(http_handler.py): add error response text in places where we expect it
* fix(factory.py): handle base case of no non-system messages to bedrock
Fixes https://github.com/BerriAI/litellm/issues/6411
* feat(cohere/embed): Support cohere image embeddings
Closes https://github.com/BerriAI/litellm/issues/6413
* fix(__init__.py): fix linting error
* docs(supported_embedding.md): add image embedding example to docs
* feat(cohere/embed): use cohere embedding returned usage for cost calc
* build(model_prices_and_context_window.json): add embed-english-v3.0 details (image cost + 'supports_image_input' flag)
* fix(cohere_transformation.py): fix linting error
* test(test_proxy_server.py): cleanup test
* test: cleanup test
* fix: fix linting errors
* feat(aws_base_llm.py): prevents recreating boto3 credentials during high traffic
Leads to 100ms perf boost in local testing
* fix(base_aws_llm.py): fix credential caching check to see if token is set
* refactor(bedrock/chat): separate converse api and invoke api + isolate converse api transformation logic
Make it easier to see how requests are transformed for /converse
* fix: fix imports
* fix(bedrock/embed): fix reordering of headers
* fix(base_aws_llm.py): fix get credential logic
* fix(converse_handler.py): fix ai21 streaming response
* fix(utils.py): return citations for perplexity streaming
Fixes https://github.com/BerriAI/litellm/issues/5535
* fix(anthropic/chat.py): support fallbacks for anthropic streaming (#5542)
* fix(anthropic/chat.py): support fallbacks for anthropic streaming
Fixes https://github.com/BerriAI/litellm/issues/5512
* fix(anthropic/chat.py): use module level http client if none given (prevents early client closure)
* fix: fix linting errors
* fix(http_handler.py): fix raise_for_status error handling
* test: retry flaky test
* fix otel type
* fix(bedrock/embed): fix error raising
* test(test_openai_batches_and_files.py): skip azure batches test (for now) quota exceeded
* fix(test_router.py): skip azure batch route test (for now) - hit batch quota limits
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* All `model_group_alias` should show up in `/models`, `/model/info` , `/model_group/info` (#5539)
* fix(router.py): support returning model_alias model names in `/v1/models`
* fix(proxy_server.py): support returning model alias'es on `/model/info`
* feat(router.py): support returning model group alias for `/model_group/info`
* fix(proxy_server.py): fix linting errors
* fix(proxy_server.py): fix linting errors
* build(model_prices_and_context_window.json): add amazon titan text premier pricing information
Closes https://github.com/BerriAI/litellm/issues/5560
* feat(litellm_logging.py): log standard logging response object for pass through endpoints. Allows bedrock /invoke agent calls to be correctly logged to langfuse + s3
* fix(success_handler.py): fix linting error
* fix(success_handler.py): fix linting errors
* fix(team_endpoints.py): Allows admin to update team member budgets
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* feat(proxy/_types.py): add lago billing to callbacks ui
Closes https://github.com/BerriAI/litellm/issues/5472
* fix(anthropic.py): return anthropic prompt caching information
Fixes https://github.com/BerriAI/litellm/issues/5364
* feat(bedrock/chat.py): support 'json_schema' for bedrock models
Closes https://github.com/BerriAI/litellm/issues/5434
* fix(bedrock/embed/embeddings.py): support async embeddings for amazon titan models
* fix: linting fixes
* fix: handle key errors
* fix(bedrock/chat.py): fix bedrock ai21 streaming object
* feat(bedrock/embed): support bedrock embedding optional params
* fix(databricks.py): fix usage chunk
* fix(internal_user_endpoints.py): apply internal user defaults, if user role updated
Fixes issue where user update wouldn't apply defaults
* feat(slack_alerting.py): provide multiple slack channels for a given alert type
multiple channels might be interested in receiving an alert for a given type
* docs(alerting.md): add multiple channel alerting to docs
* refactor(bedrock): initial commit to refactor bedrock to a folder
Improve code readability + maintainability
* refactor: more refactor work
* fix: fix imports
* feat(bedrock/embeddings.py): support translating embedding into amazon embedding formats
* fix: fix linting errors
* test: skip test on end of life model
* fix(cohere/embed.py): fix linting error
* fix(cohere/embed.py): fix typing
* fix(cohere/embed.py): fix post-call logging for cohere embedding call
* test(test_embeddings.py): fix error message assertion in test
