Ishaan Jaff
6f7e9b9728
[Feat SSO] Debug route - allow admins to debug SSO JWT fields ( #9835 )
...
* refactor SSO handler
* render sso JWT on ui
* docs debug sso
* fix sso login flow use await
* fix ui sso debug JWT
* test ui sso
* remove redis vl
* fix redisvl==0.5.1
* fix ml dtypes
* fix redisvl
* fix redis vl
* fix debug_sso_callback
* fix linting error
* fix redis semantic caching dep
2025-04-09 15:29:35 -07:00
Krrish Dholakia
0137055bad
docs: cleanup
2025-04-06 14:39:28 -07:00
Krish Dholakia
792ee079c2
Litellm 04 05 2025 release notes ( #9785 )
...
* docs: update docs
* docs: additional cleanup
* docs(index.md): add initial links
* docs: more doc updates
* docs(index.md): add more links
* docs(files.md): add gemini files API to docs
* docs(index.md): add more docs
* docs: more docs
* docs: update docs
2025-04-06 09:03:51 -07:00
Krish Dholakia
34bdf36eab
Add inference providers support for Hugging Face ( #8258 ) ( #9738 ) ( #9773 )
...
* Add inference providers support for Hugging Face (#8258 )
* add first version of inference providers for huggingface
* temporarily skipping tests
* Add documentation
* Fix titles
* remove max_retries from params and clean up
* add suggestions
* use llm http handler
* update doc
* add suggestions
* run formatters
* add tests
* revert
* revert
* rename file
* set maxsize for lru cache
* fix embeddings
* fix inference url
* fix tests following breaking change in main
* use ChatCompletionRequest
* fix tests and lint
* [Hugging Face] Remove outdated chat completion tests and fix embedding tests (#9749 )
* remove or fix tests
* fix link in doc
* fix(config_settings.md): document hf api key
---------
Co-authored-by: célina <hanouticelina@gmail.com>
2025-04-05 10:50:15 -07:00
Ishaan Jaff
82b8eb79c2
doc update
2025-04-02 23:11:22 -07:00
Ishaan Jaff
b48b8366c2
docs new deadlock fixing architecture
2025-04-02 13:24:53 -07:00
Krrish Dholakia
40a792472b
build(enterprise.md): add why enterprise to docs
2025-04-01 11:27:03 -07:00
Ishaan Jaff
31082344a4
docs release notes
Read Version from pyproject.toml / read-version (push) Successful in 21s
Helm unit test / unit-test (push) Successful in 43s
2025-03-29 23:08:20 -07:00
Ishaan Jaff
df01337bd8
docs litellm mcp
2025-03-29 22:28:03 -07:00
Ishaan Jaff
46e5ebe3c7
docs add mcp graphic to stable release notes
2025-03-29 19:00:26 -07:00
Krrish Dholakia
f1f40eba3f
docs(index.md): document new team model flow
Helm unit test / unit-test (push) Successful in 18s
Read Version from pyproject.toml / read-version (push) Successful in 22s
2025-03-27 12:14:29 -07:00
Krrish Dholakia
76df625718
docs(arize_integration.md): update docs with more details
2025-03-22 19:36:37 -07:00
Ishaan Jaff
ed74b419a3
Merge pull request #9436 from BerriAI/litellm_mcp_interface
...
[Feat] LiteLLM x MCP Bridge - Use MCP Tools with LiteLLM
2025-03-21 20:42:16 -07:00
Ishaan Jaff
b763174326
litellm mcp bridge docs
2025-03-21 17:00:56 -07:00
Ishaan Jaff
f07d127963
add diagram for litellm mcp integration
2025-03-21 16:54:55 -07:00
Krrish Dholakia
f68e27a0c8
docs(image_handling.md): architecture doc on image handling on the proxy
2025-03-21 14:19:05 -07:00
Ishaan Jaff
17e7718efc
docs custom prompt management
2025-03-19 20:25:09 -07:00
Ishaan Jaff
f505716499
docs ui credentials
Helm unit test / unit-test (push) Successful in 20s
Read Version from pyproject.toml / read-version (push) Successful in 40s
2025-03-15 14:38:14 -07:00
Ishaan Jaff
d93f2884d6
ui show logs page
2025-03-15 11:21:38 -07:00
Ishaan Jaff
b6f3d124d9
docs Stop storing Spend Logs in DB
2025-03-15 11:09:24 -07:00
Ishaan Jaff
8c849304be
docs ui improvements
2025-03-15 09:47:05 -07:00
Ishaan Jaff
88419db27b
docs - working management ui
2025-03-15 08:44:31 -07:00
Krrish Dholakia
ac3e57a501
docs: add images to release notes
2025-03-08 17:39:24 -08:00
Ishaan Jaff
b5eeafdd72
(Docs) OpenWeb x LiteLLM Docker compose + Instructions on spend tracking + logging ( #9059 )
...
* docs improve open web ui litellm doc
* docs openweb show teams + keys
* docs open web ui litellm
2025-03-07 17:01:39 -08:00
Ishaan Jaff
958e71b906
(Docs) connect litellm to open web ui ( #9040 )
...
* init doc
* working thinking tutorial
* docs open web ui with litellm
* minor edits
* docs one tab for tutorials
2025-03-06 21:13:00 -08:00
Krrish Dholakia
048dd995dc
docs: update litellm user management heirarchy doc
Read Version from pyproject.toml / read-version (push) Successful in 36s
2025-02-16 00:31:13 -08:00
Krish Dholakia
8d3a942fbd
Litellm staging ( #8270 )
...
Read Version from pyproject.toml / read-version (push) Successful in 15s
* fix(opik.py): cleanup
* docs(opik_integration.md): cleanup opik integration docs
* fix(redact_messages.py): fix redact messages check header logic
ensures stringified bool value in header is still asserted to true
allows dynamic message redaction
* feat(redact_messages.py): support `x-litellm-enable-message-redaction` request header
allows dynamic message redaction
2025-02-04 22:35:48 -08:00
Krish Dholakia
2eaa0079f2
feat(handle_jwt.py): initial commit adding custom RBAC support on jwt… ( #8037 )
...
* feat(handle_jwt.py): initial commit adding custom RBAC support on jwt auth
allows admin to define user role field and allowed roles which map to 'internal_user' on litellm
* fix(auth_checks.py): ensure user allowed to access model, when calling via personal keys
Fixes https://github.com/BerriAI/litellm/issues/8029
* feat(handle_jwt.py): support role based access with model permission control on proxy
Allows admin to just grant users roles on IDP (e.g. Azure AD/Keycloak) and user can immediately start calling models
* docs(rbac): add docs on rbac for model access control
make it clear how admin can use roles to control model access on proxy
* fix: fix linting errors
* test(test_user_api_key_auth.py): add unit testing to ensure rbac role is correctly enforced
* test(test_user_api_key_auth.py): add more testing
* test(test_users.py): add unit testing to ensure user model access is always checked for new keys
Resolves https://github.com/BerriAI/litellm/issues/8029
* test: fix unit test
* fix(dot_notation_indexing.py): fix typing to work with python 3.8
2025-01-28 16:27:06 -08:00
Yuki Watanabe
3f053fc99c
Update MLflow calllback and documentation ( #7809 )
...
* Update MLlfow tracer
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* doc update
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* doc update
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* image rename
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
2025-01-21 20:56:48 -08:00
Ishaan Jaff
bc311b7a47
ui logs - view messages / responses
2025-01-17 20:20:49 -08:00
Hugues Chocart
6fff77d131
[integrations/lunary] Improve Lunary documentaiton ( #7770 )
...
* update lunary doc
* better title
* tweaks
* Update langchain.md
* Update lunary_integration.md
2025-01-15 15:00:25 -08:00
Ishaan Jaff
f95439af26
docs v1.57.3
2025-01-08 12:08:19 -08:00
Ishaan Jaff
04eb718f7a
update docs
2025-01-07 22:35:07 -08:00
Ishaan Jaff
46d9d29bff
(Feat) Hashicorp Secret Manager - Allow storing virtual keys in secret manager ( #7549 )
...
Read Version from pyproject.toml / read-version (push) Successful in 13s
* use a base abstract class
* async_write_secret for hcorp
* fix hcorp
* async_write_secret for hashicopr secret manager
* store virtual keys in hcorp
* add delete secret
* test_hashicorp_secret_manager_write_secret
* test_hashicorp_secret_manager_delete_secret
* docs Supported Secret Managers
* docs storing keys in hcorp
* docs hcorp
* docs secret managers
* test_key_generate_with_secret_manager_call
* fix unused imports
2025-01-04 11:35:59 -08:00
Ishaan Jaff
cf60444916
(Feat) Add support for reading secrets from Hashicorp vault ( #7497 )
...
* HashicorpSecretManager
* test_hashicorp_secret_managerv
* use 1 helper initialize_secret_manager
* add HASHICORP_VAULT
* working config
* hcorp read_secret
* HashicorpSecretManager
* add secret_manager_testing
* use 1 folder for secret manager testing
* test_hashicorp_secret_manager_get_secret
* HashicorpSecretManager
* docs HCP secrets
* update folder name
* docs hcorp secret manager
* remove unused imports
* add conftest.py
* fix tests
* docs document env vars
2025-01-01 18:35:05 -08:00
Ishaan Jaff
e1fcd3ee43
(docs) Add docs on load testing benchmarks ( #7499 )
...
* docs benchmarks
* docs benchmarks
2025-01-01 18:33:20 -08:00
Ishaan Jaff
03b1db5a7d
(Feat) - Add PagerDuty Alerting Integration ( #7478 )
...
* define basic types
* fix verbose_logger.exception statement
* fix basic alerting
* test pager duty alerting
* test_pagerduty_alerting_high_failure_rate
* PagerDutyAlerting
* async_log_failure_event
* use pre_call_hook
* add _request_is_completed helper util
* update AlertingConfig
* rename PagerDutyInternalEvent
* _send_alert_if_thresholds_crossed
* use pagerduty as _custom_logger_compatible_callbacks_literal
* fix slack alerting imports
* fix imports in slack alerting
* PagerDutyAlerting
* fix _load_alerting_settings
* test_pagerduty_hanging_request_alerting
* working pager duty alerting
* fix linting
* doc pager duty alerting
* update hanging_response_handler
* fix import location
* update failure_threshold
* update async_pre_call_hook
* docs pagerduty
* test - callback_class_str_to_classType
* fix linting errors
* fix linting + testing error
* PagerDutyAlerting
* test_pagerduty_hanging_request_alerting
* fix unused imports
* docs pager duty
* @pytest.mark.flaky(retries=6, delay=2)
* test_model_info_bedrock_converse_enforcement
2025-01-01 07:12:51 -08:00
Ishaan Jaff
8c569899c0
update release note
2024-12-28 20:15:30 -08:00
Ishaan Jaff
8610c7bf93
docs release notes
Read Version from pyproject.toml / read-version (push) Successful in 41s
2024-12-27 21:41:21 -08:00
Ishaan Jaff
62753eea69
✨ (Feat) Log Guardrails run, guardrail response on logging integrations ( #7445 )
...
* add guardrail_information to SLP
* use standard_logging_guardrail_information
* track StandardLoggingGuardrailInformation
* use log_guardrail_information
* use log_guardrail_information
* docs guardrails
* docs guardrails
* update quick start
* fix presidio logging for sync functions
* update Guardrail type
* enforce add_standard_logging_guardrail_information_to_request_data
* update gd docs
2024-12-27 15:01:56 -08:00
Krish Dholakia
539f166166
Support budget/rate limit tiers for keys ( #7429 )
...
* feat(proxy/utils.py): get associated litellm budget from db in combined_view for key
allows user to create rate limit tiers and associate those to keys
* feat(proxy/_types.py): update the value of key-level tpm/rpm/model max budget metrics with the associated budget table values if set
allows rate limit tiers to be easily applied to keys
* docs(rate_limit_tiers.md): add doc on setting rate limit / budget tiers
make feature discoverable
* feat(key_management_endpoints.py): return litellm_budget_table value in key generate
make it easy for user to know associated budget on key creation
* fix(key_management_endpoints.py): document 'budget_id' param in `/key/generate`
* docs(key_management_endpoints.py): document budget_id usage
* refactor(budget_management_endpoints.py): refactor budget endpoints into separate file - makes it easier to run documentation testing against it
* docs(test_api_docs.py): add budget endpoints to ci/cd doc test + add missing param info to docs
* fix(customer_endpoints.py): use new pydantic obj name
* docs(user_management_heirarchy.md): add simple doc explaining teams/keys/org/users on litellm
* Litellm dev 12 26 2024 p2 (#7432 )
* (Feat) Add logging for `POST v1/fine_tuning/jobs` (#7426 )
* init commit ft jobs logging
* add ft logging
* add logging for FineTuningJob
* simple FT Job create test
* (docs) - show all supported Azure OpenAI endpoints in overview (#7428 )
* azure batches
* update doc
* docs azure endpoints
* docs endpoints on azure
* docs azure batches api
* docs azure batches api
* fix(key_management_endpoints.py): fix key update to actually work
* test(test_key_management.py): add e2e test asserting ui key update call works
* fix: proxy/_types - fix linting erros
* test: update test
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* fix: test
* fix(parallel_request_limiter.py): enforce tpm/rpm limits on key from tiers
* fix: fix linting errors
* test: fix test
* fix: remove unused import
* test: update test
* docs(customer_endpoints.py): document new model_max_budget param
* test: specify unique key alias
* docs(budget_management_endpoints.py): document new model_max_budget param
* test: fix test
* test: fix tests
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2024-12-26 19:05:27 -08:00
Krish Dholakia
c3edfc2c92
LiteLLM Minor Fixes & Improvements (12/23/2024) - p3 ( #7394 )
...
Read Version from pyproject.toml / read-version (push) Successful in 35s
* build(model_prices_and_context_window.json): add gemini-1.5-flash context caching
* fix(context_caching/transformation.py): just use last identified cache point
Fixes https://github.com/BerriAI/litellm/issues/6738
* fix(context_caching/transformation.py): pick first contiguous block - handles system message error from google
Fixes https://github.com/BerriAI/litellm/issues/6738
* fix(vertex_ai/gemini/): track context caching tokens
* refactor(gemini/): place transformation.py inside `chat/` folder
make it easy for user to know we support the equivalent endpoint
* fix: fix import
* refactor(vertex_ai/): move vertex_ai cost calc inside vertex_ai/ folder
make it easier to see cost calculation logic
* fix: fix linting errors
* fix: fix circular import
* feat(gemini/cost_calculator.py): support gemini context caching cost calculation
generifies anthropic's cost calculation function and uses it across anthropic + gemini
* build(model_prices_and_context_window.json): add cost tracking for gemini-1.5-flash-002 w/ context caching
Closes https://github.com/BerriAI/litellm/issues/6891
* docs(gemini.md): add gemini context caching architecture diagram
make it easier for user to understand how context caching works
* docs(gemini.md): link to relevant gemini context caching code
* docs(gemini/context_caching): add readme in github, make it easy for dev to know context caching is supported + where to go for code
* fix(llm_cost_calc/utils.py): handle gemini 128k token diff cost calc scenario
* fix(deepseek/cost_calculator.py): support deepseek context caching cost calculation
* test: fix test
2024-12-23 22:02:52 -08:00
Ishaan Jaff
1f466ec9cf
release notes
2024-12-23 21:38:56 -08:00
Ishaan Jaff
8b1ea40e7b
add img to release notes
Read Version from pyproject.toml / read-version (push) Successful in 36s
2024-12-21 21:24:16 -08:00
Krish Dholakia
27a4d08604
Litellm dev 2024 12 19 p3 ( #7322 )
...
Read Version from pyproject.toml / read-version (push) Successful in 13s
* fix(utils.py): remove unsupported optional params (if drop_params=True) before passing into map openai params
Fixes https://github.com/BerriAI/litellm/issues/7242
* test: new test for langfuse prompt management hook
Addresses https://github.com/BerriAI/litellm/issues/3893#issuecomment-2549080296
* feat(main.py): add 'get_chat_completion_prompt' customlogger hook
allows for langfuse prompt management
Addresses https://github.com/BerriAI/litellm/issues/3893#issuecomment-2549080296
* feat(langfuse_prompt_management.py): working e2e langfuse prompt management
works with `langfuse/` route
* feat(main.py): initial tracing for dynamic langfuse params
allows admin to specify langfuse keys by model in model_list
* feat(main.py): support passing langfuse credentials dynamically
* fix(langfuse_prompt_management.py): create langfuse client based on dynamic callback params
allows dynamic langfuse params to work
* fix: fix linting errors
* docs(prompt_management.md): refactor docs for sdk + proxy prompt management tutorial
* docs(prompt_management.md): cleanup doc
* docs: cleanup topnav
* docs(prompt_management.md): update docs to be easier to use
* fix: remove unused imports
* docs(prompt_management.md): add architectural overview doc
* fix(litellm_logging.py): fix dynamic param passing
* fix(langfuse_prompt_management.py): fix linting errors
* fix: fix linting errors
* fix: use typing_extensions for typealias to ensure python3.8 compatibility
* test: use stream_options in test to account for tiktoken diff
* fix: improve import error message, and check run test earlier
2024-12-20 13:30:16 -08:00
Ishaan Jaff
3c984ed60e
(feat) Add Azure Blob Storage Logging Integration ( #7265 )
...
* add path to http handler
* AzureBlobStorageLogger
* test_azure_blob_storage
* use constants for Azure storage
* use helper get_azure_ad_token_from_entrata_id
* azure blob storage support
* get_azure_ad_token_from_azure_storage
* fix import
* azure logging
* docs azure storage
* add docs on azure blobs
* add premium user check
* add azure_storage as identified logging callback
* async_upload_payload_to_azure_blob_storage
* docs azure storage
* callback_class_str_to_classType
2024-12-16 22:18:22 -08:00
Krish Dholakia
ec36353b41
fix(main.py): fix retries being multiplied when using openai sdk ( #7221 )
...
* fix(main.py): fix retries being multiplied when using openai sdk
Closes https://github.com/BerriAI/litellm/pull/7130
* docs(prompt_management.md): add langfuse prompt management doc
* feat(team_endpoints.py): allow teams to add their own models
Enables teams to call their own finetuned models via the proxy
* test: add better enforcement check testing for `/model/new` now that teams can add their own models
* docs(team_model_add.md): tutorial for allowing teams to add their own models
* test: fix test
2024-12-14 11:56:55 -08:00
Krrish Dholakia
d26ad42f86
docs(router_architecture.md): add router architecture docs
2024-11-26 12:54:38 +05:30
Krish Dholakia
b11bc0374e
Litellm dev 11 20 2024 ( #6838 )
...
* feat(customer_endpoints.py): support passing budget duration via `/customer/new` endpoint
Closes https://github.com/BerriAI/litellm/issues/5651
* docs: add missing params to swagger + api documentation test
* docs: add documentation for all key endpoints
documents all params on swagger
* docs(internal_user_endpoints.py): document all /user/new params
Ensures all params are documented
* docs(team_endpoints.py): add missing documentation for team endpoints
Ensures 100% param documentation on swagger
* docs(organization_endpoints.py): document all org params
Adds documentation for all params in org endpoint
* docs(customer_endpoints.py): add coverage for all params on /customer endpoints
ensures all /customer/* params are documented
* ci(config.yml): add endpoint doc testing to ci/cd
* fix: fix internal_user_endpoints.py
* fix(internal_user_endpoints.py): support 'duration' param
* fix(partner_models/main.py): fix anthropic re-raise exception on vertex
* fix: fix pydantic obj
* build(model_prices_and_context_window.json): add new vertex claude model names
vertex claude changed model names - causes cost tracking errors
2024-11-21 05:20:37 +05:30
Krish Dholakia
3beecfb0d4
LiteLLM Minor Fixes & Improvements (11/13/2024) ( #6729 )
...
* fix(utils.py): add logprobs support for together ai
Fixes
https://github.com/BerriAI/litellm/issues/6724
* feat(pass_through_endpoints/): add anthropic/ pass-through endpoint
adds new `anthropic/` pass-through endpoint + refactors docs
* feat(spend_management_endpoints.py): allow /global/spend/report to query team + customer id
enables seeing spend for a customer in a team
* Add integration with MLflow Tracing (#6147 )
* Add MLflow logger
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Streaming handling
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* lint
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* address comments and fix issues
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* address comments and fix issues
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Move logger construction code
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Add docs
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* async handlers
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* new picture
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* fix(mlflow.py): fix ruff linting errors
* ci(config.yml): add mlflow to ci testing
* fix: fix test
* test: fix test
* Litellm key update fix (#6710 )
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* fix(key_management_endpoints.py): fix /key/update with metadata update
* fix(key_management_endpoints.py): fix key_prepare_update helper
* fix(key_management_endpoints.py): reset value to none if set in key update
* fix: update test
'
* Litellm dev 11 11 2024 (#6693 )
* fix(__init__.py): add 'watsonx_text' as mapped llm api route
Fixes https://github.com/BerriAI/litellm/issues/6663
* fix(opentelemetry.py): fix passing parallel tool calls to otel
Fixes https://github.com/BerriAI/litellm/issues/6677
* refactor(test_opentelemetry_unit_tests.py): create a base set of unit tests for all logging integrations - test for parallel tool call handling
reduces bugs in repo
* fix(__init__.py): update provider-model mapping to include all known provider-model mappings
Fixes https://github.com/BerriAI/litellm/issues/6669
* feat(anthropic): support passing document in llm api call
* docs(anthropic.md): add pdf anthropic call to docs + expose new 'supports_pdf_input' function
* fix(factory.py): fix linting error
* add clear doc string for GCS bucket logging
* Add docs to export logs to Laminar (#6674 )
* Add docs to export logs to Laminar
* minor fix: newline at end of file
* place laminar after http and grpc
* (Feat) Add langsmith key based logging (#6682 )
* add langsmith_api_key to StandardCallbackDynamicParams
* create a file for langsmith types
* langsmith add key / team based logging
* add key based logging for langsmith
* fix langsmith key based logging
* fix linting langsmith
* remove NOQA violation
* add unit test coverage for all helpers in test langsmith
* test_langsmith_key_based_logging
* docs langsmith key based logging
* run langsmith tests in logging callback tests
* fix logging testing
* test_langsmith_key_based_logging
* test_add_callback_via_key_litellm_pre_call_utils_langsmith
* add debug statement langsmith key based logging
* test_langsmith_key_based_logging
* (fix) OpenAI's optional messages[].name does not work with Mistral API (#6701 )
* use helper for _transform_messages mistral
* add test_message_with_name to base LLMChat test
* fix linting
* add xAI on Admin UI (#6680 )
* (docs) add benchmarks on 1K RPS (#6704 )
* docs litellm proxy benchmarks
* docs GCS bucket
* doc fix - reduce clutter on logging doc title
* (feat) add cost tracking stable diffusion 3 on Bedrock (#6676 )
* add cost tracking for sd3
* test_image_generation_bedrock
* fix get model info for image cost
* add cost_calculator for stability 1 models
* add unit testing for bedrock image cost calc
* test_cost_calculator_with_no_optional_params
* add test_cost_calculator_basic
* correctly allow size Optional
* fix cost_calculator
* sd3 unit tests cost calc
* fix raise correct error 404 when /key/info is called on non-existent key (#6653 )
* fix raise correct error on /key/info
* add not_found_error error
* fix key not found in DB error
* use 1 helper for checking token hash
* fix error code on key info
* fix test key gen prisma
* test_generate_and_call_key_info
* test fix test_call_with_valid_model_using_all_models
* fix key info tests
* bump: version 1.52.4 → 1.52.5
* add defaults used for GCS logging
* LiteLLM Minor Fixes & Improvements (11/12/2024) (#6705 )
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* bump: version 1.52.5 → 1.52.6
* (feat) helm hook to sync db schema (#6715 )
* v0 migration job
* fix job
* fix migrations job.yml
* handle standalone DB on helm hook
* fix argo cd annotations
* fix db migration helm hook
* fix migration job
* doc fix Using Http/2 with Hypercorn
* (fix proxy redis) Add redis sentinel support (#6154 )
* add sentinel_password support
* add doc for setting redis sentinel password
* fix redis sentinel - use sentinel password
* Fix: Update gpt-4o costs to that of gpt-4o-2024-08-06 (#6714 )
Fixes #6713
* (fix) using Anthropic `response_format={"type": "json_object"}` (#6721 )
* add support for response_format=json anthropic
* add test_json_response_format to baseLLM ChatTest
* fix test_litellm_anthropic_prompt_caching_tools
* fix test_anthropic_function_call_with_no_schema
* test test_create_json_tool_call_for_response_format
* (feat) Add cost tracking for Azure Dall-e-3 Image Generation + use base class to ensure basic image generation tests pass (#6716 )
* add BaseImageGenTest
* use 1 class for unit testing
* add debugging to BaseImageGenTest
* TestAzureOpenAIDalle3
* fix response_cost_calculator
* test_basic_image_generation
* fix img gen basic test
* fix _select_model_name_for_cost_calc
* fix test_aimage_generation_bedrock_with_optional_params
* fix undo changes cost tracking
* fix response_cost_calculator
* fix test_cost_azure_gpt_35
* fix remove dup test (#6718 )
* (build) update db helm hook
* (build) helm db pre sync hook
* (build) helm db sync hook
* test: run test_team_logging firdst
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
* test: update test
* test: skip anthropic overloaded error
* test: cleanup test
* test: update tests
* test: fix test
* test: handle gemini overloaded model error
* test: handle internal server error
* test: handle anthropic overloaded error
* test: handle claude instability
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
Co-authored-by: Yuki Watanabe <31463517+B-Step62@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
2024-11-15 11:18:31 +05:30
