Ishaan Jaff
1890fde3f3
(Proxy) add support for DOCS_URL and REDOC_URL ( #6806 )
...
* add support for DOCS_URL and REDOC_URL
* document env vars
* add unit tests for docs url and redocs url
2024-11-19 07:02:12 -08:00
Krrish Dholakia
7550aba474
docs(gemini.md): add embeddings as a supported endpoint for gemini models
2024-11-19 10:27:02 +05:30
Krrish Dholakia
df817b9ab7
bump: version 1.52.10 → 1.52.11
2024-11-19 10:05:16 +05:30
Krish Dholakia
ba28e52ee8
Litellm lm studio embedding params ( #6746 )
...
* fix(ollama.py): fix get model info request
Fixes https://github.com/BerriAI/litellm/issues/6703
* feat(anthropic/chat/transformation.py): support passing user id to anthropic via openai 'user' param
* docs(anthropic.md): document all supported openai params for anthropic
* test: fix tests
* fix: fix tests
* feat(jina_ai/): add rerank support
Closes https://github.com/BerriAI/litellm/issues/6691
* test: handle service unavailable error
* fix(handler.py): refactor together ai rerank call
* test: update test to handle overloaded error
* test: fix test
* Litellm router trace (#6742 )
* feat(router.py): add trace_id to parent functions - allows tracking retry/fallbacks
* feat(router.py): log trace id across retry/fallback logic
allows grouping llm logs for the same request
* test: fix tests
* fix: fix test
* fix(transformation.py): only set non-none stop_sequences
* Litellm router disable fallbacks (#6743 )
* bump: version 1.52.6 → 1.52.7
* feat(router.py): enable dynamically disabling fallbacks
Allows for enabling/disabling fallbacks per key
* feat(litellm_pre_call_utils.py): support setting 'disable_fallbacks' on litellm key
* test: fix test
* fix(exception_mapping_utils.py): map 'model is overloaded' to internal server error
* fix(lm_studio/embed): support translating lm studio optional params
'
* feat(auth_checks.py): fix auth check inside route - `/team/list`
Fixes regression where non-admin w/ user_id=None able to query all teams
* docs proxy_budget_rescheduler_min_time
* helm run DISABLE_SCHEMA_UPDATE
* docs helm pre sync hook
* fix migration job.yaml
* fix DATABASE_URL
* use existing spec for migrations job
* fix yaml on migrations job
* fix migration job
* update doc on pre sync hook
* fix migrations-job.yaml
* fix migration job
* fix prisma migration
* test - handle eol model claude-2, use claude-2.1 instead
* (docs) add instructions on how to contribute to docker image
* Update code blocks huggingface.md (#6737 )
* Update prefix.md (#6734 )
* fix test_supports_response_schema
* mark Helm PreSyn as BETA
* (Feat) Add support for storing virtual keys in AWS SecretManager (#6728 )
* add SecretManager to httpxSpecialProvider
* fix importing AWSSecretsManagerV2
* add unit testing for writing keys to AWS secret manager
* use KeyManagementEventHooks for key/generated events
* us event hooks for key management endpoints
* working AWSSecretsManagerV2
* fix write secret to AWS secret manager on /key/generate
* fix KeyManagementSettings
* use tasks for key management hooks
* add async_delete_secret
* add test for async_delete_secret
* use _delete_virtual_keys_from_secret_manager
* fix test secret manager
* test_key_generate_with_secret_manager_call
* fix check for key_management_settings
* sync_read_secret
* test_aws_secret_manager
* fix sync_read_secret
* use helper to check when _should_read_secret_from_secret_manager
* test_get_secret_with_access_mode
* test - handle eol model claude-2, use claude-2.1 instead
* docs AWS secret manager
* fix test_read_nonexistent_secret
* fix test_supports_response_schema
* ci/cd run again
* LiteLLM Minor Fixes & Improvement (11/14/2024) (#6730 )
* fix(ollama.py): fix get model info request
Fixes https://github.com/BerriAI/litellm/issues/6703
* feat(anthropic/chat/transformation.py): support passing user id to anthropic via openai 'user' param
* docs(anthropic.md): document all supported openai params for anthropic
* test: fix tests
* fix: fix tests
* feat(jina_ai/): add rerank support
Closes https://github.com/BerriAI/litellm/issues/6691
* test: handle service unavailable error
* fix(handler.py): refactor together ai rerank call
* test: update test to handle overloaded error
* test: fix test
* Litellm router trace (#6742 )
* feat(router.py): add trace_id to parent functions - allows tracking retry/fallbacks
* feat(router.py): log trace id across retry/fallback logic
allows grouping llm logs for the same request
* test: fix tests
* fix: fix test
* fix(transformation.py): only set non-none stop_sequences
* Litellm router disable fallbacks (#6743 )
* bump: version 1.52.6 → 1.52.7
* feat(router.py): enable dynamically disabling fallbacks
Allows for enabling/disabling fallbacks per key
* feat(litellm_pre_call_utils.py): support setting 'disable_fallbacks' on litellm key
* test: fix test
* fix(exception_mapping_utils.py): map 'model is overloaded' to internal server error
* test: handle gemini error
* test: fix test
* fix: new run
* bump: version 1.52.7 → 1.52.8
* docs: add docs on jina ai rerank support
* docs(reliability.md): add tutorial on disabling fallbacks per key
* docs(logging.md): add 'trace_id' param to standard logging payload
* (feat) add bedrock/stability.stable-image-ultra-v1:0 (#6723 )
* add stability.stable-image-ultra-v1:0
* add pricing for stability.stable-image-ultra-v1:0
* fix test_supports_response_schema
* ci/cd run again
* [Feature]: Stop swallowing up AzureOpenAi exception responses in litellm's implementation for a BadRequestError (#6745 )
* fix azure exceptions
* test_bad_request_error_contains_httpx_response
* test_bad_request_error_contains_httpx_response
* use safe access to get exception response
* fix get attr
* [Feature]: json_schema in response support for Anthropic (#6748 )
* _convert_tool_response_to_message
* fix ModelResponseIterator
* fix test_json_response_format
* test_json_response_format_stream
* fix _convert_tool_response_to_message
* use helper _handle_json_mode_chunk
* fix _process_response
* unit testing for test_convert_tool_response_to_message_no_arguments
* update doc for JSON mode
* fix: import audio check (#6740 )
* fix imagegeneration output_cost_per_image on model cost map (#6752 )
* (feat) Vertex AI - add support for fine tuned embedding models (#6749 )
* fix use fine tuned vertex embedding models
* test_vertex_embedding_url
* add _transform_openai_request_to_fine_tuned_embedding_request
* add _transform_openai_request_to_fine_tuned_embedding_request
* add transform_openai_request_to_vertex_embedding_request
* add _transform_vertex_response_to_openai_for_fine_tuned_models
* test_vertexai_embedding for ft models
* fix test_vertexai_embedding_finetuned
* doc fine tuned / custom embedding models
* fix test test_partner_models_httpx
* bump: version 1.52.8 → 1.52.9
* LiteLLM Minor Fixes & Improvements (11/13/2024) (#6729 )
* fix(utils.py): add logprobs support for together ai
Fixes
https://github.com/BerriAI/litellm/issues/6724
* feat(pass_through_endpoints/): add anthropic/ pass-through endpoint
adds new `anthropic/` pass-through endpoint + refactors docs
* feat(spend_management_endpoints.py): allow /global/spend/report to query team + customer id
enables seeing spend for a customer in a team
* Add integration with MLflow Tracing (#6147 )
* Add MLflow logger
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Streaming handling
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* lint
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* address comments and fix issues
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* address comments and fix issues
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Move logger construction code
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Add docs
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* async handlers
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* new picture
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* fix(mlflow.py): fix ruff linting errors
* ci(config.yml): add mlflow to ci testing
* fix: fix test
* test: fix test
* Litellm key update fix (#6710 )
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* fix(key_management_endpoints.py): fix /key/update with metadata update
* fix(key_management_endpoints.py): fix key_prepare_update helper
* fix(key_management_endpoints.py): reset value to none if set in key update
* fix: update test
'
* Litellm dev 11 11 2024 (#6693 )
* fix(__init__.py): add 'watsonx_text' as mapped llm api route
Fixes https://github.com/BerriAI/litellm/issues/6663
* fix(opentelemetry.py): fix passing parallel tool calls to otel
Fixes https://github.com/BerriAI/litellm/issues/6677
* refactor(test_opentelemetry_unit_tests.py): create a base set of unit tests for all logging integrations - test for parallel tool call handling
reduces bugs in repo
* fix(__init__.py): update provider-model mapping to include all known provider-model mappings
Fixes https://github.com/BerriAI/litellm/issues/6669
* feat(anthropic): support passing document in llm api call
* docs(anthropic.md): add pdf anthropic call to docs + expose new 'supports_pdf_input' function
* fix(factory.py): fix linting error
* add clear doc string for GCS bucket logging
* Add docs to export logs to Laminar (#6674 )
* Add docs to export logs to Laminar
* minor fix: newline at end of file
* place laminar after http and grpc
* (Feat) Add langsmith key based logging (#6682 )
* add langsmith_api_key to StandardCallbackDynamicParams
* create a file for langsmith types
* langsmith add key / team based logging
* add key based logging for langsmith
* fix langsmith key based logging
* fix linting langsmith
* remove NOQA violation
* add unit test coverage for all helpers in test langsmith
* test_langsmith_key_based_logging
* docs langsmith key based logging
* run langsmith tests in logging callback tests
* fix logging testing
* test_langsmith_key_based_logging
* test_add_callback_via_key_litellm_pre_call_utils_langsmith
* add debug statement langsmith key based logging
* test_langsmith_key_based_logging
* (fix) OpenAI's optional messages[].name does not work with Mistral API (#6701 )
* use helper for _transform_messages mistral
* add test_message_with_name to base LLMChat test
* fix linting
* add xAI on Admin UI (#6680 )
* (docs) add benchmarks on 1K RPS (#6704 )
* docs litellm proxy benchmarks
* docs GCS bucket
* doc fix - reduce clutter on logging doc title
* (feat) add cost tracking stable diffusion 3 on Bedrock (#6676 )
* add cost tracking for sd3
* test_image_generation_bedrock
* fix get model info for image cost
* add cost_calculator for stability 1 models
* add unit testing for bedrock image cost calc
* test_cost_calculator_with_no_optional_params
* add test_cost_calculator_basic
* correctly allow size Optional
* fix cost_calculator
* sd3 unit tests cost calc
* fix raise correct error 404 when /key/info is called on non-existent key (#6653 )
* fix raise correct error on /key/info
* add not_found_error error
* fix key not found in DB error
* use 1 helper for checking token hash
* fix error code on key info
* fix test key gen prisma
* test_generate_and_call_key_info
* test fix test_call_with_valid_model_using_all_models
* fix key info tests
* bump: version 1.52.4 → 1.52.5
* add defaults used for GCS logging
* LiteLLM Minor Fixes & Improvements (11/12/2024) (#6705 )
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* bump: version 1.52.5 → 1.52.6
* (feat) helm hook to sync db schema (#6715 )
* v0 migration job
* fix job
* fix migrations job.yml
* handle standalone DB on helm hook
* fix argo cd annotations
* fix db migration helm hook
* fix migration job
* doc fix Using Http/2 with Hypercorn
* (fix proxy redis) Add redis sentinel support (#6154 )
* add sentinel_password support
* add doc for setting redis sentinel password
* fix redis sentinel - use sentinel password
* Fix: Update gpt-4o costs to that of gpt-4o-2024-08-06 (#6714 )
Fixes #6713
* (fix) using Anthropic `response_format={"type": "json_object"}` (#6721 )
* add support for response_format=json anthropic
* add test_json_response_format to baseLLM ChatTest
* fix test_litellm_anthropic_prompt_caching_tools
* fix test_anthropic_function_call_with_no_schema
* test test_create_json_tool_call_for_response_format
* (feat) Add cost tracking for Azure Dall-e-3 Image Generation + use base class to ensure basic image generation tests pass (#6716 )
* add BaseImageGenTest
* use 1 class for unit testing
* add debugging to BaseImageGenTest
* TestAzureOpenAIDalle3
* fix response_cost_calculator
* test_basic_image_generation
* fix img gen basic test
* fix _select_model_name_for_cost_calc
* fix test_aimage_generation_bedrock_with_optional_params
* fix undo changes cost tracking
* fix response_cost_calculator
* fix test_cost_azure_gpt_35
* fix remove dup test (#6718 )
* (build) update db helm hook
* (build) helm db pre sync hook
* (build) helm db sync hook
* test: run test_team_logging firdst
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
* test: update test
* test: skip anthropic overloaded error
* test: cleanup test
* test: update tests
* test: fix test
* test: handle gemini overloaded model error
* test: handle internal server error
* test: handle anthropic overloaded error
* test: handle claude instability
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
Co-authored-by: Yuki Watanabe <31463517+B-Step62@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Jongseob Jeon <aiden.jongseob@gmail.com>
Co-authored-by: Camden Clark <camdenaws@gmail.com>
Co-authored-by: Rasswanth <61219215+IamRash-7@users.noreply.github.com>
Co-authored-by: Yuki Watanabe <31463517+B-Step62@users.noreply.github.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
2024-11-19 09:54:50 +05:30
Ishaan Jaff
51ffe93e77
(docs) add docstrings for all /key, /user, /team, /customer endpoints ( #6804 )
...
* use helper to handle_exception_on_proxy
* add doc string for /key/regenerate
* use 1 helper for handle_exception_on_proxy
* add doc string for /key/block
* add doc string for /key/unblock
* remove deprecated function
* remove deprecated endpoints
* remove incorrect tag for endpoint
* fix linting
* fix /key/regenerate
* fix regen key
* fix use port 4000 for user endpoints
* fix clean up - use separate file for customer endpoints
* add docstring for user/update
* fix imports
* doc string /user/list
* doc string for /team/delete
* fix team block endpoint
* fix import block user
* add doc string for /team/unblock
* add doc string for /team/list
* add doc string for /team/info
* add doc string for key endpoints
* fix customer_endpoints
* add doc string for customer endpoints
* fix import new_end_user
* fix testing
* fix import new_end_user
* fix add check for allow_user_auth
2024-11-18 19:44:06 -08:00
Ishaan Jaff
994fb51016
Docs - use 1 page for all logging integrations on proxy + add logging features at top level ( #6805 )
...
* use 1 page for bucket logging
* docs logging proxy
* remove dup doc
* docs fix emoji
* docs team logging
2024-11-18 18:35:52 -08:00
dependabot[bot]
94029af328
Bump cross-spawn from 7.0.3 to 7.0.5 in /ui ( #6779 )
...
Bumps [cross-spawn](https://github.com/moxystudio/node-cross-spawn ) from 7.0.3 to 7.0.5.
- [Changelog](https://github.com/moxystudio/node-cross-spawn/blob/master/CHANGELOG.md )
- [Commits](https://github.com/moxystudio/node-cross-spawn/compare/v7.0.3...v7.0.5 )
---
updated-dependencies:
- dependency-name: cross-spawn
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-18 14:07:44 -08:00
Ishaan Jaff
7bb5304323
(docs) simplify left nav names + use a section for making llm requests ( #6799 )
...
* fix emojis on docs
* add section on making LLM requests
* docs simplify sidebar
2024-11-18 12:53:43 -08:00
Ishaan Jaff
bbdec2995a
(docs improvement) remove emojis, use guides section, categorize uncategorized docs ( #6796 )
...
* proxy - use Setup & Deployment category
* fix emoji
* use guides section to user facing usage
* docs - remove emojis
* use 1 quick start
2024-11-18 12:23:54 -08:00
Ishaan Jaff
f43768d617
(fix) httpx handler - bind to ipv4 for httpx handler ( #6785 )
...
* bind to ipv4 on httpx handler
* add force_ipv4
* use helper for _create_async_transport
* fix circular import
* document force_ipv4
* test_async_http_handler_force_ipv4
2024-11-18 12:22:51 -08:00
Krish Dholakia
b854f6c07b
build: add gemini-exp-1114 ( #6786 )
...
Fixes
2024-11-18 12:44:39 +05:30
Ishaan Jaff
128eeb4997
handle vertex ServiceUnavailableError for codestral
2024-11-17 18:45:58 -08:00
Ishaan Jaff
e1ca95672a
vertex_ai/codestral@2405 is very unstable - handle their instability in our tests
2024-11-17 18:17:14 -08:00
Ishaan Jaff
585b54e70c
handle codestral@2405 instability
2024-11-17 17:55:19 -08:00
Ishaan Jaff
5f298cb9de
bump: version 1.52.9 → 1.52.10
2024-11-16 20:09:52 -08:00
Ishaan Jaff
f5c8150ae2
new ui build
2024-11-16 20:09:29 -08:00
Ishaan Jaff
160357d54c
(fix) Azure AI Studio - using image_url in content with both text and image_url ( #6774 )
...
* use helper _audio_or_image_in_message_content
* update azure ai transf
* test_azure_ai_with_image_url
2024-11-16 20:05:24 -08:00
Ishaan Jaff
0c7360d470
(patch) using image_urls with vertex/anthropic models ( #6775 )
...
* fix is_pdf_used check anthropic
* fix validate_environment
* add comment on is_vertex_request
2024-11-16 20:05:02 -08:00
Ishaan Jaff
401531a8c9
fix test_completion_codestral_fim_api_stream
2024-11-16 20:02:27 -08:00
Ishaan Jaff
842bfc4ebf
(docs) add doc string for /key/update ( #6778 )
...
* add /key/update docstring
* fix /key/update doc string
* fix info_key_fn_v2
2024-11-16 20:01:10 -08:00
Ishaan Jaff
bb16abc043
(UI) fix - allow editing key alias on Admin UI ( #6776 )
...
* fix allow editing key alias on UI
* fix non type for budget duration
2024-11-16 19:58:26 -08:00
Ishaan Jaff
4ed270506a
(Admin UI) - Remain on Current Tab when user clicks refresh ( #6777 )
...
* UI - fix, remain on current tab after refresh
* leftnav - remain on current tab after refresh
2024-11-16 19:55:41 -08:00
Ishaan Jaff
627d6a6614
(Doc) Add section on what is stored in the DB + Add clear section on key/team based logging ( #6769 )
...
* fix clean up logging
* add Default Login, Logout URLs
* docs on tables in DB schema
* docs add section on What is stored in the DB
2024-11-16 10:28:44 -08:00
Emmanuel Ferdman
9cf3dcbbf3
Update routing references ( #6758 )
...
* Update routing references
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
* Update routing references
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
---------
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
2024-11-16 08:28:44 -08:00
paul-gauthier
73ccbc0f14
add openrouter/qwen/qwen-2.5-coder-32b-instruct ( #6731 )
2024-11-15 18:08:28 -08:00
Ishaan Jaff
41aade2cc0
(feat) Use litellm/ prefix when storing virtual keys in AWS secret manager ( #6765 )
...
* fix - storing AWS keys in secret manager
* fix test_key_generate_with_secret_manager_call
* allow using prefix_for_stored_virtual_keys
* add prefix_for_stored_virtual_keys
* test_key_generate_with_secret_manager_call
2024-11-15 18:07:43 -08:00
Ishaan Jaff
6d2ee70a9a
(fix) Fix - don't allow viewer roles to create virtual keys ( #6764 )
...
* fix ui route permissions
* fix test_is_ui_route_allowed
* fix test_is_ui_route_allowed
* test_user_role_permissions
2024-11-15 18:02:13 -08:00
Ishaan Jaff
9ba8f40bd1
(Feat) Add Vertex Model Garden llama 3.1 models ( #6763 )
...
* add VertexAIModelGardenModels
* VertexAIModelGardenModels
* test_vertexai_model_garden_model_completion
* docs model garden
2024-11-15 16:14:06 -08:00
Ishaan Jaff
0f7ea14992
feat - add us.llama 3.1 models ( #6760 )
2024-11-15 08:03:06 -08:00
Krish Dholakia
3beecfb0d4
LiteLLM Minor Fixes & Improvements (11/13/2024) ( #6729 )
...
* fix(utils.py): add logprobs support for together ai
Fixes
https://github.com/BerriAI/litellm/issues/6724
* feat(pass_through_endpoints/): add anthropic/ pass-through endpoint
adds new `anthropic/` pass-through endpoint + refactors docs
* feat(spend_management_endpoints.py): allow /global/spend/report to query team + customer id
enables seeing spend for a customer in a team
* Add integration with MLflow Tracing (#6147 )
* Add MLflow logger
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Streaming handling
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* lint
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* address comments and fix issues
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* address comments and fix issues
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Move logger construction code
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Add docs
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* async handlers
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* new picture
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* fix(mlflow.py): fix ruff linting errors
* ci(config.yml): add mlflow to ci testing
* fix: fix test
* test: fix test
* Litellm key update fix (#6710 )
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* fix(key_management_endpoints.py): fix /key/update with metadata update
* fix(key_management_endpoints.py): fix key_prepare_update helper
* fix(key_management_endpoints.py): reset value to none if set in key update
* fix: update test
'
* Litellm dev 11 11 2024 (#6693 )
* fix(__init__.py): add 'watsonx_text' as mapped llm api route
Fixes https://github.com/BerriAI/litellm/issues/6663
* fix(opentelemetry.py): fix passing parallel tool calls to otel
Fixes https://github.com/BerriAI/litellm/issues/6677
* refactor(test_opentelemetry_unit_tests.py): create a base set of unit tests for all logging integrations - test for parallel tool call handling
reduces bugs in repo
* fix(__init__.py): update provider-model mapping to include all known provider-model mappings
Fixes https://github.com/BerriAI/litellm/issues/6669
* feat(anthropic): support passing document in llm api call
* docs(anthropic.md): add pdf anthropic call to docs + expose new 'supports_pdf_input' function
* fix(factory.py): fix linting error
* add clear doc string for GCS bucket logging
* Add docs to export logs to Laminar (#6674 )
* Add docs to export logs to Laminar
* minor fix: newline at end of file
* place laminar after http and grpc
* (Feat) Add langsmith key based logging (#6682 )
* add langsmith_api_key to StandardCallbackDynamicParams
* create a file for langsmith types
* langsmith add key / team based logging
* add key based logging for langsmith
* fix langsmith key based logging
* fix linting langsmith
* remove NOQA violation
* add unit test coverage for all helpers in test langsmith
* test_langsmith_key_based_logging
* docs langsmith key based logging
* run langsmith tests in logging callback tests
* fix logging testing
* test_langsmith_key_based_logging
* test_add_callback_via_key_litellm_pre_call_utils_langsmith
* add debug statement langsmith key based logging
* test_langsmith_key_based_logging
* (fix) OpenAI's optional messages[].name does not work with Mistral API (#6701 )
* use helper for _transform_messages mistral
* add test_message_with_name to base LLMChat test
* fix linting
* add xAI on Admin UI (#6680 )
* (docs) add benchmarks on 1K RPS (#6704 )
* docs litellm proxy benchmarks
* docs GCS bucket
* doc fix - reduce clutter on logging doc title
* (feat) add cost tracking stable diffusion 3 on Bedrock (#6676 )
* add cost tracking for sd3
* test_image_generation_bedrock
* fix get model info for image cost
* add cost_calculator for stability 1 models
* add unit testing for bedrock image cost calc
* test_cost_calculator_with_no_optional_params
* add test_cost_calculator_basic
* correctly allow size Optional
* fix cost_calculator
* sd3 unit tests cost calc
* fix raise correct error 404 when /key/info is called on non-existent key (#6653 )
* fix raise correct error on /key/info
* add not_found_error error
* fix key not found in DB error
* use 1 helper for checking token hash
* fix error code on key info
* fix test key gen prisma
* test_generate_and_call_key_info
* test fix test_call_with_valid_model_using_all_models
* fix key info tests
* bump: version 1.52.4 → 1.52.5
* add defaults used for GCS logging
* LiteLLM Minor Fixes & Improvements (11/12/2024) (#6705 )
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* bump: version 1.52.5 → 1.52.6
* (feat) helm hook to sync db schema (#6715 )
* v0 migration job
* fix job
* fix migrations job.yml
* handle standalone DB on helm hook
* fix argo cd annotations
* fix db migration helm hook
* fix migration job
* doc fix Using Http/2 with Hypercorn
* (fix proxy redis) Add redis sentinel support (#6154 )
* add sentinel_password support
* add doc for setting redis sentinel password
* fix redis sentinel - use sentinel password
* Fix: Update gpt-4o costs to that of gpt-4o-2024-08-06 (#6714 )
Fixes #6713
* (fix) using Anthropic `response_format={"type": "json_object"}` (#6721 )
* add support for response_format=json anthropic
* add test_json_response_format to baseLLM ChatTest
* fix test_litellm_anthropic_prompt_caching_tools
* fix test_anthropic_function_call_with_no_schema
* test test_create_json_tool_call_for_response_format
* (feat) Add cost tracking for Azure Dall-e-3 Image Generation + use base class to ensure basic image generation tests pass (#6716 )
* add BaseImageGenTest
* use 1 class for unit testing
* add debugging to BaseImageGenTest
* TestAzureOpenAIDalle3
* fix response_cost_calculator
* test_basic_image_generation
* fix img gen basic test
* fix _select_model_name_for_cost_calc
* fix test_aimage_generation_bedrock_with_optional_params
* fix undo changes cost tracking
* fix response_cost_calculator
* fix test_cost_azure_gpt_35
* fix remove dup test (#6718 )
* (build) update db helm hook
* (build) helm db pre sync hook
* (build) helm db sync hook
* test: run test_team_logging firdst
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
* test: update test
* test: skip anthropic overloaded error
* test: cleanup test
* test: update tests
* test: fix test
* test: handle gemini overloaded model error
* test: handle internal server error
* test: handle anthropic overloaded error
* test: handle claude instability
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
Co-authored-by: Yuki Watanabe <31463517+B-Step62@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
2024-11-15 11:18:31 +05:30
Ishaan Jaff
3f8a9167ae
bump: version 1.52.8 → 1.52.9
2024-11-14 20:38:32 -08:00
Ishaan Jaff
c119bad5f9
(feat) Vertex AI - add support for fine tuned embedding models ( #6749 )
...
* fix use fine tuned vertex embedding models
* test_vertex_embedding_url
* add _transform_openai_request_to_fine_tuned_embedding_request
* add _transform_openai_request_to_fine_tuned_embedding_request
* add transform_openai_request_to_vertex_embedding_request
* add _transform_vertex_response_to_openai_for_fine_tuned_models
* test_vertexai_embedding for ft models
* fix test_vertexai_embedding_finetuned
* doc fine tuned / custom embedding models
* fix test test_partner_models_httpx
2024-11-14 20:37:55 -08:00
Ishaan Jaff
c03351328f
fix imagegeneration output_cost_per_image on model cost map ( #6752 )
2024-11-14 20:37:21 -08:00
Rasswanth
0585fd56d6
fix: import audio check ( #6740 )
2024-11-14 17:00:38 -08:00
Ishaan Jaff
6ae0bc4a11
[Feature]: json_schema in response support for Anthropic ( #6748 )
...
* _convert_tool_response_to_message
* fix ModelResponseIterator
* fix test_json_response_format
* test_json_response_format_stream
* fix _convert_tool_response_to_message
* use helper _handle_json_mode_chunk
* fix _process_response
* unit testing for test_convert_tool_response_to_message_no_arguments
* update doc for JSON mode
2024-11-14 16:59:45 -08:00
Ishaan Jaff
a70a0688d8
[Feature]: Stop swallowing up AzureOpenAi exception responses in litellm's implementation for a BadRequestError ( #6745 )
...
* fix azure exceptions
* test_bad_request_error_contains_httpx_response
* test_bad_request_error_contains_httpx_response
* use safe access to get exception response
* fix get attr
2024-11-14 15:54:28 -08:00
Ishaan Jaff
7959dc9db3
(feat) add bedrock/stability.stable-image-ultra-v1:0 ( #6723 )
...
* add stability.stable-image-ultra-v1:0
* add pricing for stability.stable-image-ultra-v1:0
* fix test_supports_response_schema
* ci/cd run again
2024-11-14 14:47:15 -08:00
Krrish Dholakia
fc685c1f74
docs(logging.md): add 'trace_id' param to standard logging payload
2024-11-15 02:01:37 +05:30
Krrish Dholakia
9593fbe5c3
docs(reliability.md): add tutorial on disabling fallbacks per key
2024-11-15 01:49:17 +05:30
Krrish Dholakia
499780eff2
docs: add docs on jina ai rerank support
2024-11-15 01:45:57 +05:30
Krrish Dholakia
89678ace00
bump: version 1.52.7 → 1.52.8
2024-11-15 01:03:49 +05:30
Krish Dholakia
e9aa492af3
LiteLLM Minor Fixes & Improvement (11/14/2024) ( #6730 )
...
* fix(ollama.py): fix get model info request
Fixes https://github.com/BerriAI/litellm/issues/6703
* feat(anthropic/chat/transformation.py): support passing user id to anthropic via openai 'user' param
* docs(anthropic.md): document all supported openai params for anthropic
* test: fix tests
* fix: fix tests
* feat(jina_ai/): add rerank support
Closes https://github.com/BerriAI/litellm/issues/6691
* test: handle service unavailable error
* fix(handler.py): refactor together ai rerank call
* test: update test to handle overloaded error
* test: fix test
* Litellm router trace (#6742 )
* feat(router.py): add trace_id to parent functions - allows tracking retry/fallbacks
* feat(router.py): log trace id across retry/fallback logic
allows grouping llm logs for the same request
* test: fix tests
* fix: fix test
* fix(transformation.py): only set non-none stop_sequences
* Litellm router disable fallbacks (#6743 )
* bump: version 1.52.6 → 1.52.7
* feat(router.py): enable dynamically disabling fallbacks
Allows for enabling/disabling fallbacks per key
* feat(litellm_pre_call_utils.py): support setting 'disable_fallbacks' on litellm key
* test: fix test
* fix(exception_mapping_utils.py): map 'model is overloaded' to internal server error
* test: handle gemini error
* test: fix test
* fix: new run
2024-11-15 01:02:54 +05:30
Ishaan Jaff
f8e700064e
(Feat) Add support for storing virtual keys in AWS SecretManager ( #6728 )
...
* add SecretManager to httpxSpecialProvider
* fix importing AWSSecretsManagerV2
* add unit testing for writing keys to AWS secret manager
* use KeyManagementEventHooks for key/generated events
* us event hooks for key management endpoints
* working AWSSecretsManagerV2
* fix write secret to AWS secret manager on /key/generate
* fix KeyManagementSettings
* use tasks for key management hooks
* add async_delete_secret
* add test for async_delete_secret
* use _delete_virtual_keys_from_secret_manager
* fix test secret manager
* test_key_generate_with_secret_manager_call
* fix check for key_management_settings
* sync_read_secret
* test_aws_secret_manager
* fix sync_read_secret
* use helper to check when _should_read_secret_from_secret_manager
* test_get_secret_with_access_mode
* test - handle eol model claude-2, use claude-2.1 instead
* docs AWS secret manager
* fix test_read_nonexistent_secret
* fix test_supports_response_schema
* ci/cd run again
2024-11-14 09:25:07 -08:00
Ishaan Jaff
da84056e59
mark Helm PreSyn as BETA
2024-11-13 22:18:12 -08:00
Ishaan Jaff
387c70c989
fix test_supports_response_schema
2024-11-13 21:59:24 -08:00
Camden Clark
b582efa3ce
Update prefix.md ( #6734 )
2024-11-14 11:18:35 +05:30
Jongseob Jeon
f3914c87d3
Update code blocks huggingface.md ( #6737 )
2024-11-14 11:17:57 +05:30
Ishaan Jaff
310669e3bc
(docs) add instructions on how to contribute to docker image
2024-11-13 20:52:17 -08:00
Ishaan Jaff
914cec3ab5
test - handle eol model claude-2, use claude-2.1 instead
2024-11-13 19:37:34 -08:00
Ishaan Jaff
f2e6025c65
fix prisma migration
2024-11-13 17:04:58 -08:00
