Ishaan Jaff
617ac63d14
(feat) add infinity rerank models ( #7321 )
...
* Support Infinity Reranker (custom reranking models) (#7247 )
* Support Infinity Reranker
* Clean code
* Included transformation.py
* Clean code
* Added Infinity reranker test
* Clean code
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* transform_rerank_response
* update handler.py
* infinity rerank updates
* ci/cd run again
* add infinity unit tests
* docs add instruction on how to add a new provider for rerank
---------
Co-authored-by: Hao Shan <53949959+haoshan98@users.noreply.github.com>
2024-12-19 18:30:28 -08:00
Ishaan Jaff
5f15b0aa20
(code refactor) - Add BaseRerankConfig. Use BaseRerankConfig for cohere/rerank and azure_ai/rerank ( #7319 )
...
* add base rerank config
* working sync cohere rerank
* update rerank types
* update base rerank config
* remove old rerank
* add new cohere handler.py
* add cohere rerank transform
* add get_provider_rerank_config
* add rerank to base llm http handler
* add rerank utils
* add arerank to llm http handler.py
* add AzureAIRerankConfig
* updates rerank config
* update test rerank
* fix unused imports
* update get_provider_rerank_config
* test_basic_rerank_caching
* fix unused import
* test rerank
2024-12-19 17:03:34 -08:00
Ishaan Jaff
c7f14e936a
(code quality) run ruff rule to ban unused imports ( #7313 )
...
* remove unused imports
* fix AmazonConverseConfig
* fix test
* fix import
* ruff check fixes
* test fixes
* fix testing
* fix imports
2024-12-19 12:33:42 -08:00
Krish Dholakia
6a45ee1ef7
fix(hosted_vllm/transformation.py): return fake api key, if none give… ( #7301 )
...
* fix(hosted_vllm/transformation.py): return fake api key, if none give. Prevents httpx error
Fixes https://github.com/BerriAI/litellm/issues/7291
* test: fix test
* fix(main.py): add hosted_vllm/ support for embeddings endpoint
Closes https://github.com/BerriAI/litellm/issues/7290
* docs(vllm.md): add docs on vllm embeddings usage
* fix(__init__.py): fix sambanova model test
* fix(base_llm_unit_tests.py): skip pydantic obj test if model takes >5s to respond
2024-12-18 18:41:53 -08:00
Krish Dholakia
2f08341a08
Litellm dev readd prompt caching ( #7299 )
...
* fix(router.py): re-add saving model id on prompt caching valid successful deployment
* fix(router.py): introduce optional pre_call_checks
isolate prompt caching logic in a separate file
* fix(prompt_caching_deployment_check.py): fix import
* fix(router.py): new 'async_filter_deployments' event hook
allows custom logger to filter deployments returned to routing strategy
* feat(prompt_caching_deployment_check.py): initial working commit of prompt caching based routing
* fix(cooldown_callbacks.py): fix linting error
* fix(budget_limiter.py): move budget logger to async_filter_deployment hook
* test: add unit test
* test(test_router_helper_utils.py): add unit testing
* fix(budget_limiter.py): fix linting errors
* docs(config_settings.md): add 'optional_pre_call_checks' to router_settings param docs
2024-12-18 15:13:49 -08:00
Ishaan Jaff
7a5dd29fe0
(fix) unable to pass input_type parameter to Voyage AI embedding mode ( #7276 )
...
Read Version from pyproject.toml / read-version (push) Successful in 46s
* VoyageEmbeddingConfig
* fix voyage logic to get params
* add voyage embedding transformation
* add get_provider_embedding_config
* use BaseEmbeddingConfig
* voyage clean up
* use llm http handler for embedding transformations
* test_voyage_ai_embedding_extra_params
* add voyage async
* test_voyage_ai_embedding_extra_params
* add async for llm http handler
* update BaseLLMEmbeddingTest
* test_voyage_ai_embedding_extra_params
* fix linting
* fix get_provider_embedding_config
* fix anthropic text test
* update location of base/chat/transformation
* fix import path
* fix IBMWatsonXAIConfig
2024-12-17 19:23:49 -08:00
Ishaan Jaff
3c984ed60e
(feat) Add Azure Blob Storage Logging Integration ( #7265 )
...
* add path to http handler
* AzureBlobStorageLogger
* test_azure_blob_storage
* use constants for Azure storage
* use helper get_azure_ad_token_from_entrata_id
* azure blob storage support
* get_azure_ad_token_from_azure_storage
* fix import
* azure logging
* docs azure storage
* add docs on azure blobs
* add premium user check
* add azure_storage as identified logging callback
* async_upload_payload_to_azure_blob_storage
* docs azure storage
* callback_class_str_to_classType
2024-12-16 22:18:22 -08:00
Ishaan Jaff
2a92a60168
ci/cd run again
2024-12-16 08:19:14 -08:00
Ishaan Jaff
7103198805
(feat) Add Tag-based budgets on litellm router / proxy ( #7236 )
...
Read Version from pyproject.toml / read-version (push) Successful in 46s
* add BudgetConfig
* add _get_tags_from_request_kwargs
* test_tag_budgets_e2e_test_expect_to_fail
* add a check for request tags
* fix _async_get_cache_keys_for_router_budget_limiting
* fix test
* fix _sync_in_memory_spend_with_redis
* _async_get_cache_keys_for_router_budget_limiting
* fix _init_tag_budgets
* fix type casting
* docs show error for tag budget limit hit
* fix _get_tags_from_request_kwargs
* fix undo change
2024-12-14 17:28:36 -08:00
Krish Dholakia
516c2a6a70
Litellm remove circular imports ( #7232 )
...
* fix(utils.py): initial commit to remove circular imports - moves llmproviders to utils.py
* fix(router.py): fix 'litellm.EmbeddingResponse' import from router.py
'
* refactor: fix litellm.ModelResponse import on pass through endpoints
* refactor(litellm_logging.py): fix circular import for custom callbacks literal
* fix(factory.py): fix circular imports inside prompt factory
* fix(cost_calculator.py): fix circular import for 'litellm.Usage'
* fix(proxy_server.py): fix potential circular import with `litellm.Router'
* fix(proxy/utils.py): fix potential circular import in `litellm.Router`
* fix: remove circular imports in 'auth_checks' and 'guardrails/'
* fix(prompt_injection_detection.py): fix router impor t
* fix(vertex_passthrough_logging_handler.py): fix potential circular imports in vertex pass through
* fix(anthropic_pass_through_logging_handler.py): fix potential circular imports
* fix(slack_alerting.py-+-ollama_chat.py): fix modelresponse import
* fix(base.py): fix potential circular import
* fix(handler.py): fix potential circular ref in codestral + cohere handler's
* fix(azure.py): fix potential circular imports
* fix(gpt_transformation.py): fix modelresponse import
* fix(litellm_logging.py): add logging base class - simplify typing
makes it easy for other files to type check the logging obj without introducing circular imports
* fix(azure_ai/embed): fix potential circular import on handler.py
* fix(databricks/): fix potential circular imports in databricks/
* fix(vertex_ai/): fix potential circular imports on vertex ai embeddings
* fix(vertex_ai/image_gen): fix import
* fix(watsonx-+-bedrock): cleanup imports
* refactor(anthropic-pass-through-+-petals): cleanup imports
* refactor(huggingface/): cleanup imports
* fix(ollama-+-clarifai): cleanup circular imports
* fix(openai_like/): fix impor t
* fix(openai_like/): fix embedding handler
cleanup imports
* refactor(openai.py): cleanup imports
* fix(sagemaker/transformation.py): fix import
* ci(config.yml): add circular import test to ci/cd
2024-12-14 16:28:34 -08:00
Krish Dholakia
e68bb4e051
Litellm dev 12 12 2024 ( #7203 )
...
Read Version from pyproject.toml / read-version (push) Successful in 47s
* fix(azure/): support passing headers to azure openai endpoints
Fixes https://github.com/BerriAI/litellm/issues/6217
* fix(utils.py): move default tokenizer to just openai
hf tokenizer makes network calls when trying to get the tokenizer - this slows down execution time calls
* fix(router.py): fix pattern matching router - add generic "*" to it as well
Fixes issue where generic "*" model access group wouldn't show up
* fix(pattern_match_deployments.py): match to more specific pattern
match to more specific pattern
allows setting generic wildcard model access group and excluding specific models more easily
* fix(proxy_server.py): fix _delete_deployment to handle base case where db_model list is empty
don't delete all router models b/c of empty list
Fixes https://github.com/BerriAI/litellm/issues/7196
* fix(anthropic/): fix handling response_format for anthropic messages with anthropic api
* fix(fireworks_ai/): support passing response_format + tool call in same message
Addresses https://github.com/BerriAI/litellm/issues/7135
* Revert "fix(fireworks_ai/): support passing response_format + tool call in same message"
This reverts commit 6a30dc6929
2024-12-13 08:54:03 -08:00
Ishaan Jaff
90f9aded9f
ci/cd run release pipeline
2024-12-12 10:48:47 -08:00
Krish Dholakia
481645e49c
fix(acompletion): support fallbacks on acompletion ( #7184 )
...
Read Version from pyproject.toml / read-version (push) Successful in 45s
* fix(acompletion): support fallbacks on acompletion
allows health checks for wildcard routes to use fallback models
* test: update cohere generate api testing
* add max tokens to health check (#7000 )
* fix: fix health check test
* test: update testing
---------
Co-authored-by: Cameron <561860+wallies@users.noreply.github.com>
2024-12-11 19:20:54 -08:00
Krrish Dholakia
02dd0c6e7e
build: Squashed commit of https://github.com/BerriAI/litellm/pull/7171
...
Closes https://github.com/BerriAI/litellm/pull/7171
2024-12-11 01:10:12 -08:00
Krrish Dholakia
06074bb13b
build: Squashed commit of https://github.com/BerriAI/litellm/pull/7170
...
Closes https://github.com/BerriAI/litellm/pull/7170
2024-12-11 01:03:57 -08:00
Krrish Dholakia
b9b34a7b99
build: Squashed commit of https://github.com/BerriAI/litellm/pull/7165
...
Closes https://github.com/BerriAI/litellm/pull/7165
2024-12-11 01:00:33 -08:00
Ishaan Jaff
78d132c1fb
(Refactor) Code Quality improvement - rename text_completion_codestral.py -> codestral/completion/ ( #7172 )
...
* rename files
* fix codestral fim organization
* fix CodestralTextCompletionConfig
* fix import CodestralTextCompletion
* fix BaseLLM
* fix imports
* fix CodestralTextCompletionConfig
* fix imports CodestralTextCompletion
2024-12-11 00:55:47 -08:00
Ishaan Jaff
400eb28a91
Code Quality Improvement - move aleph_alpha to deprecated_providers ( #7168 )
...
* move aleph alpha to deprecated providers
* fix import location
* fix aleph_alpha
* pytest skip
* undo change to test file
2024-12-11 00:50:40 -08:00
Ishaan Jaff
21003c4337
Code Quality Improvement - use vertex_ai/ as folder name for vertexAI ( #7166 )
...
* fix rename vertex ai
* run ci/cd again
2024-12-11 00:32:41 -08:00
Krish Dholakia
350cfc36f7
Litellm merge pr ( #7161 )
...
* build: merge branch
* test: fix openai naming
* fix(main.py): fix openai renaming
* style: ignore function length for config factory
* fix(sagemaker/): fix routing logic
* fix: fix imports
* fix: fix override
2024-12-10 22:49:26 -08:00
Krish Dholakia
d5aae81c6d
Litellm vllm refactor ( #7158 )
...
* refactor(vllm/): move vllm to use base llm config
* test: mark flaky test
2024-12-10 21:48:35 -08:00
Krish Dholakia
405080396d
Litellm ollama refactor ( #7162 )
...
* refactor(ollama/): refactor ollama `/api/generate` to use base llm config
Addresses https://github.com/andrewyng/aisuite/issues/113#issuecomment-2512369132
* test: skip unresponsive test
* test(test_secret_manager.py): mark flaky test
* test: fix google sm test
* fix: fix init.py
2024-12-10 21:45:35 -08:00
Krish Dholakia
488913c69f
Revert "LiteLLM Common Base LLM Config (pt.4): Move Ollama to Base LLM Config…" ( #7160 )
...
This reverts commit 40a22eb4c6
2024-12-10 21:44:54 -08:00
Krish Dholakia
40a22eb4c6
LiteLLM Common Base LLM Config (pt.4): Move Ollama to Base LLM Config ( #7157 )
...
* refactor(ollama/): refactor ollama `/api/generate` to use base llm config
Addresses https://github.com/andrewyng/aisuite/issues/113#issuecomment-2512369132
* test: skip unresponsive test
* test(test_secret_manager.py): mark flaky test
* test: fix google sm test
2024-12-10 21:39:28 -08:00
Ishaan Jaff
bfb6891eb7
rename llms/OpenAI/ -> llms/openai/ ( #7154 )
...
* rename OpenAI -> openai
* fix file rename
* fix rename changes
* fix organization of openai/transcription
* fix import OA fine tuning API
* fix openai ft handler
* fix handler import
2024-12-10 20:14:07 -08:00
Krish Dholakia
e903fe6038
refactor(sagemaker/): separate chat + completion routes + make them b… ( #7151 )
...
* refactor(sagemaker/): separate chat + completion routes + make them both use base llm config
Addresses https://github.com/andrewyng/aisuite/issues/113#issuecomment-2512369132
* fix(main.py): pass hf model name + custom prompt dict to litellm params
2024-12-10 19:40:05 -08:00
Krish Dholakia
1e87782215
LiteLLM Common Base LLM Config (pt.3): Move all OAI compatible providers to base llm config ( #7148 )
...
Read Version from pyproject.toml / read-version (push) Successful in 45s
* refactor(fireworks_ai/): inherit from openai like base config
refactors fireworks ai to use a common config
* test: fix import in test
* refactor(watsonx/): refactor watsonx to use llm base config
refactors chat + completion routes to base config path
* fix: fix linting error
* refactor: inherit base llm config for oai compatible routes
* test: fix test
* test: fix test
2024-12-10 17:12:42 -08:00
Krish Dholakia
311432ca17
refactor(fireworks_ai/): inherit from openai like base config ( #7146 )
...
* refactor(fireworks_ai/): inherit from openai like base config
refactors fireworks ai to use a common config
* test: fix import in test
* refactor(watsonx/): refactor watsonx to use llm base config
refactors chat + completion routes to base config path
* fix: fix linting error
* test: fix test
* fix: fix test
2024-12-10 16:15:19 -08:00
Ishaan Jaff
bdb20821ea
(Refactor) Code Quality improvement - Use Common base handler for anthropic_text/ ( #7143 )
...
* add anthropic text provider
* add ANTHROPIC_TEXT to LlmProviders
* fix anthropic text implementation
* working anthropic text claude-2
* test_acompletion_claude2_stream
* add param mapping for anthropic text
* fix unused imports
* fix anthropic completion handler.py
2024-12-10 12:23:58 -08:00
Ishaan Jaff
bd39e1ab5d
(Refactor) Code Quality improvement - Use Common base handler for cloudflare/ provider ( #7127 )
...
* add get_complete_url to base config
* cloudflare - refactor to following existing pattern
* migrate cloudflare chat completions to base llm http handler
* fix unused import
* fix fake stream in cloudflare
* fix cloudflare transformation
* fix naming for BaseModelResponseIterator
* add async cloudflare streaming test
* test cloudflare
* add handler.py
* add handler.py in cohere handler.py
2024-12-10 10:12:22 -08:00
Krish Dholakia
5bbf906c83
Litellm code qa common config ( #7113 )
...
Read Version from pyproject.toml / read-version (push) Successful in 44s
* feat(base_llm): initial commit for common base config class
Addresses code qa critique https://github.com/andrewyng/aisuite/issues/113#issuecomment-2512369132
* feat(base_llm/): add transform request/response abstract methods to base config class
* feat(cohere-+-clarifai): refactor integrations to use common base config class
* fix: fix linting errors
* refactor(anthropic/): move anthropic + vertex anthropic to use base config
* test: fix xai test
* test: fix tests
* fix: fix linting errors
* test: comment out WIP test
* fix(transformation.py): fix is pdf used check
* fix: fix linting error
2024-12-09 15:58:25 -08:00
Krish Dholakia
0c0498dd60
Litellm dev 12 07 2024 ( #7086 )
...
Read Version from pyproject.toml / read-version (push) Successful in 11s
* fix(main.py): support passing max retries to azure/openai embedding integrations
Fixes https://github.com/BerriAI/litellm/issues/7003
* feat(team_endpoints.py): allow updating team model aliases
Closes https://github.com/BerriAI/litellm/issues/6956
* feat(router.py): allow specifying model id as fallback - skips any cooldown check
Allows a default model to be checked if all models in cooldown
s/o @micahjsmith
* docs(reliability.md): add fallback to specific model to docs
* fix(utils.py): new 'is_prompt_caching_valid_prompt' helper util
Allows user to identify if messages/tools have prompt caching
Related issue: https://github.com/BerriAI/litellm/issues/6784
* feat(router.py): store model id for prompt caching valid prompt
Allows routing to that model id on subsequent requests
* fix(router.py): only cache if prompt is valid prompt caching prompt
prevents storing unnecessary items in cache
* feat(router.py): support routing prompt caching enabled models to previous deployments
Closes https://github.com/BerriAI/litellm/issues/6784
* test: fix linting errors
* feat(databricks/): convert basemodel to dict and exclude none values
allow passing pydantic message to databricks
* fix(utils.py): ensure all chat completion messages are dict
* (feat) Track `custom_llm_provider` in LiteLLMSpendLogs (#7081 )
* add custom_llm_provider to SpendLogsPayload
* add custom_llm_provider to SpendLogs
* add custom llm provider to SpendLogs payload
* test_spend_logs_payload
* Add MLflow to the side bar (#7031 )
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* (bug fix) SpendLogs update DB catch all possible DB errors for retrying (#7082 )
* catch DB_CONNECTION_ERROR_TYPES
* fix DB retry mechanism for SpendLog updates
* use DB_CONNECTION_ERROR_TYPES in auth checks
* fix exp back off for writing SpendLogs
* use _raise_failed_update_spend_exception to ensure errors print as NON blocking
* test_update_spend_logs_multiple_batches_with_failure
* (Feat) Add StructuredOutputs support for Fireworks.AI (#7085 )
* fix model cost map fireworks ai "supports_response_schema": true,
* fix supports_response_schema
* fix map openai params fireworks ai
* test_map_response_format
* test_map_response_format
* added deepinfra/Meta-Llama-3.1-405B-Instruct (#7084 )
* bump: version 1.53.9 → 1.54.0
* fix deepinfra
* litellm db fixes LiteLLM_UserTable (#7089 )
* ci/cd queue new release
* fix llama-3.3-70b-versatile
* refactor - use consistent file naming convention `AI21/` -> `ai21` (#7090 )
* fix refactor - use consistent file naming convention
* ci/cd run again
* fix naming structure
* fix use consistent naming (#7092 )
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Yuki Watanabe <31463517+B-Step62@users.noreply.github.com>
Co-authored-by: ali sayyah <ali.sayyah2@gmail.com>
2024-12-08 00:30:33 -08:00
Ishaan Jaff
36e99ebce7
fix use consistent naming ( #7092 )
Read Version from pyproject.toml / read-version (push) Successful in 11s
2024-12-07 22:01:00 -08:00
Ishaan Jaff
7d4a1cb4e2
refactor - use consistent file naming convention AI21/ -> ai21 ( #7090 )
...
* fix refactor - use consistent file naming convention
* ci/cd run again
* fix naming structure
2024-12-07 21:46:34 -08:00
Ishaan Jaff
191a0fefbc
ci/cd queue new release
2024-12-07 19:09:57 -08:00
Krish Dholakia
19a4273fda
feat(langfuse/): support langfuse prompt management ( #7073 )
...
* feat(langfuse/): support langfuse prompt management
Initial working commit for langfuse prompt management support
Closes https://github.com/BerriAI/litellm/issues/6269
* test: update test
* fix(litellm_logging.py): suppress linting error
2024-12-06 23:10:22 -08:00
Krish Dholakia
816f0ef8d2
LiteLLM Minor Fixes & Improvements (12/05/2024) ( #7051 )
...
* fix(cost_calculator.py): move to using `.get_model_info()` for cost per token calculations
ensures cost tracking is reliable - handles edge cases of parsing model cost map
* build(model_prices_and_context_window.json): add 'supports_response_schema' for select tgai models
Fixes https://github.com/BerriAI/litellm/pull/7037#discussion_r1872157329
* build(model_prices_and_context_window.json): remove 'pdf input' and 'vision' support from nova micro in model map
Bedrock docs indicate no support for micro - https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html
* fix(converse_transformation.py): support amazon nova tool use
* fix(opentelemetry): Add missing LLM request type attribute to spans (#7041 )
* feat(opentelemetry): add LLM request type attribute to spans
* lint
* fix: curl usage (#7038 )
curl -d, --data <data> is lowercase d
curl -D, --dump-header <filename> is uppercase D
references:
https://curl.se/docs/manpage.html#-d
https://curl.se/docs/manpage.html#-D
* fix(spend_tracking.py): handle empty 'id' in model response - when creating spend log
Fixes https://github.com/BerriAI/litellm/issues/7023
* fix(streaming_chunk_builder.py): handle initial id being empty string
Fixes https://github.com/BerriAI/litellm/issues/7023
* fix(anthropic_passthrough_logging_handler.py): add end user cost tracking for anthropic pass through endpoint
* docs(pass_through/): refactor docs location + add table on supported features for pass through endpoints
* feat(anthropic_passthrough_logging_handler.py): support end user cost tracking via anthropic sdk
* docs(anthropic_completion.md): add docs on passing end user param for cost tracking on anthropic sdk
* fix(litellm_logging.py): use standard logging payload if present in kwargs
prevent datadog logging error for pass through endpoints
* docs(bedrock.md): add rerank api usage example to docs
* bugfix/change dummy tool name format (#7053 )
* fix viewing keys (#7042 )
* ui new build
* build(model_prices_and_context_window.json): add bedrock region models to model cost map (#7044 )
* bye (#6982 )
* (fix) litellm router.aspeech (#6962 )
* doc Migrating Databases
* fix aspeech on router
* test_audio_speech_router
* test_audio_speech_router
* docs show supported providers on batches api doc
* change dummy tool name format
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: yujonglee <yujonglee.dev@gmail.com>
* fix: fix linting errors
* test: update test
* fix(litellm_logging.py): fix pass through check
* fix(test_otel_logging.py): fix test
* fix(cost_calculator.py): update handling for cost per second
* fix(cost_calculator.py): fix cost check
* test: fix test
* (fix) adding public routes when using custom header (#7045 )
* get_api_key_from_custom_header
* add test_get_api_key_from_custom_header
* fix testing use 1 file for test user api key auth
* fix test user api key auth
* test_custom_api_key_header_name
* build: update ui build
---------
Co-authored-by: Doron Kopit <83537683+doronkopit5@users.noreply.github.com>
Co-authored-by: lloydchang <lloydchang@gmail.com>
Co-authored-by: hgulersen <haymigulersen@gmail.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: yujonglee <yujonglee.dev@gmail.com>
2024-12-06 14:29:53 -08:00
Krish Dholakia
6bb934c0ac
fix(key_management_endpoints.py): override metadata field value on up… ( #7008 )
...
* fix(key_management_endpoints.py): override metadata field value on update
allow user to override tags
* feat(__init__.py): expose new disable_end_user_cost_tracking_prometheus_only metric
allow disabling end user cost tracking on prometheus - fixes cardinality issue
* fix(litellm_pre_call_utils.py): add key/team level enforced params
Fixes https://github.com/BerriAI/litellm/issues/6652
* fix(key_management_endpoints.py): allow user to pass in `enforced_params` as a top level param on /key/generate and /key/update
* docs(enterprise.md): add docs on enforcing required params for llm requests
* Add support of Galadriel API (#7005 )
* fix(router.py): robust retry after handling
set retry after time to 0 if >0 healthy deployments. handle base case = 1 deployment
* test(test_router.py): fix test
* feat(bedrock/): add support for 'nova' models
also adds explicit 'converse/' route for simpler routing
* fix: fix 'supports_pdf_input'
return if model supports pdf input on get_model_info
* feat(converse_transformation.py): support bedrock pdf input
* docs(document_understanding.md): add document understanding to docs
* fix(litellm_pre_call_utils.py): fix linting error
* fix(init.py): fix passing of bedrock converse models
* feat(bedrock/converse): support 'response_format={"type": "json_object"}'
* fix(converse_handler.py): fix linting error
* fix(base_llm_unit_tests.py): fix test
* fix: fix test
* test: fix test
* test: fix test
* test: remove duplicate test
---------
Co-authored-by: h4n0 <4738254+h4n0@users.noreply.github.com>
2024-12-03 23:03:50 -08:00
Ishaan Jaff
d558b643be
queue new release
Read Version from pyproject.toml / read-version (push) Successful in 45s
2024-12-03 20:54:25 -08:00
Ishaan Jaff
ee16b36cba
ci/cd run release pipeline
2024-12-03 20:40:32 -08:00
Ishaan Jaff
573377ea49
ci/cd queue release
2024-12-03 12:09:28 -08:00
Krrish Dholakia
0caf804f4c
feat(databricks/chat): support structured outputs on databricks
...
Closes https://github.com/BerriAI/litellm/pull/6978
- handles content as list for dbrx, - handles streaming+response_format for dbrx
2024-12-02 23:08:19 -08:00
Krish Dholakia
1c8438d475
Litellm dev 11 30 2024 ( #6974 )
...
Read Version from pyproject.toml / read-version (push) Successful in 11s
* feat(cohere/chat.py): return citations in model response
Closes https://github.com/BerriAI/litellm/issues/6814
* fix(cohere/chat.py): fix linting errors
* fix(langsmith.py): support 'run_id' for langsmith
Fixes https://github.com/BerriAI/litellm/issues/6862
* fix(langsmith.py): fix langsmith quickstart
Fixes https://github.com/BerriAI/litellm/issues/6861
* fix: suppress linting error
* LiteLLM Minor Fixes & Improvements (11/29/2024) (#6965 )
* fix(factory.py): ensure tool call converts image url
Fixes https://github.com/BerriAI/litellm/issues/6953
* fix(transformation.py): support mp4 + pdf url's for vertex ai
Fixes https://github.com/BerriAI/litellm/issues/6936
* fix(http_handler.py): mask gemini api key in error logs
Fixes https://github.com/BerriAI/litellm/issues/6963
* docs(prometheus.md): update prometheus FAQs
* feat(auth_checks.py): ensure specific model access > wildcard model access
if wildcard model is in access group, but specific model is not - deny access
* fix(auth_checks.py): handle auth checks for team based model access groups
handles scenario where model access group used for wildcard models
* fix(internal_user_endpoints.py): support adding guardrails on `/user/update`
Fixes https://github.com/BerriAI/litellm/issues/6942
* fix(key_management_endpoints.py): fix prepare_metadata_fields helper
* fix: fix tests
* build(requirements.txt): bump openai dep version
fixes proxies argument
* test: fix tests
* fix(http_handler.py): fix error message masking
* fix(bedrock_guardrails.py): pass in prepped data
* test: fix test
* test: fix nvidia nim test
* fix(http_handler.py): return original response headers
* fix: revert maskedhttpstatuserror
* test: update tests
* test: cleanup test
* fix(key_management_endpoints.py): fix metadata field update logic
* fix(key_management_endpoints.py): maintain initial order of guardrails in key update
* fix(key_management_endpoints.py): handle prepare metadata
* fix: fix linting errors
* fix: fix linting errors
* fix: fix linting errors
* fix: fix key management errors
* fix(key_management_endpoints.py): update metadata
* test: update test
* refactor: add more debug statements
* test: skip flaky test
* test: fix test
* fix: fix test
* fix: fix update metadata logic
* fix: fix test
* ci(config.yml): change db url for e2e ui testing
* test: add more debug logs to langsmith
* fix: test change
* build(config.yml): fix db url
'
2024-12-02 21:03:33 -08:00
Ishaan Jaff
aea68cbeb6
(feat) DataDog Logger - Add Failure logging + use Standard Logging payload ( #6929 )
...
* add async_log_failure_event for dd
* use standard logging payload for DD logging
* use standard logging payload for DD
* fix use SLP status
* allow opting into _create_v0_logging_payload
* add unit tests for DD logging payload
* fix dd logging tests
2024-11-26 19:27:06 -08:00
Krish Dholakia
7e9d8b58f6
LiteLLM Minor Fixes & Improvements (11/23/2024) ( #6870 )
...
* feat(pass_through_endpoints/): support logging anthropic/gemini pass through calls to langfuse/s3/etc.
* fix(utils.py): allow disabling end user cost tracking with new param
Allows proxy admin to disable cost tracking for end user - keeps prometheus metrics small
* docs(configs.md): add disable_end_user_cost_tracking reference to docs
* feat(key_management_endpoints.py): add support for restricting access to `/key/generate` by team/proxy level role
Enables admin to restrict key creation, and assign team admins to handle distributing keys
* test(test_key_management.py): add unit testing for personal / team key restriction checks
* docs: add docs on restricting key creation
* docs(finetuned_models.md): add new guide on calling finetuned models
* docs(input.md): cleanup anthropic supported params
Closes https://github.com/BerriAI/litellm/issues/6856
* test(test_embedding.py): add test for passing extra headers via embedding
* feat(cohere/embed): pass client to async embedding
* feat(rerank.py): add `/v1/rerank` if missing for cohere base url
Closes https://github.com/BerriAI/litellm/issues/6844
* fix(main.py): pass extra_headers param to openai
Fixes https://github.com/BerriAI/litellm/issues/6836
* fix(litellm_logging.py): don't disable global callbacks when dynamic callbacks are set
Fixes issue where global callbacks - e.g. prometheus were overriden when langfuse was set dynamically
* fix(handler.py): fix linting error
* fix: fix typing
* build: add conftest to proxy_admin_ui_tests/
* test: fix test
* fix: fix linting errors
* test: fix test
* fix: fix pass through testing
2024-11-23 15:17:40 +05:30
Ishaan Jaff
920f4c9f82
(fix) add linting check to ban creating AsyncHTTPHandler during LLM calling ( #6855 )
...
* fix triton
* fix TEXT_COMPLETION_CODESTRAL
* fix REPLICATE
* fix CLARIFAI
* fix HUGGINGFACE
* add test_no_async_http_handler_usage
* fix PREDIBASE
* fix anthropic use get_async_httpx_client
* fix vertex fine tuning
* fix dbricks get_async_httpx_client
* fix get_async_httpx_client vertex
* fix get_async_httpx_client
* fix get_async_httpx_client
* fix make_async_azure_httpx_request
* fix check_for_async_http_handler
* test: cleanup mistral model
* add check for AsyncClient
* fix check_for_async_http_handler
* fix get_async_httpx_client
* fix tests using in_memory_llm_clients_cache
* fix langfuse import
* fix import
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
2024-11-21 19:03:02 -08:00
Krish Dholakia
ba28e52ee8
Litellm lm studio embedding params ( #6746 )
...
* fix(ollama.py): fix get model info request
Fixes https://github.com/BerriAI/litellm/issues/6703
* feat(anthropic/chat/transformation.py): support passing user id to anthropic via openai 'user' param
* docs(anthropic.md): document all supported openai params for anthropic
* test: fix tests
* fix: fix tests
* feat(jina_ai/): add rerank support
Closes https://github.com/BerriAI/litellm/issues/6691
* test: handle service unavailable error
* fix(handler.py): refactor together ai rerank call
* test: update test to handle overloaded error
* test: fix test
* Litellm router trace (#6742 )
* feat(router.py): add trace_id to parent functions - allows tracking retry/fallbacks
* feat(router.py): log trace id across retry/fallback logic
allows grouping llm logs for the same request
* test: fix tests
* fix: fix test
* fix(transformation.py): only set non-none stop_sequences
* Litellm router disable fallbacks (#6743 )
* bump: version 1.52.6 → 1.52.7
* feat(router.py): enable dynamically disabling fallbacks
Allows for enabling/disabling fallbacks per key
* feat(litellm_pre_call_utils.py): support setting 'disable_fallbacks' on litellm key
* test: fix test
* fix(exception_mapping_utils.py): map 'model is overloaded' to internal server error
* fix(lm_studio/embed): support translating lm studio optional params
'
* feat(auth_checks.py): fix auth check inside route - `/team/list`
Fixes regression where non-admin w/ user_id=None able to query all teams
* docs proxy_budget_rescheduler_min_time
* helm run DISABLE_SCHEMA_UPDATE
* docs helm pre sync hook
* fix migration job.yaml
* fix DATABASE_URL
* use existing spec for migrations job
* fix yaml on migrations job
* fix migration job
* update doc on pre sync hook
* fix migrations-job.yaml
* fix migration job
* fix prisma migration
* test - handle eol model claude-2, use claude-2.1 instead
* (docs) add instructions on how to contribute to docker image
* Update code blocks huggingface.md (#6737 )
* Update prefix.md (#6734 )
* fix test_supports_response_schema
* mark Helm PreSyn as BETA
* (Feat) Add support for storing virtual keys in AWS SecretManager (#6728 )
* add SecretManager to httpxSpecialProvider
* fix importing AWSSecretsManagerV2
* add unit testing for writing keys to AWS secret manager
* use KeyManagementEventHooks for key/generated events
* us event hooks for key management endpoints
* working AWSSecretsManagerV2
* fix write secret to AWS secret manager on /key/generate
* fix KeyManagementSettings
* use tasks for key management hooks
* add async_delete_secret
* add test for async_delete_secret
* use _delete_virtual_keys_from_secret_manager
* fix test secret manager
* test_key_generate_with_secret_manager_call
* fix check for key_management_settings
* sync_read_secret
* test_aws_secret_manager
* fix sync_read_secret
* use helper to check when _should_read_secret_from_secret_manager
* test_get_secret_with_access_mode
* test - handle eol model claude-2, use claude-2.1 instead
* docs AWS secret manager
* fix test_read_nonexistent_secret
* fix test_supports_response_schema
* ci/cd run again
* LiteLLM Minor Fixes & Improvement (11/14/2024) (#6730 )
* fix(ollama.py): fix get model info request
Fixes https://github.com/BerriAI/litellm/issues/6703
* feat(anthropic/chat/transformation.py): support passing user id to anthropic via openai 'user' param
* docs(anthropic.md): document all supported openai params for anthropic
* test: fix tests
* fix: fix tests
* feat(jina_ai/): add rerank support
Closes https://github.com/BerriAI/litellm/issues/6691
* test: handle service unavailable error
* fix(handler.py): refactor together ai rerank call
* test: update test to handle overloaded error
* test: fix test
* Litellm router trace (#6742 )
* feat(router.py): add trace_id to parent functions - allows tracking retry/fallbacks
* feat(router.py): log trace id across retry/fallback logic
allows grouping llm logs for the same request
* test: fix tests
* fix: fix test
* fix(transformation.py): only set non-none stop_sequences
* Litellm router disable fallbacks (#6743 )
* bump: version 1.52.6 → 1.52.7
* feat(router.py): enable dynamically disabling fallbacks
Allows for enabling/disabling fallbacks per key
* feat(litellm_pre_call_utils.py): support setting 'disable_fallbacks' on litellm key
* test: fix test
* fix(exception_mapping_utils.py): map 'model is overloaded' to internal server error
* test: handle gemini error
* test: fix test
* fix: new run
* bump: version 1.52.7 → 1.52.8
* docs: add docs on jina ai rerank support
* docs(reliability.md): add tutorial on disabling fallbacks per key
* docs(logging.md): add 'trace_id' param to standard logging payload
* (feat) add bedrock/stability.stable-image-ultra-v1:0 (#6723 )
* add stability.stable-image-ultra-v1:0
* add pricing for stability.stable-image-ultra-v1:0
* fix test_supports_response_schema
* ci/cd run again
* [Feature]: Stop swallowing up AzureOpenAi exception responses in litellm's implementation for a BadRequestError (#6745 )
* fix azure exceptions
* test_bad_request_error_contains_httpx_response
* test_bad_request_error_contains_httpx_response
* use safe access to get exception response
* fix get attr
* [Feature]: json_schema in response support for Anthropic (#6748 )
* _convert_tool_response_to_message
* fix ModelResponseIterator
* fix test_json_response_format
* test_json_response_format_stream
* fix _convert_tool_response_to_message
* use helper _handle_json_mode_chunk
* fix _process_response
* unit testing for test_convert_tool_response_to_message_no_arguments
* update doc for JSON mode
* fix: import audio check (#6740 )
* fix imagegeneration output_cost_per_image on model cost map (#6752 )
* (feat) Vertex AI - add support for fine tuned embedding models (#6749 )
* fix use fine tuned vertex embedding models
* test_vertex_embedding_url
* add _transform_openai_request_to_fine_tuned_embedding_request
* add _transform_openai_request_to_fine_tuned_embedding_request
* add transform_openai_request_to_vertex_embedding_request
* add _transform_vertex_response_to_openai_for_fine_tuned_models
* test_vertexai_embedding for ft models
* fix test_vertexai_embedding_finetuned
* doc fine tuned / custom embedding models
* fix test test_partner_models_httpx
* bump: version 1.52.8 → 1.52.9
* LiteLLM Minor Fixes & Improvements (11/13/2024) (#6729 )
* fix(utils.py): add logprobs support for together ai
Fixes
https://github.com/BerriAI/litellm/issues/6724
* feat(pass_through_endpoints/): add anthropic/ pass-through endpoint
adds new `anthropic/` pass-through endpoint + refactors docs
* feat(spend_management_endpoints.py): allow /global/spend/report to query team + customer id
enables seeing spend for a customer in a team
* Add integration with MLflow Tracing (#6147 )
* Add MLflow logger
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Streaming handling
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* lint
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* address comments and fix issues
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* address comments and fix issues
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Move logger construction code
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Add docs
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* async handlers
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* new picture
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* fix(mlflow.py): fix ruff linting errors
* ci(config.yml): add mlflow to ci testing
* fix: fix test
* test: fix test
* Litellm key update fix (#6710 )
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* fix(key_management_endpoints.py): fix /key/update with metadata update
* fix(key_management_endpoints.py): fix key_prepare_update helper
* fix(key_management_endpoints.py): reset value to none if set in key update
* fix: update test
'
* Litellm dev 11 11 2024 (#6693 )
* fix(__init__.py): add 'watsonx_text' as mapped llm api route
Fixes https://github.com/BerriAI/litellm/issues/6663
* fix(opentelemetry.py): fix passing parallel tool calls to otel
Fixes https://github.com/BerriAI/litellm/issues/6677
* refactor(test_opentelemetry_unit_tests.py): create a base set of unit tests for all logging integrations - test for parallel tool call handling
reduces bugs in repo
* fix(__init__.py): update provider-model mapping to include all known provider-model mappings
Fixes https://github.com/BerriAI/litellm/issues/6669
* feat(anthropic): support passing document in llm api call
* docs(anthropic.md): add pdf anthropic call to docs + expose new 'supports_pdf_input' function
* fix(factory.py): fix linting error
* add clear doc string for GCS bucket logging
* Add docs to export logs to Laminar (#6674 )
* Add docs to export logs to Laminar
* minor fix: newline at end of file
* place laminar after http and grpc
* (Feat) Add langsmith key based logging (#6682 )
* add langsmith_api_key to StandardCallbackDynamicParams
* create a file for langsmith types
* langsmith add key / team based logging
* add key based logging for langsmith
* fix langsmith key based logging
* fix linting langsmith
* remove NOQA violation
* add unit test coverage for all helpers in test langsmith
* test_langsmith_key_based_logging
* docs langsmith key based logging
* run langsmith tests in logging callback tests
* fix logging testing
* test_langsmith_key_based_logging
* test_add_callback_via_key_litellm_pre_call_utils_langsmith
* add debug statement langsmith key based logging
* test_langsmith_key_based_logging
* (fix) OpenAI's optional messages[].name does not work with Mistral API (#6701 )
* use helper for _transform_messages mistral
* add test_message_with_name to base LLMChat test
* fix linting
* add xAI on Admin UI (#6680 )
* (docs) add benchmarks on 1K RPS (#6704 )
* docs litellm proxy benchmarks
* docs GCS bucket
* doc fix - reduce clutter on logging doc title
* (feat) add cost tracking stable diffusion 3 on Bedrock (#6676 )
* add cost tracking for sd3
* test_image_generation_bedrock
* fix get model info for image cost
* add cost_calculator for stability 1 models
* add unit testing for bedrock image cost calc
* test_cost_calculator_with_no_optional_params
* add test_cost_calculator_basic
* correctly allow size Optional
* fix cost_calculator
* sd3 unit tests cost calc
* fix raise correct error 404 when /key/info is called on non-existent key (#6653 )
* fix raise correct error on /key/info
* add not_found_error error
* fix key not found in DB error
* use 1 helper for checking token hash
* fix error code on key info
* fix test key gen prisma
* test_generate_and_call_key_info
* test fix test_call_with_valid_model_using_all_models
* fix key info tests
* bump: version 1.52.4 → 1.52.5
* add defaults used for GCS logging
* LiteLLM Minor Fixes & Improvements (11/12/2024) (#6705 )
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* bump: version 1.52.5 → 1.52.6
* (feat) helm hook to sync db schema (#6715 )
* v0 migration job
* fix job
* fix migrations job.yml
* handle standalone DB on helm hook
* fix argo cd annotations
* fix db migration helm hook
* fix migration job
* doc fix Using Http/2 with Hypercorn
* (fix proxy redis) Add redis sentinel support (#6154 )
* add sentinel_password support
* add doc for setting redis sentinel password
* fix redis sentinel - use sentinel password
* Fix: Update gpt-4o costs to that of gpt-4o-2024-08-06 (#6714 )
Fixes #6713
* (fix) using Anthropic `response_format={"type": "json_object"}` (#6721 )
* add support for response_format=json anthropic
* add test_json_response_format to baseLLM ChatTest
* fix test_litellm_anthropic_prompt_caching_tools
* fix test_anthropic_function_call_with_no_schema
* test test_create_json_tool_call_for_response_format
* (feat) Add cost tracking for Azure Dall-e-3 Image Generation + use base class to ensure basic image generation tests pass (#6716 )
* add BaseImageGenTest
* use 1 class for unit testing
* add debugging to BaseImageGenTest
* TestAzureOpenAIDalle3
* fix response_cost_calculator
* test_basic_image_generation
* fix img gen basic test
* fix _select_model_name_for_cost_calc
* fix test_aimage_generation_bedrock_with_optional_params
* fix undo changes cost tracking
* fix response_cost_calculator
* fix test_cost_azure_gpt_35
* fix remove dup test (#6718 )
* (build) update db helm hook
* (build) helm db pre sync hook
* (build) helm db sync hook
* test: run test_team_logging firdst
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
* test: update test
* test: skip anthropic overloaded error
* test: cleanup test
* test: update tests
* test: fix test
* test: handle gemini overloaded model error
* test: handle internal server error
* test: handle anthropic overloaded error
* test: handle claude instability
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
Co-authored-by: Yuki Watanabe <31463517+B-Step62@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Jongseob Jeon <aiden.jongseob@gmail.com>
Co-authored-by: Camden Clark <camdenaws@gmail.com>
Co-authored-by: Rasswanth <61219215+IamRash-7@users.noreply.github.com>
Co-authored-by: Yuki Watanabe <31463517+B-Step62@users.noreply.github.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
2024-11-19 09:54:50 +05:30
Ishaan Jaff
f43768d617
(fix) httpx handler - bind to ipv4 for httpx handler ( #6785 )
...
* bind to ipv4 on httpx handler
* add force_ipv4
* use helper for _create_async_transport
* fix circular import
* document force_ipv4
* test_async_http_handler_force_ipv4
2024-11-18 12:22:51 -08:00
Krish Dholakia
3beecfb0d4
LiteLLM Minor Fixes & Improvements (11/13/2024) ( #6729 )
...
* fix(utils.py): add logprobs support for together ai
Fixes
https://github.com/BerriAI/litellm/issues/6724
* feat(pass_through_endpoints/): add anthropic/ pass-through endpoint
adds new `anthropic/` pass-through endpoint + refactors docs
* feat(spend_management_endpoints.py): allow /global/spend/report to query team + customer id
enables seeing spend for a customer in a team
* Add integration with MLflow Tracing (#6147 )
* Add MLflow logger
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Streaming handling
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* lint
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* address comments and fix issues
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* address comments and fix issues
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Move logger construction code
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Add docs
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* async handlers
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* new picture
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* fix(mlflow.py): fix ruff linting errors
* ci(config.yml): add mlflow to ci testing
* fix: fix test
* test: fix test
* Litellm key update fix (#6710 )
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* fix(key_management_endpoints.py): fix /key/update with metadata update
* fix(key_management_endpoints.py): fix key_prepare_update helper
* fix(key_management_endpoints.py): reset value to none if set in key update
* fix: update test
'
* Litellm dev 11 11 2024 (#6693 )
* fix(__init__.py): add 'watsonx_text' as mapped llm api route
Fixes https://github.com/BerriAI/litellm/issues/6663
* fix(opentelemetry.py): fix passing parallel tool calls to otel
Fixes https://github.com/BerriAI/litellm/issues/6677
* refactor(test_opentelemetry_unit_tests.py): create a base set of unit tests for all logging integrations - test for parallel tool call handling
reduces bugs in repo
* fix(__init__.py): update provider-model mapping to include all known provider-model mappings
Fixes https://github.com/BerriAI/litellm/issues/6669
* feat(anthropic): support passing document in llm api call
* docs(anthropic.md): add pdf anthropic call to docs + expose new 'supports_pdf_input' function
* fix(factory.py): fix linting error
* add clear doc string for GCS bucket logging
* Add docs to export logs to Laminar (#6674 )
* Add docs to export logs to Laminar
* minor fix: newline at end of file
* place laminar after http and grpc
* (Feat) Add langsmith key based logging (#6682 )
* add langsmith_api_key to StandardCallbackDynamicParams
* create a file for langsmith types
* langsmith add key / team based logging
* add key based logging for langsmith
* fix langsmith key based logging
* fix linting langsmith
* remove NOQA violation
* add unit test coverage for all helpers in test langsmith
* test_langsmith_key_based_logging
* docs langsmith key based logging
* run langsmith tests in logging callback tests
* fix logging testing
* test_langsmith_key_based_logging
* test_add_callback_via_key_litellm_pre_call_utils_langsmith
* add debug statement langsmith key based logging
* test_langsmith_key_based_logging
* (fix) OpenAI's optional messages[].name does not work with Mistral API (#6701 )
* use helper for _transform_messages mistral
* add test_message_with_name to base LLMChat test
* fix linting
* add xAI on Admin UI (#6680 )
* (docs) add benchmarks on 1K RPS (#6704 )
* docs litellm proxy benchmarks
* docs GCS bucket
* doc fix - reduce clutter on logging doc title
* (feat) add cost tracking stable diffusion 3 on Bedrock (#6676 )
* add cost tracking for sd3
* test_image_generation_bedrock
* fix get model info for image cost
* add cost_calculator for stability 1 models
* add unit testing for bedrock image cost calc
* test_cost_calculator_with_no_optional_params
* add test_cost_calculator_basic
* correctly allow size Optional
* fix cost_calculator
* sd3 unit tests cost calc
* fix raise correct error 404 when /key/info is called on non-existent key (#6653 )
* fix raise correct error on /key/info
* add not_found_error error
* fix key not found in DB error
* use 1 helper for checking token hash
* fix error code on key info
* fix test key gen prisma
* test_generate_and_call_key_info
* test fix test_call_with_valid_model_using_all_models
* fix key info tests
* bump: version 1.52.4 → 1.52.5
* add defaults used for GCS logging
* LiteLLM Minor Fixes & Improvements (11/12/2024) (#6705 )
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* bump: version 1.52.5 → 1.52.6
* (feat) helm hook to sync db schema (#6715 )
* v0 migration job
* fix job
* fix migrations job.yml
* handle standalone DB on helm hook
* fix argo cd annotations
* fix db migration helm hook
* fix migration job
* doc fix Using Http/2 with Hypercorn
* (fix proxy redis) Add redis sentinel support (#6154 )
* add sentinel_password support
* add doc for setting redis sentinel password
* fix redis sentinel - use sentinel password
* Fix: Update gpt-4o costs to that of gpt-4o-2024-08-06 (#6714 )
Fixes #6713
* (fix) using Anthropic `response_format={"type": "json_object"}` (#6721 )
* add support for response_format=json anthropic
* add test_json_response_format to baseLLM ChatTest
* fix test_litellm_anthropic_prompt_caching_tools
* fix test_anthropic_function_call_with_no_schema
* test test_create_json_tool_call_for_response_format
* (feat) Add cost tracking for Azure Dall-e-3 Image Generation + use base class to ensure basic image generation tests pass (#6716 )
* add BaseImageGenTest
* use 1 class for unit testing
* add debugging to BaseImageGenTest
* TestAzureOpenAIDalle3
* fix response_cost_calculator
* test_basic_image_generation
* fix img gen basic test
* fix _select_model_name_for_cost_calc
* fix test_aimage_generation_bedrock_with_optional_params
* fix undo changes cost tracking
* fix response_cost_calculator
* fix test_cost_azure_gpt_35
* fix remove dup test (#6718 )
* (build) update db helm hook
* (build) helm db pre sync hook
* (build) helm db sync hook
* test: run test_team_logging firdst
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
* test: update test
* test: skip anthropic overloaded error
* test: cleanup test
* test: update tests
* test: fix test
* test: handle gemini overloaded model error
* test: handle internal server error
* test: handle anthropic overloaded error
* test: handle claude instability
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
Co-authored-by: Yuki Watanabe <31463517+B-Step62@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
2024-11-15 11:18:31 +05:30
Rasswanth
0585fd56d6
fix: import audio check ( #6740 )
2024-11-14 17:00:38 -08:00
