* build(ui/create_key_button.tsx): support adding tags for cost tracking/routing when making key
* LiteLLM Minor Fixes & Improvements (11/23/2024) (#6870)
* feat(pass_through_endpoints/): support logging anthropic/gemini pass through calls to langfuse/s3/etc.
* fix(utils.py): allow disabling end user cost tracking with new param
Allows proxy admin to disable cost tracking for end user - keeps prometheus metrics small
* docs(configs.md): add disable_end_user_cost_tracking reference to docs
* feat(key_management_endpoints.py): add support for restricting access to `/key/generate` by team/proxy level role
Enables admin to restrict key creation, and assign team admins to handle distributing keys
* test(test_key_management.py): add unit testing for personal / team key restriction checks
* docs: add docs on restricting key creation
* docs(finetuned_models.md): add new guide on calling finetuned models
* docs(input.md): cleanup anthropic supported params
Closes https://github.com/BerriAI/litellm/issues/6856
* test(test_embedding.py): add test for passing extra headers via embedding
* feat(cohere/embed): pass client to async embedding
* feat(rerank.py): add `/v1/rerank` if missing for cohere base url
Closes https://github.com/BerriAI/litellm/issues/6844
* fix(main.py): pass extra_headers param to openai
Fixes https://github.com/BerriAI/litellm/issues/6836
* fix(litellm_logging.py): don't disable global callbacks when dynamic callbacks are set
Fixes issue where global callbacks - e.g. prometheus were overriden when langfuse was set dynamically
* fix(handler.py): fix linting error
* fix: fix typing
* build: add conftest to proxy_admin_ui_tests/
* test: fix test
* fix: fix linting errors
* test: fix test
* fix: fix pass through testing
* feat(key_management_endpoints.py): allow proxy_admin to enforce params on key creation
allows admin to force team keys to have tags
* build(ui/): show teams in leftnav + allow team admin to add new members
* build(ui/): show created tags in dropdown
makes it easier for admin to add tags to keys
* test(test_key_management.py): fix test
* test: fix test
* fix playwright e2e ui test
* fix e2e ui testing deps
* fix: fix linting errors
* fix e2e ui testing
* fix e2e ui testing, only run e2e ui testing in playwright
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* feat(pass_through_endpoints/): support logging anthropic/gemini pass through calls to langfuse/s3/etc.
* fix(utils.py): allow disabling end user cost tracking with new param
Allows proxy admin to disable cost tracking for end user - keeps prometheus metrics small
* docs(configs.md): add disable_end_user_cost_tracking reference to docs
* feat(key_management_endpoints.py): add support for restricting access to `/key/generate` by team/proxy level role
Enables admin to restrict key creation, and assign team admins to handle distributing keys
* test(test_key_management.py): add unit testing for personal / team key restriction checks
* docs: add docs on restricting key creation
* docs(finetuned_models.md): add new guide on calling finetuned models
* docs(input.md): cleanup anthropic supported params
Closes https://github.com/BerriAI/litellm/issues/6856
* test(test_embedding.py): add test for passing extra headers via embedding
* feat(cohere/embed): pass client to async embedding
* feat(rerank.py): add `/v1/rerank` if missing for cohere base url
Closes https://github.com/BerriAI/litellm/issues/6844
* fix(main.py): pass extra_headers param to openai
Fixes https://github.com/BerriAI/litellm/issues/6836
* fix(litellm_logging.py): don't disable global callbacks when dynamic callbacks are set
Fixes issue where global callbacks - e.g. prometheus were overriden when langfuse was set dynamically
* fix(handler.py): fix linting error
* fix: fix typing
* build: add conftest to proxy_admin_ui_tests/
* test: fix test
* fix: fix linting errors
* test: fix test
* fix: fix pass through testing
* feat(customer_endpoints.py): support passing budget duration via `/customer/new` endpoint
Closes https://github.com/BerriAI/litellm/issues/5651
* docs: add missing params to swagger + api documentation test
* docs: add documentation for all key endpoints
documents all params on swagger
* docs(internal_user_endpoints.py): document all /user/new params
Ensures all params are documented
* docs(team_endpoints.py): add missing documentation for team endpoints
Ensures 100% param documentation on swagger
* docs(organization_endpoints.py): document all org params
Adds documentation for all params in org endpoint
* docs(customer_endpoints.py): add coverage for all params on /customer endpoints
ensures all /customer/* params are documented
* ci(config.yml): add endpoint doc testing to ci/cd
* fix: fix internal_user_endpoints.py
* fix(internal_user_endpoints.py): support 'duration' param
* fix(partner_models/main.py): fix anthropic re-raise exception on vertex
* fix: fix pydantic obj
* add SecretManager to httpxSpecialProvider
* fix importing AWSSecretsManagerV2
* add unit testing for writing keys to AWS secret manager
* use KeyManagementEventHooks for key/generated events
* us event hooks for key management endpoints
* working AWSSecretsManagerV2
* fix write secret to AWS secret manager on /key/generate
* fix KeyManagementSettings
* use tasks for key management hooks
* add async_delete_secret
* add test for async_delete_secret
* use _delete_virtual_keys_from_secret_manager
* fix test secret manager
* test_key_generate_with_secret_manager_call
* fix check for key_management_settings
* sync_read_secret
* test_aws_secret_manager
* fix sync_read_secret
* use helper to check when _should_read_secret_from_secret_manager
* test_get_secret_with_access_mode
* test - handle eol model claude-2, use claude-2.1 instead
* docs AWS secret manager
* fix test_read_nonexistent_secret
* fix test_supports_response_schema
* ci/cd run again
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* fix(key_management_endpoints.py): fix /key/update with metadata update
* fix(key_management_endpoints.py): fix key_prepare_update helper
* fix(key_management_endpoints.py): reset value to none if set in key update
* fix: update test
'
* Litellm dev 11 11 2024 (#6693)
* fix(__init__.py): add 'watsonx_text' as mapped llm api route
Fixes https://github.com/BerriAI/litellm/issues/6663
* fix(opentelemetry.py): fix passing parallel tool calls to otel
Fixes https://github.com/BerriAI/litellm/issues/6677
* refactor(test_opentelemetry_unit_tests.py): create a base set of unit tests for all logging integrations - test for parallel tool call handling
reduces bugs in repo
* fix(__init__.py): update provider-model mapping to include all known provider-model mappings
Fixes https://github.com/BerriAI/litellm/issues/6669
* feat(anthropic): support passing document in llm api call
* docs(anthropic.md): add pdf anthropic call to docs + expose new 'supports_pdf_input' function
* fix(factory.py): fix linting error
* add clear doc string for GCS bucket logging
* Add docs to export logs to Laminar (#6674)
* Add docs to export logs to Laminar
* minor fix: newline at end of file
* place laminar after http and grpc
* (Feat) Add langsmith key based logging (#6682)
* add langsmith_api_key to StandardCallbackDynamicParams
* create a file for langsmith types
* langsmith add key / team based logging
* add key based logging for langsmith
* fix langsmith key based logging
* fix linting langsmith
* remove NOQA violation
* add unit test coverage for all helpers in test langsmith
* test_langsmith_key_based_logging
* docs langsmith key based logging
* run langsmith tests in logging callback tests
* fix logging testing
* test_langsmith_key_based_logging
* test_add_callback_via_key_litellm_pre_call_utils_langsmith
* add debug statement langsmith key based logging
* test_langsmith_key_based_logging
* (fix) OpenAI's optional messages[].name does not work with Mistral API (#6701)
* use helper for _transform_messages mistral
* add test_message_with_name to base LLMChat test
* fix linting
* add xAI on Admin UI (#6680)
* (docs) add benchmarks on 1K RPS (#6704)
* docs litellm proxy benchmarks
* docs GCS bucket
* doc fix - reduce clutter on logging doc title
* (feat) add cost tracking stable diffusion 3 on Bedrock (#6676)
* add cost tracking for sd3
* test_image_generation_bedrock
* fix get model info for image cost
* add cost_calculator for stability 1 models
* add unit testing for bedrock image cost calc
* test_cost_calculator_with_no_optional_params
* add test_cost_calculator_basic
* correctly allow size Optional
* fix cost_calculator
* sd3 unit tests cost calc
* fix raise correct error 404 when /key/info is called on non-existent key (#6653)
* fix raise correct error on /key/info
* add not_found_error error
* fix key not found in DB error
* use 1 helper for checking token hash
* fix error code on key info
* fix test key gen prisma
* test_generate_and_call_key_info
* test fix test_call_with_valid_model_using_all_models
* fix key info tests
* bump: version 1.52.4 → 1.52.5
* add defaults used for GCS logging
* LiteLLM Minor Fixes & Improvements (11/12/2024) (#6705)
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* bump: version 1.52.5 → 1.52.6
* (feat) helm hook to sync db schema (#6715)
* v0 migration job
* fix job
* fix migrations job.yml
* handle standalone DB on helm hook
* fix argo cd annotations
* fix db migration helm hook
* fix migration job
* doc fix Using Http/2 with Hypercorn
* (fix proxy redis) Add redis sentinel support (#6154)
* add sentinel_password support
* add doc for setting redis sentinel password
* fix redis sentinel - use sentinel password
* Fix: Update gpt-4o costs to that of gpt-4o-2024-08-06 (#6714)
Fixes#6713
* (fix) using Anthropic `response_format={"type": "json_object"}` (#6721)
* add support for response_format=json anthropic
* add test_json_response_format to baseLLM ChatTest
* fix test_litellm_anthropic_prompt_caching_tools
* fix test_anthropic_function_call_with_no_schema
* test test_create_json_tool_call_for_response_format
* (feat) Add cost tracking for Azure Dall-e-3 Image Generation + use base class to ensure basic image generation tests pass (#6716)
* add BaseImageGenTest
* use 1 class for unit testing
* add debugging to BaseImageGenTest
* TestAzureOpenAIDalle3
* fix response_cost_calculator
* test_basic_image_generation
* fix img gen basic test
* fix _select_model_name_for_cost_calc
* fix test_aimage_generation_bedrock_with_optional_params
* fix undo changes cost tracking
* fix response_cost_calculator
* fix test_cost_azure_gpt_35
* fix remove dup test (#6718)
* (build) update db helm hook
* (build) helm db pre sync hook
* (build) helm db sync hook
* test: run test_team_logging firdst
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
* fix raise correct error on /key/info
* add not_found_error error
* fix key not found in DB error
* use 1 helper for checking token hash
* fix error code on key info
* fix test key gen prisma
* test_generate_and_call_key_info
* test fix test_call_with_valid_model_using_all_models
* fix key info tests
* use CustomBatchLogger for GCS
* add GCS bucket logging type
* use batch logging for GCs bucket
* add gcs_bucket
* allow setting flush_interval on CustomBatchLogger
* set GCS_FLUSH_INTERVAL to 1s
* fix test_key_logging
* fix test_key_logging
* add docs on new env vars
* store sso provider in user metadata
* store user metadata
* store user auth_provider in user metadata
* add "metadata" for LiteLLM_UserTable
* fix sso test
* fix(cost_calculator.py): move to debug for noisy warning message on cost calculation error
Fixes https://github.com/BerriAI/litellm/issues/5610
* fix(databricks/cost_calculator.py): Handles model name issues for databricks models
* fix(main.py): fix stream chunk builder for multiple tool calls
Fixes https://github.com/BerriAI/litellm/issues/5591
* fix: correctly set user_alias when passed in
Fixes https://github.com/BerriAI/litellm/issues/5612
* fix(types/utils.py): allow passing role for message object
https://github.com/BerriAI/litellm/issues/5621
* fix(litellm_logging.py): Fix langfuse logging across multiple projects
Fixes issue where langfuse logger was re-using the old logging object
* feat(proxy/_types.py): support adding key-based tags for tag-based routing
Enable tag based routing at key-level
* fix(proxy/_types.py): fix inheritance
* test(test_key_generate_prisma.py): fix test
* test: fix test
* fix(litellm_logging.py): return used callback object
