Commit graph

19081 commits

Author SHA1 Message Date
Krish Dholakia
6eb2346fd6 QA: ensure all bedrock regional models have same supported_ as base + Anthropic nested pydantic object support (#7844)
* build: ensure all regional bedrock models have same supported values as base bedrock model

prevents drift

* test(base_llm_unit_tests.py): add testing for nested pydantic objects

* fix(test_utils.py): add test_get_potential_model_names

* fix(anthropic/chat/transformation.py): support nested pydantic objects

Fixes https://github.com/BerriAI/litellm/issues/7755
2025-01-17 19:49:12 -08:00
Ishaan Jaff
37ed49fe72 bump: version 1.58.4 → 1.59.0 2025-01-17 19:35:15 -08:00
Ishaan Jaff
bc6a9cd29c [Hashicorp - secret manager] - use vault namespace for tls auth (#7834)
* hcorp - use x-vault-namespace

* _get_tls_cert_auth_body

* HCP_VAULT_CERT_ROLE

* test_hashicorp_secret_manager_tls_cert_auth

* HCP_VAULT_CERT_ROLE
2025-01-17 19:27:56 -08:00
Ishaan Jaff
c8d6254b78 ui new build 2025-01-17 19:23:41 -08:00
Ishaan Jaff
dfd5407c0d rename ui tab to ExperimentOutlined 2025-01-17 19:22:32 -08:00
Ishaan Jaff
d4d6498e14 ui new build 2025-01-17 19:14:44 -08:00
Ishaan Jaff
f37e848f7f [fix dd llm obs] - use env vars for setting dd tags, service name (#7835)
* fix custom logger

* fix debugging dd llm obs
2025-01-17 18:57:16 -08:00
Ishaan Jaff
16673ab488 (UI - View SpendLogs Table) (#7842)
* litellm log messages / responses

* add messages/response to schema.prisma

* add support for logging messages / responses in DB

* test_spend_logs_payload_with_prompts_enabled

* _get_messages_for_spend_logs_payload

* ui_view_spend_logs endpoint

* add tanstack and moment

* add uiSpendLogsCall

* ui view logs table

* ui view spendLogs table

* ui_view_spend_logs

* fix code quality

* test_spend_logs_payload_with_prompts_enabled

* _get_messages_for_spend_logs_payload

* test_spend_logs_payload_with_prompts_enabled

* test_spend_logs_payload_with_prompts_enabled

* ui view spend logs

* minor ui fix

* ui - update leftnav

* ui - clean up ui

* fix leftnav

* ui fix navbar

* ui fix moving chat ui tab
2025-01-17 18:53:45 -08:00
Krish Dholakia
fc7a931485 fix(key_management_endpoints.py): fix default allowed team member roles (#7843)
admin and user, not admin and member
2025-01-17 17:15:22 -08:00
yujonglee
b0e30906e0 add key and team level budget (#7831) 2025-01-17 09:04:12 -08:00
Ishaan Jaff
4538df580e bump: version 1.58.3 → 1.58.4 2025-01-17 08:29:13 -08:00
Ishaan Jaff
23fca72fb8 Revert "fix custom logger"
This reverts commit 9d2707ecfe.
2025-01-17 08:26:59 -08:00
Ishaan Jaff
4ef821984d fix custom logger 2025-01-17 07:39:49 -08:00
Ishaan Jaff
fbdb80ac52 Revert "fix: fix test"
This reverts commit 0642a78abb.
2025-01-17 07:21:19 -08:00
Ishaan Jaff
081da2b1c3 Revert "test_completion_mistral_api_mistral_large_function_call"
This reverts commit ef9177f0a8.
2025-01-17 07:20:46 -08:00
Krrish Dholakia
f6780ca16d fix: fix test 2025-01-17 07:16:32 -08:00
Ishaan Jaff
2e91ab52ec _handle_tool_call_message linting 2025-01-16 22:34:16 -08:00
Ishaan Jaff
cac13c032c test_watsonx_token_in_env_var 2025-01-16 22:28:37 -08:00
Ishaan Jaff
b6d1ab6152 test_completion_mistral_api_mistral_large_function_call 2025-01-16 22:27:48 -08:00
Ishaan Jaff
e89c3bad5f sec fix minor (#7810) 2025-01-16 22:03:28 -08:00
Ishaan Jaff
b4735fbbc0 (Fix + Testing) - Add dd-trace-run to litellm ci/cd pipeline + fix bug caused by dd-trace patching OpenAI sdk (#7820)
* add dd trace to e2e docker run tests

* update dd trace v

* fix entrypoint

* dd trace fixes

* proxy_build_from_pip_tests

* build python3.13

* use py 3.13

* fix build from pip

* dd trace fix

* proxy_build_from_pip_tests

* bump build from pip
2025-01-16 22:03:09 -08:00
Ishaan Jaff
559006643d (fix) IBM Watsonx using ZenApiKey (#7821)
* ibm watsonx fix

* test ZenAPIKey

* fix zenapikey
2025-01-16 22:02:36 -08:00
Ishaan Jaff
2177bdc836 (datadog llm observability) - fixes + improvements for using datadog llm observability logging integration (#7824)
* dd llm obs fixes

* _ensure_string_content

* fix _get_dd_llm_obs_payload_metadata
2025-01-16 22:02:24 -08:00
Ishaan Jaff
5821985ca3 run ci/cd again 2025-01-16 22:02:03 -08:00
Ishaan Jaff
a2364b1fd6 test_completion_mistral_api_mistral_large_function_call 2025-01-16 21:50:56 -08:00
Ishaan Jaff
6e3828ac84 test_async_vertexai_streaming_response 2025-01-16 21:45:12 -08:00
Ishaan Jaff
5e9ea29ffe llama-v3p1-8b-instruct 2025-01-16 21:34:42 -08:00
Ishaan Jaff
fb34ea7623 fireworks ai use llama-v3p1-8b-instruct 2025-01-16 21:28:44 -08:00
Krrish Dholakia
56e07618e0 test: fix unit test 2025-01-16 21:11:17 -08:00
Ishaan Jaff
28dd22767c test commit on main 2025-01-16 20:52:55 -08:00
Krish Dholakia
0fccea39e4 build: bump certifi version - see if that fixes asyncio ssl issue on python 3.13 (#7800) 2025-01-15 22:02:45 -08:00
Krish Dholakia
f37dc43e92 test: initial commit enforcing testing on all anthropic pass through … (#7794)
* test: initial commit enforcing testing on all anthropic pass through functions

prevents future regressions

* test(test_unit_test_anthropic_pass_through.py): add unit test for '_get_user_from_metadata' function

* test(test_unit_test_anthropic_passthrough.py): add unit test for handle_logging_anthropic_collected_chunks

* test(test_unit_test_anthropic_pass_through): add coverage for all anthropic pass through functions
2025-01-15 22:02:35 -08:00
Krish Dholakia
fbdd88d79c test: initial test to enforce all functions in user_api_key_auth.py h… (#7797)
* test: initial test to enforce all functions in user_api_key_auth.py have direct testing

* test(test_user_api_key_auth.py): add is_allowed_route unit test

* test(test_user_api_key_auth.py): add more tests

* test(test_user_api_key_auth.py): add complete testing coverage for all functions in `user_api_key_auth.py`

* test(test_db_schema_changes.py): add a unit test to ensure all db schema changes are backwards compatible

gives user an easy rollback path

* test: fix schema compatibility test filepath

* test: fix test
2025-01-15 21:52:45 -08:00
Krish Dholakia
6473f9ad02 Add back in non root image fixes (#7781) (#7795)
* Add back in non root image fixes (#7781)

* Add back in non root image fixes

* Fix dockerfile

* Fix perms

* Add in container structure tests for the nonroot image (#7796)

* feat(helm): add securityContext and pull policy values to migration job (#7652)

* fix(helm): corrected indentation in migration-job.yaml

* feat(helm): add securityContext and pull policy values to migration job

* fix confusing save button label (#7778)

* [integrations/lunary] Improve Lunary documentaiton (#7770)

* update lunary doc

* better title

* tweaks

* Update langchain.md

* Update lunary_integration.md

* Fix wrong URL for internal user invitation (#7762)

* format

* done

* Update instructor tutorial (#7784)

* Add in container structure tests for the nonroot image

---------

Co-authored-by: Zackeus Bengtsson <32719220+Hexoplon@users.noreply.github.com>
Co-authored-by: yujonglee <yujonglee.dev@gmail.com>
Co-authored-by: Hugues Chocart <chocart.hugues@icloud.com>
Co-authored-by: Nikolaiev Dmytro <dima.nikol.99@gmail.com>

---------

Co-authored-by: Rajat Vig <rajatvig@users.noreply.github.com>
Co-authored-by: Zackeus Bengtsson <32719220+Hexoplon@users.noreply.github.com>
Co-authored-by: yujonglee <yujonglee.dev@gmail.com>
Co-authored-by: Hugues Chocart <chocart.hugues@icloud.com>
Co-authored-by: Nikolaiev Dmytro <dima.nikol.99@gmail.com>
2025-01-15 21:49:03 -08:00
Krish Dholakia
543655adc7 Litellm dev 01 14 2025 p2 (#7772)
* feat(pass_through_endpoints.py): fix anthropic end user cost tracking

* fix(anthropic/chat/transformation.py): use returned provider model for anthropic

handles anthropic `-latest` tag in request body throwing cost calculation errors

ensures we can be accurate in our model cost tracking

* feat(model_prices_and_context_window.json): add gemini-2.0-flash-thinking-exp pricing

* test: update test to use assumption that user_api_key_dict can get anthropic user id

* test: fix test

* fix: fix test

* fix(anthropic_pass_through.py): uncomment previous anthropic end-user cost tracking code block

can't guarantee user api key dict always has end user id - too many code paths

* fix(user_api_key_auth.py): this allows end user id from request body to always be read and set in auth object

* fix(auth_check.py): fix linting error

* test: fix auth check

* fix(auth_utils.py): fix get end user id to handle metadata = None
2025-01-15 21:34:50 -08:00
Ishaan Jaff
73c004cfe5 (helm) - allow specifying envVars on values.yaml + add helm lint test (#7789)
* litellm use envVars values.yaml

* fix values.yaml

* add helm lint to ci/cd pipeline

* working values.yaml

* add helm tests to ci/cd

* fix helm chart testing

* update helm tests

* fix helm test

* fix use test values in ci

* fix busy box testing on helm

* fix test-values.yaml

* update helm tests

* fix test connection
2025-01-15 21:11:17 -08:00
Nikolaiev Dmytro
b51c46200c Update instructor tutorial (#7784) 2025-01-15 15:10:50 -08:00
yujonglee
3c47d6599c Fix wrong URL for internal user invitation (#7762)
* format

* done
2025-01-15 15:10:32 -08:00
Hugues Chocart
c919e3ca45 [integrations/lunary] Improve Lunary documentaiton (#7770)
* update lunary doc

* better title

* tweaks

* Update langchain.md

* Update lunary_integration.md
2025-01-15 15:00:25 -08:00
yujonglee
1bc114ffff fix confusing save button label (#7778) 2025-01-15 13:51:07 -08:00
Zackeus Bengtsson
9696afcb33 feat(helm): add securityContext and pull policy values to migration job (#7652)
* fix(helm): corrected indentation in migration-job.yaml

* feat(helm): add securityContext and pull policy values to migration job
2025-01-15 12:51:08 -08:00
Krrish Dholakia
037842e866 build: bump: version 1.58.2 → 1.58.3 2025-01-15 07:46:57 -08:00
Krish Dholakia
26c1b86f4e Litellm dev 01 2025 p4 (#7776)
* fix(gemini/): support gemini 'frequency_penalty' and 'presence_penalty'

Closes https://github.com/BerriAI/litellm/issues/7748

* feat(proxy_server.py): new env var to disable prisma health check on startup

* test: fix test
2025-01-14 21:49:25 -08:00
Krish Dholakia
142662a504 build(pyproject.toml): bump uvicorn depedency requirement (#7773)
* build(pyproject.toml): bump uvicorn depedency requirement

Fixes https://github.com/BerriAI/litellm/issues/7768

* fix(anthropic/chat/transformation.py): fix is_vertex_request check to actually use optional param passed in

Fixes https://github.com/BerriAI/litellm/issues/6898#issuecomment-2590860695

* fix(o1_transformation.py): fix azure o1 'is_o1_model' check to just check for o1 in model string

https://github.com/BerriAI/litellm/issues/7743

* test: load vertex creds
2025-01-14 21:47:11 -08:00
Ishaan Jaff
dbe94a903d bump: version 1.58.1 → 1.58.2 2025-01-14 20:18:29 -08:00
Ishaan Jaff
6a4e8c33b3 (fix) BaseAWSLLM - cache IAM role credentials when used (#7775)
* fix base aws llm

* fix auth with aws role

* test aws base llm

* fix base aws llm init

* run ci/cd again

* fix get_credentials

* ci/cd run again

* _auth_with_aws_role
2025-01-14 20:16:22 -08:00
Ishaan Jaff
25ae1e9117 (Feat) prometheus - emit remaining team budget metric on proxy startup (#7777)
* fix get_paginated_teams

* use _initialize_remaining_budget_metrics

* fix prom metric

* run ci/cd again

* fix run async func

* fix _initialize_prometheus_startup_metrics

* fix _initialize_prometheus_startup_metrics

* prom unit tests

* test_get_paginated_teams
2025-01-14 20:08:23 -08:00
Ishaan Jaff
fadc224fcd docs iam role based access for bedrock (#7774) 2025-01-14 19:02:02 -08:00
Krish Dholakia
178cfe3c57 Litellm dev 01 13 2025 p2 (#7758)
* fix(factory.py): fix bedrock document url check

Make check more generic - if starts with 'text' or 'application' assume it's a document and let it go through

 Fixes https://github.com/BerriAI/litellm/issues/7746

* feat(key_management_endpoints.py): support writing new key alias to aws secret manager - on key rotation

adds rotation endpoint to aws key management hook - allows for rotated litellm virtual keys with new key alias to be written to it

* feat(key_management_event_hooks.py): support rotating keys and updating secret manager

* refactor(base_secret_manager.py): support rotate secret at the base level

since it's just an abstraction function, it's easy to implement at the base manager level

* style: cleanup unused imports
2025-01-14 17:04:01 -08:00
Krish Dholakia
d7a13ad561 Support temporary budget increases on keys (#7754)
* fix(gpt_transformation.py): fix response_format translation check for 4o models

Fixes https://github.com/BerriAI/litellm/issues/7616

* feat(key_management_endpoints.py): support 'temp_budget_increase' and 'temp_budget_expiry' fields

Allow proxy admin to grant temporary budget increases to keys

* fix(proxy/_types.py): enforce temp_budget_increase and temp_budget_expiry are always passed together

* feat(user_api_key_auth.py): initial working temp budget increase logic

ensures key budget exceeded error checks for temp budget in key metadata

* feat(proxy_server.py): return the key max budget and key spend in the response headers

Allows clientside user to know their remaining limits

* test: add unit testing for new proxy utils

Ensures new key budget is correctly handled

* docs(temporary_budget_increase.md): add doc on temporary budget increase

* fix(utils.py): remove 3.5 from response_format check for now

not all azure  3.5 models support response_format

* fix(user_api_key_auth.py): return valid user api key auth object on all paths
2025-01-14 17:03:11 -08:00