Commit graph

21496 commits

Author SHA1 Message Date
Ishaan Jaff
441c7275ed
test fix post call rules (#9826) 2025-04-08 13:55:37 -07:00
Ishaan Jaff
e6403b717c
[Security fix - CVE-2025-0330] - Leakage of Langfuse API keys in team exception handling (#9830)
* fix team id exception in get team config

* test_team_info_masking

* test ref
2025-04-08 13:55:20 -07:00
Krrish Dholakia
367f48004d build(model_prices_and_context_window.json): consistent params 2025-04-08 12:45:33 -07:00
Peter Dave Hello
6b67006b0c
Remove redundant apk update in Dockerfiles (cc #5016) (#9055)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 17s
Helm unit test / unit-test (push) Successful in 18s
The `apk` commands can utilize the `--no-cache` option, making the
`update` step superfluous and ensuring the latest packages are used
without maintaining a local cache. An additional `apk update` in the
Dockerfile will just make the image larger with no benefits.
2025-04-08 09:03:25 -07:00
Ishaan Jaff
ff3a6830a4
[Feat] LiteLLM Tag/Policy Management (#9813)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 15s
Helm unit test / unit-test (push) Successful in 21s
* rendering tags on UI

* use /models for building tags

* CRUD endpoints for Tag management

* fix tag management

* working api for LIST tags

* working tag management

* refactor UI components

* fixes ui tag management

* clean up ui tag management

* fix tag management ui

* fix show allowed llms

* e2e tag controls

* stash change for rendering tags on UI

* ui working tag selector on Test Key page

* fixes for tag management

* clean up tag info

* fix code quality

* test for tag management

* ui clarify what tag routing is
2025-04-07 21:54:24 -07:00
Krish Dholakia
ac9f03beae
Allow passing thinking param to litellm proxy via client sdk + Code QA Refactor on get_optional_params (get correct values) (#9386)
* fix(litellm_proxy/chat/transformation.py): support 'thinking' param

Fixes https://github.com/BerriAI/litellm/issues/9380

* feat(azure/gpt_transformation.py): add azure audio model support

Closes https://github.com/BerriAI/litellm/issues/6305

* fix(utils.py): use provider_config in common functions

* fix(utils.py): add missing provider configs to get_chat_provider_config

* test: fix test

* fix: fix path

* feat(utils.py): make bedrock invoke nova config baseconfig compatible

* fix: fix linting errors

* fix(azure_ai/transformation.py): remove buggy optional param filtering for azure ai

Removes incorrect check for support tool choice when calling azure ai - prevented calling models with response_format unless on litell model cost map

* fix(amazon_cohere_transformation.py): fix bedrock invoke cohere transformation to inherit from coherechatconfig

* test: fix azure ai tool choice mapping

* fix: fix model cost map to add 'supports_tool_choice' to cohere models

* fix(get_supported_openai_params.py): check if custom llm provider in llm providers

* fix(get_supported_openai_params.py): fix llm provider in list check

* fix: fix ruff check errors

* fix: support defs when calling bedrock nova

* fix(factory.py): fix test
2025-04-07 21:04:11 -07:00
Krish Dholakia
fcf17d114f
Litellm dev 04 05 2025 p2 (#9774)
* test: move test to just checking async

* fix(transformation.py): handle function call with no schema

* fix(utils.py): handle pydantic base model in message tool calls

Fix https://github.com/BerriAI/litellm/issues/9321

* fix(vertex_and_google_ai_studio.py): handle tools=[]

Fixes https://github.com/BerriAI/litellm/issues/9080

* test: remove max token restriction

* test: fix basic test

* fix(get_supported_openai_params.py): fix check

* fix(converse_transformation.py): support fake streaming for meta.llama3-3-70b-instruct-v1:0

* fix: fix test

* fix: parse out empty dictionary on dbrx streaming + tool calls

* fix(handle-'strict'-param-when-calling-fireworks-ai): fireworks ai does not support 'strict' param

* fix: fix ruff check

'

* fix: handle no strict in function

* fix: revert bedrock change - handle in separate PR
2025-04-07 21:02:52 -07:00
Ishaan Jaff
d8f47fc9e5 databricks/databricks-meta-llama-3-3-70b-instruct 2025-04-07 20:16:24 -07:00
Krish Dholakia
8d338aee78
fix(databricks/chat/transformation.py): remove reasoning_effort from request (#9811)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 16s
Helm unit test / unit-test (push) Successful in 27s
Fixes https://github.com/BerriAI/litellm/issues/9700#issuecomment-2784431995
2025-04-07 19:43:19 -07:00
Krrish Dholakia
fef2af0b17 test: fix flaky test 2025-04-07 19:42:58 -07:00
Krish Dholakia
8e3c7b2de0
fix(vertex_ai.py): move to only passing in accepted keys by vertex ai response schema (#8992)
* fix(vertex_ai.py): common_utils.py

move to only passing in accepted keys by vertex ai

prevent json schema compatible keys like $id, and $comment from causing vertex ai openapi calls to fail

* fix(test_vertex.py): add testing to ensure only accepted schema params passed in

* fix(common_utils.py): fix linting error

* test: update test

* test: accept function
2025-04-07 18:07:01 -07:00
Krish Dholakia
4a128cfd64
Realtime API Cost tracking (#9795)
* fix(proxy_server.py): log realtime calls to spendlogs

Fixes https://github.com/BerriAI/litellm/issues/8410

* feat(realtime/): OpenAI Realtime API cost tracking

Closes https://github.com/BerriAI/litellm/issues/8410

* test: add unit testing for coverage

* test: add more unit testing

* fix: handle edge cases
2025-04-07 16:43:12 -07:00
Krish Dholakia
9a60cd9deb
fix(gemini/transformation.py): handle file_data being passed in (#9786) 2025-04-07 16:32:08 -07:00
Krrish Dholakia
0307a0133b docs: fix doc
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 17s
Helm unit test / unit-test (push) Successful in 23s
2025-04-07 07:21:00 -07:00
Krrish Dholakia
3a7d729d88 docs: cleanup
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 15s
Helm unit test / unit-test (push) Successful in 24s
2025-04-06 14:42:35 -07:00
Krrish Dholakia
0137055bad docs: cleanup 2025-04-06 14:39:28 -07:00
KX
0ac896a6f2
feat: add offline swagger docs (#7653) 2025-04-06 13:55:06 -07:00
Krrish Dholakia
f4c9dce211 docs: cleanup docs
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 16s
Helm unit test / unit-test (push) Successful in 23s
2025-04-06 09:40:17 -07:00
Krish Dholakia
792ee079c2
Litellm 04 05 2025 release notes (#9785)
* docs: update docs

* docs: additional cleanup

* docs(index.md): add initial links

* docs: more doc updates

* docs(index.md): add more links

* docs(files.md): add gemini files API to docs

* docs(index.md): add more docs

* docs: more docs

* docs: update docs
2025-04-06 09:03:51 -07:00
Ishaan Jaff
52b35cd809
[UI Polish] - Polish login screen (#9778)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 21s
Helm unit test / unit-test (push) Successful in 24s
* fix admin ui utils login screen

* ui - add layer of polish on login screen

* ui fix design of login page

* ui fix color scheme on login page
2025-04-05 14:56:03 -07:00
Ishaan Jaff
3769c5cc30 docs release notes 2025-04-05 14:54:47 -07:00
Ishaan Jaff
7262606411 test_completion_cost_databricks 2025-04-05 13:30:17 -07:00
Ishaan Jaff
d87bb9bb6e test_completion_cost_databricks 2025-04-05 13:13:25 -07:00
Ishaan Jaff
1638872762 databricks/databricks-meta-llama-3.3-70b-instruct" 2025-04-05 13:12:21 -07:00
Ishaan Jaff
7f6de81196 ui new build 2025-04-05 12:30:37 -07:00
Ishaan Jaff
80eb1ac8fa
[UI QA/Bug Fix] - Don't change team, key, org, model values on scroll (#9776)
* UI - use 1 component for numerical input

* disable scroll number values on models page

* team edit - disable numerical value scroll

* fix numerical input view

* use numerical component on create key

* add NumericalInput

* ui fix org numerical input

* remove file in incorrect location

* fix NumericalInput
2025-04-05 12:29:31 -07:00
Ishaan Jaff
3a7061a05c
bug fix de depluciate model list (#9775) 2025-04-05 12:29:11 -07:00
Krish Dholakia
34bdf36eab
Add inference providers support for Hugging Face (#8258) (#9738) (#9773)
* Add inference providers support for Hugging Face (#8258)

* add first version of inference providers for huggingface

* temporarily skipping tests

* Add documentation

* Fix titles

* remove max_retries from params and clean up

* add suggestions

* use llm http handler

* update doc

* add suggestions

* run formatters

* add tests

* revert

* revert

* rename file

* set maxsize for lru cache

* fix embeddings

* fix inference url

* fix tests following breaking change in main

* use ChatCompletionRequest

* fix tests and lint

* [Hugging Face] Remove outdated chat completion tests and fix embedding tests (#9749)

* remove or fix tests

* fix link in doc

* fix(config_settings.md): document hf api key

---------

Co-authored-by: célina <hanouticelina@gmail.com>
2025-04-05 10:50:15 -07:00
Krish Dholakia
0d503ad8ad
Move daily user transaction logging outside of 'disable_spend_logs' flag - different tables (#9772)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 16s
Helm unit test / unit-test (push) Successful in 18s
* refactor(db_spend_update_writer.py): aggregate table is entirely different

* test(test_db_spend_update_writer.py): add unit test to ensure if disable_spend_logs is true daily user transactions is still logged

* test: fix test
2025-04-05 09:58:16 -07:00
Michael Clark
cd0a1e6000
Update model_prices (#9768) 2025-04-05 09:20:01 -07:00
Krish Dholakia
d4d3c4f697
build: bump litellm-proxy-extras version (#9771) 2025-04-05 09:02:52 -07:00
Krrish Dholakia
af9db827fc fix(databricks/chat/transformation.py): handle empty headers case 2025-04-05 08:33:56 -07:00
Ishaan Jaff
a771d17794 add prometheus-client to dev deps
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 17s
Helm unit test / unit-test (push) Successful in 22s
2025-04-04 22:28:45 -07:00
Ishaan Jaff
dabbb58cd8 test_nova_optional_params_tool_choice 2025-04-04 22:20:04 -07:00
Krish Dholakia
5099aac1a5
Add DBRX Anthropic w/ thinking + response_format support (#9744)
* feat(databricks/chat/): add anthropic w/ reasoning content support via databricks

Allows user to call claude-3-7-sonnet with thinking via databricks

* refactor: refactor choices transformation + add unit testing

* fix(databricks/chat/transformation.py): support thinking blocks on databricks response streaming

* feat(databricks/chat/transformation.py): support response_format for claude models

* fix(databricks/chat/transformation.py): correctly handle response_format={"type": "text"}

* feat(databricks/chat/transformation.py): support 'reasoning_effort' param mapping for anthropic

* fix: fix ruff errors

* fix: fix linting error

* test: update test

* fix(databricks/chat/transformation.py): handle json mode output parsing

* fix(databricks/chat/transformation.py): handle json mode on streaming

* test: update test

* test: update dbrx testing

* test: update testing

* fix(base_model_iterator.py): handle non-json chunk

* test: update tests

* fix: fix ruff check

* fix: fix databricks config import

* fix: handle _tool = none

* test: skip invalid test
2025-04-04 22:13:32 -07:00
Krish Dholakia
e3b231bc11
fix(litellm-proxy-extras/utils.py): check migrations from correct directory + place prisma schema inside litellm-proxy-extras dir (#9767)
Allows prisma migrate deploy to work as expected on new db's
2025-04-04 22:11:07 -07:00
Ishaan Jaff
220fa23d2b watsonx/ibm/granite-3-8b-instruct 2025-04-04 21:46:02 -07:00
Ishaan Jaff
e2bb203075 update watsonx/ibm/granite-3-8b-instruct" 2025-04-04 21:45:04 -07:00
Ishaan Jaff
f0f2f819bd
Merge pull request #9760 from BerriAI/litellm_prometheus_error_monitoring
[Reliability] Prometheus emit llm provider on failure metric - make it easy to differentiate litellm error vs llm api error
2025-04-04 21:37:28 -07:00
Ishaan Jaff
b7cd4cef07 test_get_exception_class_name 2025-04-04 21:32:55 -07:00
Ishaan Jaff
df4593d58b test prom unit tests 2025-04-04 21:30:05 -07:00
Ishaan Jaff
f4353973bd
Merge pull request #9766 from BerriAI/litellm_add_auth_metrics_endpoint
[Security feature] Allow adding authentication on /metrics endpoints
2025-04-04 21:28:18 -07:00
Ishaan Jaff
b89ed69257
Merge branch 'main' into litellm_add_auth_metrics_endpoint 2025-04-04 21:28:06 -07:00
Ishaan Jaff
f402e9bbd1 _get_exception_class_name 2025-04-04 21:23:21 -07:00
Ishaan Jaff
8559bcc252 DB Transaction Queue Health Metrics 2025-04-04 21:16:12 -07:00
Ishaan Jaff
8c3670e192
Merge pull request #9719 from BerriAI/litellm_metrics_pod_lock_manager
[Reliability] Emit operational metrics for new DB Transaction architecture
2025-04-04 21:12:06 -07:00
Ishaan Jaff
df51d8bcfa Merge branch 'main' into litellm_metrics_pod_lock_manager 2025-04-04 21:11:39 -07:00
Ishaan Jaff
fc4c453cb9 test_no_auth_metrics_when_disabled 2025-04-04 21:02:29 -07:00
Krrish Dholakia
7cd7bdbd0f build: fix model cost map 2025-04-04 20:48:29 -07:00
Krrish Dholakia
5826108c9a build: bump 2025-04-04 20:45:27 -07:00