phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-25 10:44:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
21459 commits 2480 branches 894 tags 534 MiB
Commit graph

1610 commits

Author SHA1 Message Date
Ishaan Jaff
d87bb9bb6e test_completion_cost_databricks 2025-04-05 13:13:25 -07:00
Ishaan Jaff
1638872762 databricks/databricks-meta-llama-3.3-70b-instruct" 2025-04-05 13:12:21 -07:00
Krish Dholakia
34bdf36eab
Add inference providers support for Hugging Face (#8258) (#9738) (#9773) 
* Add inference providers support for Hugging Face (#8258)

* add first version of inference providers for huggingface

* temporarily skipping tests

* Add documentation

* Fix titles

* remove max_retries from params and clean up

* add suggestions

* use llm http handler

* update doc

* add suggestions

* run formatters

* add tests

* revert

* revert

* rename file

* set maxsize for lru cache

* fix embeddings

* fix inference url

* fix tests following breaking change in main

* use ChatCompletionRequest

* fix tests and lint

* [Hugging Face] Remove outdated chat completion tests and fix embedding tests (#9749)

* remove or fix tests

* fix link in doc

* fix(config_settings.md): document hf api key

---------

Co-authored-by: célina <hanouticelina@gmail.com>
 2025-04-05 10:50:15 -07:00
Krish Dholakia
0d503ad8ad
Move daily user transaction logging outside of 'disable_spend_logs' flag - different tables (#9772)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 16s
Details
Helm unit test / unit-test (push) Successful in 18s
Details 
* refactor(db_spend_update_writer.py): aggregate table is entirely different

* test(test_db_spend_update_writer.py): add unit test to ensure if disable_spend_logs is true daily user transactions is still logged

* test: fix test
 2025-04-05 09:58:16 -07:00
Krrish Dholakia
af9db827fc fix(databricks/chat/transformation.py): handle empty headers case 2025-04-05 08:33:56 -07:00
Krish Dholakia
5099aac1a5
Add DBRX Anthropic w/ thinking + response_format support (#9744) 
* feat(databricks/chat/): add anthropic w/ reasoning content support via databricks

Allows user to call claude-3-7-sonnet with thinking via databricks

* refactor: refactor choices transformation + add unit testing

* fix(databricks/chat/transformation.py): support thinking blocks on databricks response streaming

* feat(databricks/chat/transformation.py): support response_format for claude models

* fix(databricks/chat/transformation.py): correctly handle response_format={"type": "text"}

* feat(databricks/chat/transformation.py): support 'reasoning_effort' param mapping for anthropic

* fix: fix ruff errors

* fix: fix linting error

* test: update test

* fix(databricks/chat/transformation.py): handle json mode output parsing

* fix(databricks/chat/transformation.py): handle json mode on streaming

* test: update test

* test: update dbrx testing

* test: update testing

* fix(base_model_iterator.py): handle non-json chunk

* test: update tests

* fix: fix ruff check

* fix: fix databricks config import

* fix: handle _tool = none

* test: skip invalid test
 2025-04-04 22:13:32 -07:00
Ishaan Jaff
f0f2f819bd
Merge pull request #9760 from BerriAI/litellm_prometheus_error_monitoring 
[Reliability] Prometheus emit llm provider on failure metric - make it easy to differentiate litellm error vs llm api error
 2025-04-04 21:37:28 -07:00
Ishaan Jaff
b7cd4cef07 test_get_exception_class_name 2025-04-04 21:32:55 -07:00
Ishaan Jaff
df4593d58b test prom unit tests 2025-04-04 21:30:05 -07:00
Ishaan Jaff
b89ed69257
Merge branch 'main' into litellm_add_auth_metrics_endpoint 2025-04-04 21:28:06 -07:00
Ishaan Jaff
f402e9bbd1 _get_exception_class_name 2025-04-04 21:23:21 -07:00
Ishaan Jaff
8c3670e192
Merge pull request #9719 from BerriAI/litellm_metrics_pod_lock_manager 
[Reliability] Emit operational metrics for new DB Transaction architecture
 2025-04-04 21:12:06 -07:00
Ishaan Jaff
df51d8bcfa Merge branch 'main' into litellm_metrics_pod_lock_manager 2025-04-04 21:11:39 -07:00
Ishaan Jaff
fc4c453cb9 test_no_auth_metrics_when_disabled 2025-04-04 21:02:29 -07:00
Krish Dholakia
af42e5855f
Gemini image generation output support (#9646) 
* fix(gemini/transformation.py): make GET request to get uri details, if cannot be inferred

* fix: fix linting errors

* Revert "fix: fix linting errors"

This reverts commit 926a5a527f.

* fix(gemini/transformation.py): modalities param support

Partially resolves https://github.com/BerriAI/litellm/issues/9237

* feat(google_ai_studio/): add image generation support

Closes https://github.com/BerriAI/litellm/issues/9237

* fix: fix types

* fix: fix ruff check
 2025-04-04 20:37:48 -07:00
Krish Dholakia
90a4dfab3c
fix(xai/chat/transformation.py): filter out 'name' param for xai non-… (#9761) 
* fix(xai/chat/transformation.py): filter out 'name' param for xai non-user roles

Fixes https://github.com/BerriAI/litellm/issues/9720

* test fix test_hf_chat_template

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
 2025-04-04 20:37:08 -07:00
Krrish Dholakia
b5851769fc fix: fix import 2025-04-04 20:26:11 -07:00
Krrish Dholakia
6395bd8d65 test: mark flaky test 2025-04-04 20:25:05 -07:00
Krish Dholakia
c555c15ad7
fix(router.py): support reusable credentials via passthrough router (#9758) 
* fix(router.py): support reusable credentials via passthrough router

enables reusable vertex credentials to be used in passthrough

* test: fix test

* test(test_router_adding_deployments.py): add unit testing
 2025-04-04 18:40:14 -07:00
Ishaan Jaff
150e77cd7d Merge branch 'main' into litellm_reliability_fix_db_txs 2025-04-04 16:46:46 -07:00
Ishaan Jaff
d3018a4c28 Merge branch 'main' into litellm_metrics_pod_lock_manager 2025-04-04 16:46:32 -07:00
Ishaan Jaff
5c2bc796b1 test fix test_hf_chat_template 2025-04-04 16:45:20 -07:00
Krish Dholakia
e1f7bcb47d
Fix VertexAI Credential Caching issue (#9756) 
* refactor(vertex_llm_base.py): Prevent credential misrouting for projects

Fixes https://github.com/BerriAI/litellm/issues/7904

* fix: passing unit tests

* fix(vertex_llm_base.py): common auth logic across sync + async vertex ai calls

prevents credential caching issue across both flows

* test: fix test

* fix(vertex_llm_base.py): handle project id in default cause

* fix(factory.py): don't pass cache control if not set

bedrock invoke does not support this

* test: fix test

* fix(vertex_llm_base.py): add .exception message in load_auth

* fix: fix ruff error
 2025-04-04 16:38:08 -07:00
Ishaan Jaff
1cdee4b331 Merge branch 'main' into litellm_metrics_pod_lock_manager 2025-04-04 16:33:16 -07:00
Ishaan Jaff
decb6649ec test_queue_flush_limit 2025-04-04 16:29:06 -07:00
Ishaan Jaff
e77a178a37 test_queue_size_reduction_with_large_volume 2025-04-04 16:21:29 -07:00
Ishaan Jaff
dc063fdfec test_queue_size_reduction_with_large_volume 2025-04-04 15:59:35 -07:00
Ishaan Jaff
5bed0b7557 aggregated values 2025-04-04 15:55:14 -07:00
Ishaan Jaff
cdd351a03b
Merge pull request #9745 from BerriAI/litellm_sso_fixes_dev 
[Feat] Allow assigning SSO users to teams on MSFT SSO
 2025-04-04 15:40:19 -07:00
Ishaan Jaff
888446256c fix vertex failing test 2025-04-04 15:37:48 -07:00
Krrish Dholakia
ad90871ad6 fix(factory.py): don't pass cache control if not set 
bedrock invoke does not support this
 2025-04-04 12:37:34 -07:00
Adrian Lyjak
d640bc0a00
fix #8425, passthrough kwargs during acompletion, and unwrap extra_body for openrouter (#9747) 2025-04-03 22:19:40 -07:00
Ishaan Jaff
0745f306c7 test_microsoft_sso_handler_with_empty_response 2025-04-03 22:17:06 -07:00
Albert Örwall
bd5a8d582b
Fix prompt caching for Anthropic tool calls (#9706) 
* Add prompt cache support to Anhtropic tool calls

* Fix linting issue and add test
 2025-04-03 20:19:21 -07:00
Ishaan Jaff
ef6bf02ac4 test_nova_optional_params_tool_choice
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 17s
Details
Helm unit test / unit-test (push) Successful in 23s
Details
2025-04-03 12:27:21 -07:00
sajda
4a4328b5bb
fix:Gemini Flash 2.0 implementation is not returning the logprobs (#9713) 
* fix:Gemini Flash 2.0 implementation is not returning the logprobs

* fix: linting error by adding a helper method called _process_candidates
 2025-04-03 11:53:41 -07:00
Krish Dholakia
6dda1ba6dd
LiteLLM Minor Fixes & Improvements (04/02/2025) (#9725) 
* Add date picker to usage tab + Add reasoning_content token tracking across all providers on streaming (#9722)

* feat(new_usage.tsx): add date picker for new usage tab

allow user to look back on their usage data

* feat(anthropic/chat/transformation.py): report reasoning tokens in completion token details

allows usage tracking on how many reasoning tokens are actually being used

* feat(streaming_chunk_builder.py): return reasoning_tokens in anthropic/openai streaming response

allows tracking reasoning_token usage across providers

* Fix update team metadata + fix bulk adding models on Ui  (#9721)

* fix(handle_add_model_submit.tsx): fix bulk adding models

* fix(team_info.tsx): fix team metadata update

Fixes https://github.com/BerriAI/litellm/issues/9689

* (v0) Unified file id - allow calling multiple providers with same file id (#9718)

* feat(files_endpoints.py): initial commit adding 'target_model_names' support

allow developer to specify all the models they want to call with the file

* feat(files_endpoints.py): return unified files endpoint

* test(test_files_endpoints.py): add validation test - if invalid purpose submitted

* feat: more updates

* feat: initial working commit of unified file id translation

* fix: additional fixes

* fix(router.py): remove model replace logic in jsonl on acreate_file

enables file upload to work for chat completion requests as well

* fix(files_endpoints.py): remove whitespace around model name

* fix(azure/handler.py): return acreate_file with correct response type

* fix: fix linting errors

* test: fix mock test to run on github actions

* fix: fix ruff errors

* fix: fix file too large error

* fix(utils.py): remove redundant var

* test: modify test to work on github actions

* test: update tests

* test: more debug logs to understand ci/cd issue

* test: fix test for respx

* test: skip mock respx test

fails on ci/cd - not clear why

* fix: fix ruff check

* fix: fix test

* fix(model_connection_test.tsx): fix linting error

* test: update unit tests
 2025-04-03 11:48:52 -07:00
Ishaan Jaff
e0dbd328be test_bedrock_nova_json.py
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 17s
Details
Helm unit test / unit-test (push) Successful in 23s
Details
2025-04-03 08:37:59 -07:00
Ishaan Jaff
afcd00bdc0 test_redis_caching_llm_caching_ttl 2025-04-02 21:54:35 -07:00
Ishaan Jaff
dd2d1dc2f4 Merge branch 'main' into litellm_metrics_pod_lock_manager 2025-04-02 21:35:55 -07:00
Ishaan Jaff
e68603e176 test create and update gauge 2025-04-02 21:31:19 -07:00
Krish Dholakia
8ee32291e0
Squashed commit of the following: (#9709) 
commit b12a9892b7
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Apr 2 08:09:56 2025 -0700

    fix(utils.py): don't modify openai_token_counter

commit 294de31803
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 21:22:40 2025 -0700

    fix: fix linting error

commit cb6e9fbe40
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 19:52:45 2025 -0700

    refactor: complete migration

commit bfc159172d
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 19:09:59 2025 -0700

    refactor: refactor more constants

commit 43ffb6a558
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:45:24 2025 -0700

    fix: test

commit 04dbe4310c
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:28:58 2025 -0700

    refactor: refactor: move more constants into constants.py

commit 3c26284aff
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:14:46 2025 -0700

    refactor: migrate hardcoded constants out of __init__.py

commit c11e0de69d
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:11:21 2025 -0700

    build: migrate all constants into constants.py

commit 7882bdc787
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:07:37 2025 -0700

    build: initial test banning hardcoded numbers in repo
 2025-04-02 21:24:54 -07:00
Ishaan Jaff
0155b9f212 Merge branch 'main' into litellm_refactor_pod_lock_manager 2025-04-02 21:05:18 -07:00
Ishaan Jaff
5222cce510 Merge branch 'main' into litellm_metrics_pod_lock_manager 2025-04-02 21:04:44 -07:00
Ishaan Jaff
20d84ddef1 get_base_image_generation_call_args 2025-04-02 21:04:06 -07:00
Ishaan Jaff
acf920a41a
Merge branch 'main' into litellm_fix_azure_o_series 2025-04-02 20:58:52 -07:00
Ishaan Jaff
c3341a1e18 test fixes - azure deprecated dall-e-2 2025-04-02 20:56:20 -07:00
Ishaan Jaff
74550df197 get_base_image_generation_call_args 2025-04-02 20:52:16 -07:00
Ishaan Jaff
4ed0ab5b1c Revert "remove google dns for img tests" 
This reverts commit d3fc8b563c.
 2025-04-02 20:42:29 -07:00
Ishaan Jaff
d3fc8b563c remove google dns for img tests 2025-04-02 20:34:47 -07:00