Commit graph

262 commits

Author SHA1 Message Date
Krrish Dholakia
db92956ae3 fix(redis_cache.py): add 5s default timeout 2025-03-17 14:27:36 -07:00
Krrish Dholakia
c0a76427d2 fix(streaming_handler.py): pass complete streaming response on completion 2025-03-16 20:22:12 -07:00
Krrish Dholakia
612d5a284d refactor(litellm_logging.py): delegate returning a complete response to the streaming_handler
Removes incorrect logic for calculating complete streaming response from litellm logging
2025-03-15 09:55:33 -07:00
Krrish Dholakia
cc82d42d25 test(test_streaming_handler.py): add unit test to ensure model response stream with usage is always used 2025-03-15 09:35:48 -07:00
Krrish Dholakia
82252ecac1 test(test_streaming_handler.py): assert chunk is non-empty when usage block given 2025-03-15 09:26:01 -07:00
Krish Dholakia
d4caaae1be
Merge pull request #9274 from BerriAI/litellm_contributor_rebase_branch
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 43s
Helm unit test / unit-test (push) Successful in 50s
Litellm contributor rebase branch
2025-03-14 21:57:49 -07:00
Krrish Dholakia
8a6e4715aa feat(converse_transformation.py): fix type for bedrock cache usage block 2025-03-13 19:33:22 -07:00
Krrish Dholakia
96bba9354e test(tests/litellm): add unit test for transform usage function 2025-03-13 15:56:55 -07:00
Krish Dholakia
2c011d9a93
Merge pull request #9123 from omrishiv/8911-fix-model-encoding
Fixes bedrock modelId encoding for Inference Profiles
2025-03-13 10:42:32 -07:00
sven
2d4e58cf23 add test code 2025-03-13 14:00:12 +09:00
Ishaan Jaff
d623183eee test_ensure_initialize_azure_sdk_client_always_used 2025-03-12 18:51:19 -07:00
Krish Dholakia
2d957a0ed9
Merge branch 'main' into litellm_dev_03_10_2025_p3 2025-03-12 14:56:01 -07:00
Ishaan Jaff
2460f3cbab test_validate_environment 2025-03-12 12:57:40 -07:00
Ishaan Jaff
f88380cfdf TestResponseAPILoggingUtils 2025-03-12 12:45:03 -07:00
Ishaan Jaff
f6f5420f0a TestResponsesAPIRequestUtils 2025-03-12 12:42:20 -07:00
Krrish Dholakia
23bf7b5700 fix(azure/completions): migrate completions endpoint to support base azure llm class
enables consistent auth logic across all azure calls
2025-03-11 18:47:30 -07:00
Krrish Dholakia
42af49cd87 fix: fix merge conflicts 2025-03-11 18:41:41 -07:00
Krrish Dholakia
687b2e6300 test: fix test 2025-03-11 18:13:27 -07:00
Krrish Dholakia
cbc2e84044 refactor(azure.py): refactor to have client init work across all endpoints 2025-03-11 17:27:24 -07:00
Krrish Dholakia
d99d60a182 refactor(batches/main.py): working refactored azure client init on batches 2025-03-11 14:36:38 -07:00
Krrish Dholakia
af71e14d79 refactor(azure/audio_transcriptions.py): support client init with common logic 2025-03-11 14:24:12 -07:00
Krrish Dholakia
152bc67d22 refactor(azure.py): working azure client init on audio speech endpoint 2025-03-11 14:19:45 -07:00
Krrish Dholakia
f7d9cce536 refactor(azure.py): refactor acompletion to use base azure sdk client 2025-03-11 13:59:13 -07:00
Krrish Dholakia
b58edb7fa1 test(test_azure_common_utils.py): add unit testing for common azure client params function 2025-03-11 12:24:08 -07:00
omrishiv
e2adbae9f8 Merge branch 'main' into 8911-fix-model-encoding 2025-03-11 08:28:33 -07:00
omrishiv
d25693b96c update test
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
2025-03-10 20:34:52 -07:00
omrishiv
338722beba add test
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
2025-03-10 20:28:55 -07:00
Utkash Dubey
074c8efdc1 cleanup 2025-03-10 20:13:03 -07:00
Utkash Dubey
14736283bc test original file instead of the backup 2025-03-10 20:09:05 -07:00
Utkash Dubey
7dbc434318 remove incorrect moderations mode 2025-03-10 19:33:40 -07:00
Utkash Dubey
a9d50a615b move test file to correct location 2025-03-10 19:21:51 -07:00
vivek-athina
cd4a53d6f2
Merge pull request #4 from BerriAI/main
Update main
2025-03-10 11:13:21 +05:30
Teja Vishwanadha
8c049dfffc
support bytes.IO for audio transcription (#9071) 2025-03-08 08:47:15 -08:00
Ishaan Jaff
e2d612efd9
Bug fix - String data: stripped from entire content in streamed Gemini responses (#9070)
* _strip_sse_data_from_chunk

* use _strip_sse_data_from_chunk

* use _strip_sse_data_from_chunk

* use _strip_sse_data_from_chunk

* _strip_sse_data_from_chunk

* test_strip_sse_data_from_chunk

* _strip_sse_data_from_chunk

* testing

* _strip_sse_data_from_chunk
2025-03-07 21:06:39 -08:00
Ishaan Jaff
b02af305de
[Feat] - Display thinking tokens on OpenWebUI (Bedrock, Anthropic, Deepseek) (#9029)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 14s
* if merge_reasoning_content_in_choices

* _optional_combine_thinking_block_in_choices

* stash changes

* working merge_reasoning_content_in_choices with bedrock

* fix litellm_params accessor

* fix streaming handler

* merge_reasoning_content_in_choices

* _optional_combine_thinking_block_in_choices

* test_bedrock_stream_thinking_content_openwebui

* merge_reasoning_content_in_choices

* fix for _optional_combine_thinking_block_in_choices

* linting error fix
2025-03-06 18:32:58 -08:00
Krish Dholakia
f6535ae6ad
Support format param for specifying image type (#9019)
* fix(transformation.py): support a 'format' parameter for image's

allow user to specify mime type

* fix: pass mimetype via 'format' param

* feat(gemini/chat/transformation.py): support 'format' param for gemini

* fix(factory.py): support 'format' param on sync bedrock converse calls

* feat(bedrock/converse_transformation.py): support 'format' param for bedrock async calls

* refactor(factory.py): move to supporting 'format' param in base helper

ensures consistency in param support

* feat(gpt_transformation.py): filter out 'format' param

don't send invalid param to openai

* fix(gpt_transformation.py): fix translation

* fix: fix translation error
2025-03-05 19:52:53 -08:00
Ishaan Jaff
8d6815ce98 Revert "(UI) - Security Improvement, move to JWT Auth for Admin UI Sessions (#8995)"
This reverts commit 01a44a4e47.
2025-03-05 08:49:20 -08:00
Ishaan Jaff
01a44a4e47
(UI) - Security Improvement, move to JWT Auth for Admin UI Sessions (#8995)
* (UI) - Improvements to session handling logic  (#8970)

* add cookieUtils

* use utils for clearing cookies

* on logout use clearTokenCookies

* ui use correct clearTokenCookies

* navbar show userEmail on UserID page

* add timestamp on token cookie

* update generate_authenticated_redirect_response

* use common getAuthToken

* fix clearTokenCookies

* fixes for get auth token

* fix invitation link sign in logic

* Revert "fix invitation link sign in logic"

This reverts commit 30e5308cb3.

* fix getAuthToken

* update setAuthToken

* fix ui session handling

* fix ui session handler

* bug fix stop generating LiteLLM Virtual keys for access

* working JWT insert into cookies

* use central place to build UI JWT token

* add _validate_ui_token

* fix ui session handler

* fix fetchWithCredentials

* check allowed routes for ui session tokens

* expose validate_session endpoint

* validate session endpoint

* call sso/session/validate

* getUISessionDetails

* ui move to getUISessionDetails

* /sso/session/validate

* fix cookie utils

* use getUISessionDetails

* use ui_session_id

* "/spend/logs/ui" in spend_tracking_routes

* working sign in JWT flow for proxy admin

* allow proxy admin to access ui routes

* use check_route_access

* update types

* update login method

* fixes to ui session handler

* working flow for admin and internal users

* fixes for invite links

* use JWTs for SSO sign in

* fix /invitation/new flow

* fix code quality checks

* fix _get_ui_session_token_from_cookies

* /organization/list

* ui sso sign in

* TestUISessionHandler

* TestUISessionHandler
2025-03-04 21:48:23 -08:00
Ishaan Jaff
42931638df
(bug fix) - Fix Cache Health Check for Redis when redis_version is float (#8979)
* fix allow flexible types for redis version

* test_cache_ping_with_redis_version_float

* test_cache_ping_with_redis_version_float
2025-03-04 21:26:18 -08:00
Krish Dholakia
2fc6262675
fix(route_llm_request.py): move to using common router, even for clie… (#8966)
* fix(route_llm_request.py): move to using common router, even for client-side credentials

ensures fallbacks / cooldown logic still works

* test(test_route_llm_request.py): add unit test for route request

* feat(router.py): generate unique model id when clientside credential passed in

Prevents cooldowns for api key 1 from impacting api key 2

* test(test_router.py): update testing to ensure original litellm params not mutated

* fix(router.py): upsert clientside call into llm router model list

enables cooldown logic to work accurately

* fix: fix linting error

* test(test_router_utils.py): add direct test for new util on router
2025-03-03 22:57:08 -08:00
Krish Dholakia
94d28d59e4
Fix deepseek 'reasoning_content' error (#8963)
* fix(streaming_handler.py): fix deepseek reasoning content streaming

Fixes https://github.com/BerriAI/litellm/issues/8939

* test(test_streaming_handler.py): add unit test to streaming handle 'is_chunk_non_empty' function

ensures 'reasoning_content' is handled correctly
2025-03-03 14:34:10 -08:00
Krish Dholakia
54b7f17ca6
fix(proxy_server.py): fix setting router redis cache, if cache enable… (#8859)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 14s
* fix(proxy_server.py): fix setting router redis cache, if cache enabled on litellm_settings

enables configurations like namespace to just work

* fix(redis_cache.py): fix key for async increment, to use the set namespace

prevents collisions if redis instance shared across environments

* fix load tests on litellm release notes

* fix caching on main branch (#8858)

* fix(streaming_handler.py): fix is delta empty check to handle empty str

* fix(streaming_handler.py): fix delta chunk on final response

* [Bug]: Deepseek error on proxy after upgrading to 1.61.13-stable (#8860)

* fix deepseek error

* test_deepseek_provider_async_completion

* fix get_complete_url

* bump: version 1.61.17 → 1.61.18

* bump: version 1.61.18 → 1.61.19

* vertex ai anthropic thinking param support (#8853)

* fix(vertex_llm_base.py): handle credentials passed in as dictionary

* fix(router.py): support vertex credentials as json dict

* test(test_vertex.py): allows easier testing

mock anthropic thinking response for vertex ai

* test(vertex_ai_partner_models/): don't remove "@" from model

breaks anthropic cost calculation

* test: move testing

* fix: fix linting error

* fix: fix linting error

* fix(vertex_ai_partner_models/main.py): split @ for codestral model

* test: fix test

* fix: fix stripping "@" on mistral models

* fix: fix test

* test: fix test

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-03-02 08:39:06 -08:00
Ishaan Jaff
df095b6022
(UI) - Allow Internal Users to View their own logs (#8933)
* ui fix leftnav, allow internal users to view their own logs

* pass user_id in uiSpendLogs call

* ui filter logs for internal user

* fix internal users page

* ui show correct message when store prompts is disabled

* fix internal user logs

* test_ui_view_spend_logs_with_user_id

* test spend management endpoint
2025-03-01 17:26:02 -08:00
Ishaan Jaff
1008da7c41
(UI) Error Logs improvements - Store Raw proxy server request for success and failure (#8917)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 13s
* ui render error logs

* add proxy_server_request to spend logs metadata

* _add_proxy_server_request_to_metadata

* proxy track cost callback fixes

* _add_proxy_server_request_to_metadata

* test spend logs

* test_async_post_call_failure_hook
2025-03-01 16:26:47 -08:00
Ishaan Jaff
ee7cd60fdb Revert "(bug fix) - don't log messages, prompt, input in model_parameters in StandardLoggingPayload (#8923)"
This reverts commit a119cb420b.
2025-03-01 11:05:33 -08:00
Ishaan Jaff
6fc9aa1612
(bug fix) - dd tracer, only send traces when user opts into sending dd-trace (#8928)
* fix dd tracing null tracer bug

* fix dd tracing

* fix base aws llm

* test_should_use_dd_tracer
2025-03-01 10:53:36 -08:00
Ishaan Jaff
a119cb420b
(bug fix) - don't log messages, prompt, input in model_parameters in StandardLoggingPayload (#8923)
* fix _get_model_parameters

* test litellm logging

* test litellm logging
2025-03-01 10:27:24 -08:00
Ishaan Jaff
3a086cee06
(Feat) - Show Error Logs on LiteLLM UI (#8904)
* fix test_moderations_bad_model

* use async_post_call_failure_hook

* basic logging errors in DB

* show status on ui

* show status on ui

* ui show request / response side by side

* stash fixes

* working, track raw request

* track error info in metadata

* fix showing error / request / response logs

* show traceback on error viewer

* ui with traceback of error

* fix async_post_call_failure_hook

* fix(http_parsing_utils.py): orjson can throw errors on some emoji's in text, default to json.loads

* test_get_error_information

* fix code quality

* rename proxy track cost callback test

* _should_store_errors_in_spend_logs

* feature flag error logs

* Revert "_should_store_errors_in_spend_logs"

This reverts commit 7f345df477.

* Revert "feature flag error logs"

This reverts commit 0e90c022bb.

* test_spend_logs_payload

* fix OTEL log_db_metrics

* fix import json

* fix ui linting error

* test_async_post_call_failure_hook

* test_chat_completion_bad_model_with_spend_logs

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
2025-02-28 20:10:09 -08:00
Vivek Aditya
ed75dd61c2 Removed prints and added unit tests 2025-02-28 21:48:13 +05:30
Ishaan Jaff
51a6a219cd
(Improvements) use /openai/ pass through with OpenAI Ruby for Assistants API (#8884)
* add ruby assistants testing

* _join_url_paths

* run ruby tests on ci/cd

* TestBaseOpenAIPassThroughHandler

* _join_url_paths

* fix _join_url_paths

* Install Ruby and Bundler

* Install Ruby and Bundler
2025-02-27 20:01:16 -08:00