phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-25 10:44:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
21468 commits 2480 branches 894 tags 534 MiB
Commit graph

353 commits

Author SHA1 Message Date
Ishaan Jaff
9e90676058 docs prompt cache controls 2025-04-15 22:22:36 -07:00
Ishaan Jaff
c86e678809
[Docs] v1.66.0-stable fixes (#9953) 
* add categories for spend tracking improvements

* xai reasoning usage

* docs tag management

* docs tag based routing

* [Beta] Routing based

* docs tag based routing

* docs tag routing

* docs enterprise web search
 2025-04-12 16:57:25 -07:00
Ishaan Jaff
f5c5c79ea4 update docs 2025-04-10 20:18:54 -07:00
Ishaan Jaff
94a553dbb2
[Feat] Emit Key, Team Budget metrics on a cron job schedule (#9528) 
* _initialize_remaining_budget_metrics

* initialize_budget_metrics_cron_job

* initialize_budget_metrics_cron_job

* initialize_budget_metrics_cron_job

* test_initialize_budget_metrics_cron_job

* LITELLM_PROXY_ADMIN_NAME

* fix code qa checks

* test_initialize_budget_metrics_cron_job

* test_initialize_budget_metrics_cron_job

* pod lock manager allow dynamic cron job ID

* fix pod lock manager

* require cronjobid for PodLockManager

* fix DB_SPEND_UPDATE_JOB_NAME acquire / release lock

* add comment on prometheus logger

* add debug statements for emitting key, team budget metrics

* test_pod_lock_manager.py

* test_initialize_budget_metrics_cron_job

* initialize_budget_metrics_cron_job

* initialize_remaining_budget_metrics

* remove outdated test
 2025-04-10 16:59:14 -07:00
Ishaan Jaff
e6403b717c
[Security fix - CVE-2025-0330] - Leakage of Langfuse API keys in team exception handling (#9830) 
* fix team id exception in get team config

* test_team_info_masking

* test ref
 2025-04-08 13:55:20 -07:00
Ishaan Jaff
ff3a6830a4
[Feat] LiteLLM Tag/Policy Management (#9813)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 15s
Details
Helm unit test / unit-test (push) Successful in 21s
Details 
* rendering tags on UI

* use /models for building tags

* CRUD endpoints for Tag management

* fix tag management

* working api for LIST tags

* working tag management

* refactor UI components

* fixes ui tag management

* clean up ui tag management

* fix tag management ui

* fix show allowed llms

* e2e tag controls

* stash change for rendering tags on UI

* ui working tag selector on Test Key page

* fixes for tag management

* clean up tag info

* fix code quality

* test for tag management

* ui clarify what tag routing is
 2025-04-07 21:54:24 -07:00
Ishaan Jaff
b89ed69257
Merge branch 'main' into litellm_add_auth_metrics_endpoint 2025-04-04 21:28:06 -07:00
Ishaan Jaff
86b473d267 allow adding auth on /metrics endpoint 2025-04-04 20:37:17 -07:00
Ishaan Jaff
96ce5dbf7d _should_run_auth_on_metrics_endpoint 2025-04-04 20:32:04 -07:00
Ishaan Jaff
253060cb09 allow requiring auth for /metrics endpoint 2025-04-04 17:35:02 -07:00
Ishaan Jaff
901d6fe7b7 add operational metrics for pod lock manager v2 arch 2025-04-04 16:41:07 -07:00
Ishaan Jaff
5222cce510 Merge branch 'main' into litellm_metrics_pod_lock_manager 2025-04-02 21:04:44 -07:00
Ishaan Jaff
05b30e28db clean up service metrics 2025-04-02 17:50:41 -07:00
Ishaan Jaff
8b12a2e5dc fix pod lock manager 2025-04-02 14:52:55 -07:00
Ishaan Jaff
b48b8366c2 docs new deadlock fixing architecture 2025-04-02 13:24:53 -07:00
Ishaan Jaff
4ddca7a79c
Merge branch 'main' into litellm_fix_service_account_behavior 2025-04-01 12:04:28 -07:00
Ishaan Jaff
f8ce30763e fix check_if_token_is_service_account 2025-04-01 07:36:35 -07:00
Ishaan Jaff
95e674d1e9 test spend accuracy 2025-03-31 19:01:17 -07:00
Ishaan Jaff
cc80370e0c docs mcp litellm 2025-03-29 21:59:58 -07:00
Ishaan Jaff
db12adb3db docs mcp tools 2025-03-29 20:42:14 -07:00
Ishaan Jaff
e0cff7595c endpoints to list and call tools 2025-03-29 14:31:35 -07:00
Ishaan Jaff
1cf5cbab50 REST API endpoint for MCP 2025-03-29 13:35:46 -07:00
Ishaan Jaff
a1ec0dd0e2 add testing mcp server 2025-03-29 12:52:46 -07:00
Ishaan Jaff
69d5734685 fix - locking in memory leads to failing tests 2025-03-28 20:10:58 -07:00
Ishaan Jaff
ba550e2147 test local spend accuracy 2025-03-28 19:52:39 -07:00
Ishaan Jaff
fc46f6b861 fix db spend update buffer 2025-03-27 22:34:15 -07:00
Ishaan Jaff
b4e745323a add test config 2025-03-25 19:21:51 -07:00
Ishaan Jaff
7826c9bd21 add litellm mcp endpoints 2025-03-20 21:12:56 -07:00
Ishaan Jaff
f44a5a86db litellm mcp routes 2025-03-20 17:55:43 -07:00
Ishaan Jaff
9f22cf729a mock config for MCP tools 2025-03-20 17:16:55 -07:00
Ishaan Jaff
1bd7443c25
Merge pull request #9384 from BerriAI/litellm_prompt_management_custom 
[Feat] - Allow building custom prompt management integration
 2025-03-19 21:06:41 -07:00
Ishaan Jaff
805542005a CustomPromptManagement 2025-03-19 16:38:59 -07:00
Ishaan Jaff
3458c69eb0 fix common utils 2025-03-18 11:04:02 -07:00
Ishaan Jaff
98718f6f09 working responses_api 2025-03-12 17:26:12 -07:00
Ishaan Jaff
b5eeafdd72
(Docs) OpenWeb x LiteLLM Docker compose + Instructions on spend tracking + logging (#9059) 
* docs improve open web ui litellm doc

* docs openweb show teams + keys

* docs open web ui litellm
 2025-03-07 17:01:39 -08:00
Ishaan Jaff
73448412e1
ui allow ui or eu api base adding model (#9042) 2025-03-06 21:22:03 -08:00
Ishaan Jaff
958e71b906
(Docs) connect litellm to open web ui (#9040) 
* init doc

* working thinking tutorial

* docs open web ui with litellm

* minor edits

* docs one tab for tutorials
 2025-03-06 21:13:00 -08:00
Ishaan Jaff
04e839d846
(AWS Secret Manager) - Using K/V pairs in 1 AWS Secret (#9039) 
* fixes for primary_secret_kv_pairs

* _parse_primary_secret

* Using K/V pairs in 1 AWS Secret

* test_primary_secret_functionality
 2025-03-06 19:30:18 -08:00
Ishaan Jaff
b02af305de
[Feat] - Display thinking tokens on OpenWebUI (Bedrock, Anthropic, Deepseek) (#9029)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 14s
Details 
* if merge_reasoning_content_in_choices

* _optional_combine_thinking_block_in_choices

* stash changes

* working merge_reasoning_content_in_choices with bedrock

* fix litellm_params accessor

* fix streaming handler

* merge_reasoning_content_in_choices

* _optional_combine_thinking_block_in_choices

* test_bedrock_stream_thinking_content_openwebui

* merge_reasoning_content_in_choices

* fix for _optional_combine_thinking_block_in_choices

* linting error fix
 2025-03-06 18:32:58 -08:00
Ishaan Jaff
f47987e673
(Refactor) /v1/messages to follow simpler logic for Anthropic API spec (#9013) 
* anthropic_messages_handler v0

* fix /messages

* working messages with router methods

* test_anthropic_messages_handler_litellm_router_non_streaming

* test_anthropic_messages_litellm_router_non_streaming_with_logging

* AnthropicMessagesConfig

* _handle_anthropic_messages_response_logging

* working with /v1/messages endpoint

* working /v1/messages endpoint

* refactor to use router factory function

* use aanthropic_messages

* use BaseConfig for Anthropic /v1/messages

* track api key, team on /v1/messages endpoint

* fix get_logging_payload

* BaseAnthropicMessagesTest

* align test config

* test_anthropic_messages_with_thinking

* test_anthropic_streaming_with_thinking

* fix - display anthropic url for debugging

* test_bad_request_error_handling

* test_anthropic_messages_router_streaming_with_bad_request

* fix ProxyException

* test_bad_request_error_handling_streaming

* use provider_specific_header

* test_anthropic_messages_with_extra_headers

* test_anthropic_messages_to_wildcard_model

* fix gcs pub sub test

* standard_logging_payload

* fix unit testing for anthopic /v1/messages support

* fix pass through anthropic messages api

* delete dead code

* fix anthropic pass through response

* revert change to spend tracking utils

* fix get_litellm_metadata_from_kwargs

* fix spend logs payload json

* proxy_pass_through_endpoint_tests

* TestAnthropicPassthroughBasic

* fix pass through tests

* test_async_vertex_proxy_route_api_key_auth

* _handle_anthropic_messages_response_logging

* vertex_credentials

* test_set_default_vertex_config

* test_anthropic_messages_litellm_router_non_streaming_with_logging

* test_ageneric_api_call_with_fallbacks_basic

* test__aadapter_completion
 2025-03-06 00:43:08 -08:00
Ishaan Jaff
3a086cee06
(Feat) - Show Error Logs on LiteLLM UI (#8904) 
* fix test_moderations_bad_model

* use async_post_call_failure_hook

* basic logging errors in DB

* show status on ui

* show status on ui

* ui show request / response side by side

* stash fixes

* working, track raw request

* track error info in metadata

* fix showing error / request / response logs

* show traceback on error viewer

* ui with traceback of error

* fix async_post_call_failure_hook

* fix(http_parsing_utils.py): orjson can throw errors on some emoji's in text, default to json.loads

* test_get_error_information

* fix code quality

* rename proxy track cost callback test

* _should_store_errors_in_spend_logs

* feature flag error logs

* Revert "_should_store_errors_in_spend_logs"

This reverts commit 7f345df477.

* Revert "feature flag error logs"

This reverts commit 0e90c022bb.

* test_spend_logs_payload

* fix OTEL log_db_metrics

* fix import json

* fix ui linting error

* test_async_post_call_failure_hook

* test_chat_completion_bad_model_with_spend_logs

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
 2025-02-28 20:10:09 -08:00
Ishaan Jaff
7021f2f244
(Bug fix) dd-trace used by default on litellm proxy (#8817) 
* fix _should_use_dd_tracer

* fix _should_use_dd_tracer

* _should_use_dd_tracer

* _should_use_dd_tracer

* _should_use_dd_tracer

* _init_dd_tracer

* _should_use_dd_tracer

* fix should use dd-tracer

* fix dd tracer
 2025-02-25 19:54:22 -08:00
Ishaan Jaff
ccfbb77b73
(Redis fix) - use mget_non_atomic (#8682) 
* use mget_nonatomic

* redis cluster override MGET op

* fix redis cluster + MGET

* test redis cluster
 2025-02-20 17:51:31 -08:00
Ishaan Jaff
fff15543d9
(UI + Proxy) Cache Health Check Page - Cleanup/Improvements (#8665)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 14s
Details 
* fixes for redis cache ping serialization

* fix cache ping check

* fix cache health check ui

* working error details on ui

* ui expand / collapse error

* move cache health check to diff file

* fix displaying error from cache health check

* ui allow copying errors

* ui cache health fixes

* show redis details

* clean up cache health page

* ui polish fixes

* fix error handling on cache health page

* fix redis_cache_params on cache ping response

* error handling

* cache health ping response

* fx error response from cache ping

* parsedLitellmParams

* fix cache health check

* fix cache health page

* cache safely handle json dumps issues

* test caching routes

* test_primitive_types

* fix caching routes

* litellm_mapped_tests

* fix pytest-mock

* fix _serialize

* fix linting on safe dumps

* test_default_max_depth

* pip install "pytest-mock==3.12.0"

* litellm_mapped_tests_coverage

* add readme on new litellm test dir
 2025-02-19 19:08:50 -08:00
Ishaan Jaff
00c596a852
(Feat) - Allow viewing Request/Response Logs stored in GCS Bucket (#8449) 
* BaseRequestResponseFetchFromCustomLogger

* get_active_base_request_response_fetch_from_custom_logger

* get_request_response_payload

* ui_view_request_response_for_request_id

* fix uiSpendLogDetailsCall

* fix get_request_response_payload

* ui fix RequestViewer

* use 1 class AdditionalLoggingUtils

* ui_view_request_response_for_request_id

* cache the prefetch logs details

* refactor prefetch

* test view request/resp logs

* fix code quality

* fix get_request_response_payload

* uninstall posthog
prevent it from being added in ci/cd

* fix posthog

* fix traceloop test

* fix linting error
 2025-02-10 20:38:55 -08:00
Ishaan Jaff
64a4229606
(e2e testing) - add tests for using litellm /team/ updates in multi-instance deployments with Redis (#8440) 
* add team block/unblock test

* test_team_blocking_behavior_multi_instance

* proxy_multi_instance_tests

* test - Run Docker container 2
 2025-02-10 19:33:27 -08:00
Ishaan Jaff
7e1b79d446
(Bug fix) - Langfuse / Callback settings stored in DB (#8251) 
* fix _decrypt_and_set_db_env_variables

* fix proxy config

* test callbacks in DB

* test langfuse callbacks in db

* test_e2e_langfuse_callbacks_in_db

* proxy_store_model_in_db_tests

* fix proxy_store_model_in_db_tests

* proxy_store_model_in_db_tests

* fix store_model_db_config.yaml

* fix check_langfuse_request

* fix test langfuse base url

* ci/cd run again
 2025-02-04 21:09:37 -08:00
Ishaan Jaff
c501ec9b6d
(UI) Fix SpendLogs page - truncate bedrock models + show end_user (#8118) 
* ui spend logs table truncate bedrock page

* ui - show user / internal user fields
 2025-01-30 13:59:13 -08:00
Ishaan Jaff
892581ffc3
(UI) - View Logs Page - Refinement (#8087)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 14s
Details 
* working refetch interval

* ui show provider logo in SpendLogs Table

* fix padding

* improve time range filter

* ui fix diff minutes

* fix refresh button placement
 2025-01-29 08:46:20 -08:00
Ishaan Jaff
c0cc2ea4bb
(UI) - Adding new models enhancement - show provider logo (#8033) 
* ui allow wildcard models

* ui show model dashboard

* add advanced settings in card

* fix button

* ui - add provider logos on admin ui
 2025-01-27 13:15:42 -08:00