phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-24 18:24:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
21590 commits 2475 branches 894 tags 524 MiB
Commit graph

21590 commits

This branch
This branch
All branches
Author SHA1 Message Date
Krish Dholakia
78879c68a9
Revert avglogprobs change + Add azure/gpt-4o-realtime-audio cost tracking (#9893) 
* test: initial commit fixing gemini logprobs

Fixes https://github.com/BerriAI/litellm/issues/9888

* fix(vertex_and_google_ai_studio.py): Revert avglogprobs change

Fixes https://github.com/BerriAI/litellm/issues/8890

* build(model_prices_and_context_window.json): add gpt-4o-realtime-preview cost to model cost map

Fixes https://github.com/BerriAI/litellm/issues/9814

* test: add cost calculation unit testing

* test: fix test

* test: update test
 2025-04-10 21:23:55 -07:00
Ishaan Jaff
892964272f docs msft SSO
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 21s
Details
Helm unit test / unit-test (push) Successful in 25s
Details
2025-04-10 20:33:14 -07:00
Ishaan Jaff
c71e98b05a docs MSFT sso 2025-04-10 20:30:52 -07:00
Ishaan Jaff
9ebbf52249 docs self serve msft SSO 2025-04-10 20:25:43 -07:00
Ishaan Jaff
1197746ab3 bump: version 1.65.6 → 1.65.7 2025-04-10 20:23:08 -07:00
Ishaan Jaff
595c0cbb42 docs msft default team settings 2025-04-10 20:22:46 -07:00
Ishaan Jaff
34b1bf2c75 docs msft_default_settings 2025-04-10 20:21:14 -07:00
Ishaan Jaff
f5c5c79ea4 update docs 2025-04-10 20:18:54 -07:00
Ishaan Jaff
98e34cbf5d
[Docs] Tutorial using MSFT auto team assignment with LiteLLM (#9898) 
* add default_team_params as a config.yaml setting

* create_litellm_team_from_sso_group

* test_default_team_params

* test_create_team_without_default_params

* docs default team settings

* docs msft entra id tutorial

* commit litellm docs msft group assignment

* litellm MSFT sso

* member, team assignment on litellm

* docs msft auto assignment

* bug fix default team setting

* docs litellm default team settings

* test_default_team_params
 2025-04-10 20:07:55 -07:00
Ishaan Jaff
72a12e91c4
[Bug Fix MSFT SSO] Use correct field for user email when using MSFT SSO (#9886) 
* fix openid_from_response

* test_microsoft_sso_handler_openid_from_response_user_principal_name

* test upsert_sso_user
 2025-04-10 17:40:58 -07:00
Ishaan Jaff
94a553dbb2
[Feat] Emit Key, Team Budget metrics on a cron job schedule (#9528) 
* _initialize_remaining_budget_metrics

* initialize_budget_metrics_cron_job

* initialize_budget_metrics_cron_job

* initialize_budget_metrics_cron_job

* test_initialize_budget_metrics_cron_job

* LITELLM_PROXY_ADMIN_NAME

* fix code qa checks

* test_initialize_budget_metrics_cron_job

* test_initialize_budget_metrics_cron_job

* pod lock manager allow dynamic cron job ID

* fix pod lock manager

* require cronjobid for PodLockManager

* fix DB_SPEND_UPDATE_JOB_NAME acquire / release lock

* add comment on prometheus logger

* add debug statements for emitting key, team budget metrics

* test_pod_lock_manager.py

* test_initialize_budget_metrics_cron_job

* initialize_budget_metrics_cron_job

* initialize_remaining_budget_metrics

* remove outdated test
 2025-04-10 16:59:14 -07:00
Ishaan Jaff
90d862b041
[Feat SSO] - Allow admins to set default_team_params to have default params for when litellm SSO creates default teams (#9895) 
* add default_team_params as a config.yaml setting

* create_litellm_team_from_sso_group

* test_default_team_params

* test_create_team_without_default_params

* docs default team settings
 2025-04-10 16:58:28 -07:00
Krrish Dholakia
7d383fc0c1 test: update testing 2025-04-10 14:15:58 -07:00
Krrish Dholakia
b168f8b744 test: update test 2025-04-10 14:04:57 -07:00
Krrish Dholakia
cd878bdd71 bump: version 1.65.5 → 1.65.6
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 17s
Details
Helm unit test / unit-test (push) Successful in 24s
Details
2025-04-09 22:32:50 -07:00
Krish Dholakia
0dbd663877
fix(cost_calculator.py): handle custom pricing at deployment level fo… (#9855) 
* fix(cost_calculator.py): handle custom pricing at deployment level for router

* test: add unit tests

* fix(router.py): show custom pricing on UI

check correct model str

* fix: fix linting error

* docs(custom_pricing.md): clarify custom pricing for proxy

Fixes https://github.com/BerriAI/litellm/issues/8573#issuecomment-2790420740

* test: update code qa test

* fix: cleanup traceback

* fix: handle litellm param custom pricing

* test: update test

* fix(cost_calculator.py): add router model id to list of potential model names

* fix(cost_calculator.py): fix router model id check

* fix: router.py - maintain older model registry approach

* fix: fix ruff check

* fix(router.py): router get deployment info

add custom values to mapped dict

* test: update test

* fix(utils.py): update only if value is non-null

* test: add unit test
 2025-04-09 22:13:10 -07:00
Krish Dholakia
0c5b4aa96d
feat(realtime/): add token tracking + log usage object in spend logs … (#9843) 
* feat(realtime/): add token tracking + log usage object in spend logs metadata

* test: fix test

* test: update tests

* test: update testing

* test: update test

* test: update test

* test: update test

* test: update test

* test: update tesdt

* test: update test
 2025-04-09 22:11:00 -07:00
Krish Dholakia
87733c8193
Fix anthropic prompt caching cost calc + trim logged message in db (#9838) 
* fix(spend_tracking_utils.py): prevent logging entire mp4 files to db

Fixes https://github.com/BerriAI/litellm/issues/9732

* fix(anthropic/chat/transformation.py): Fix double counting cache creation input tokens

Fixes https://github.com/BerriAI/litellm/issues/9812

* refactor(anthropic/chat/transformation.py): refactor streaming to use same usage calculation block as non-streaming

reduce errors

* fix(bedrock/chat/converse_transformation.py): don't increment prompt tokens with cache_creation_input_tokens

* build: remove redisvl from requirements.txt (temporary)

* fix(spend_tracking_utils.py): handle circular references

* test: update code cov test

* test: update test
 2025-04-09 21:26:43 -07:00
Ishaan Jaff
00c5c23d97 docs Microsoft Entra ID SSO group assignment 2025-04-09 21:07:47 -07:00
Ishaan Jaff
aed8d4ce21 bump: version 1.65.4 → 1.65.5
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 14s
Details
Helm unit test / unit-test (push) Successful in 23s
Details
2025-04-09 20:34:51 -07:00
Ishaan Jaff
1359e6d7a6
[SSO] Connect LiteLLM to Azure Entra ID Enterprise Application (#9872) 
* refactor SSO handler

* render sso JWT on ui

* docs debug sso

* fix sso login flow use await

* fix ui sso debug JWT

* test ui sso

* remove redis vl

* fix redisvl==0.5.1

* fix ml dtypes

* fix redisvl

* fix redis vl

* fix debug_sso_callback

* fix linting error

* fix redis semantic caching dep

* working graph api assignment

* test msft sso handler openid

* testing for msft group assignment

* fix debug graph api sso flow

* fix linting errors

* add_user_to_teams_from_sso_response

* ui sso fix team assignments

* linting fix _get_group_ids_from_graph_api_response

* add MicrosoftServicePrincipalTeam

* create_litellm_teams_from_service_principal_team_ids

* create_litellm_teams_from_service_principal_team_ids

* docs MICROSOFT_SERVICE_PRINCIPAL_ID

* fix linting errors
 2025-04-09 20:26:59 -07:00
Krish Dholakia
ac4f32fb1e
Cost tracking for gemini-2.5-pro (#9837) 
* build(model_prices_and_context_window.json): add google/gemini-2.0-flash-lite-001 versioned pricing

Closes https://github.com/BerriAI/litellm/issues/9829

* build(model_prices_and_context_window.json): add initial support for 'supported_output_modalities' param

* build(model_prices_and_context_window.json): add initial support for 'supported_output_modalities' param

* build(model_prices_and_context_window.json): add supported endpoints to gemini-2.5-pro

* build(model_prices_and_context_window.json): add gemini 200k+ pricing

* feat(utils.py): support cost calculation for gemini-2.5-pro above 200k tokens

Fixes https://github.com/BerriAI/litellm/issues/9807

* build: test dockerfile change

* build: revert apk change

* ci(config.yml): pip install wheel

* ci: test problematic package first

* ci(config.yml): pip install only binary

* ci: try more things

* ci: test different ml_dtypes version

* ci(config.yml): check ml_dtypes==0.4.0

* ci: test

* ci: cleanup config.yml

* ci: specify ml dtypes in requirements.txt

* ci: remove redisvl depedency (temporary)

* fix: fix linting errors

* test: update test

* test: fix test
 2025-04-09 18:48:43 -07:00
Ishaan Jaff
4c1bb74c3d
[Feat] - SSO - Use MSFT Graph API to assign users to teams (#9865) 
* refactor SSO handler

* render sso JWT on ui

* docs debug sso

* fix sso login flow use await

* fix ui sso debug JWT

* test ui sso

* remove redis vl

* fix redisvl==0.5.1

* fix ml dtypes

* fix redisvl

* fix redis vl

* fix debug_sso_callback

* fix linting error

* fix redis semantic caching dep

* working graph api assignment

* test msft sso handler openid

* testing for msft group assignment

* fix debug graph api sso flow

* fix linting errors

* add_user_to_teams_from_sso_response

* fix linting error
 2025-04-09 18:26:43 -07:00
Krrish Dholakia
a1433da4a7 fix: transform_request.tsx 
don't hardcode to localhost
 2025-04-09 17:50:13 -07:00
Krrish Dholakia
86bfb8cd66 Revert "docs: initial commit adding api playground to docs" 
This reverts commit 9d68008152.
 2025-04-09 17:50:13 -07:00
Krrish Dholakia
5ca93a1950 docs: initial commit adding api playground to docs 
makes it easy to see how litellm transforms your request
 2025-04-09 17:50:13 -07:00
Krrish Dholakia
3f3afabda9 feat(leftnav.tsx): show api playground on UI 
allows easy testing on UI
 2025-04-09 17:50:13 -07:00
Krrish Dholakia
b11c08bde3 fix(new_usage.tsx): increase page size + iterate through all pages if multiple pages 2025-04-09 17:50:13 -07:00
Krrish Dholakia
9ec1972926 fix(internal_user_endpoints.py): increase default page size for /user/daily/activity 2025-04-09 17:50:13 -07:00
Ishaan Jaff
6f7e9b9728
[Feat SSO] Debug route - allow admins to debug SSO JWT fields (#9835) 
* refactor SSO handler

* render sso JWT on ui

* docs debug sso

* fix sso login flow use await

* fix ui sso debug JWT

* test ui sso

* remove redis vl

* fix redisvl==0.5.1

* fix ml dtypes

* fix redisvl

* fix redis vl

* fix debug_sso_callback

* fix linting error

* fix redis semantic caching dep
 2025-04-09 15:29:35 -07:00
Ishaan Jaff
08a3620414
[Bug Fix] Add support for UploadFile on LLM Pass through endpoints (OpenAI, Azure etc) (#9853) 
* http passthrough file handling

* fix make_multipart_http_request

* test_pass_through_file_operations

* unit tests for file handling
 2025-04-09 15:29:20 -07:00
Krish Dholakia
6ba3c4a4f8
VertexAI non-jsonl file storage support (#9781) 
* test: add initial e2e test

* fix(vertex_ai/files): initial commit adding sync file create support

* refactor: initial commit of vertex ai non-jsonl files reaching gcp endpoint

* fix(vertex_ai/files/transformation.py): initial working commit of non-jsonl file call reaching backend endpoint

* fix(vertex_ai/files/transformation.py): working e2e non-jsonl file upload

* test: working e2e jsonl call

* test: unit testing for jsonl file creation

* fix(vertex_ai/transformation.py): reset file pointer after read

allow multiple reads on same file object

* fix: fix linting errors

* fix: fix ruff linting errors

* fix: fix import

* fix: fix linting error

* fix: fix linting error

* fix(vertex_ai/files/transformation.py): fix linting error

* test: update test

* test: update tests

* fix: fix linting errors

* fix: fix test

* fix: fix linting error
 2025-04-09 14:01:48 -07:00
qvalentin
93532e00db
feat: add enterpriseWebSearch tool for vertex-ai (#9856) 2025-04-09 13:17:48 -07:00
Emerson Gomes
d5e362459c
Update Azure Phi-4 pricing (#9862) 
Updates Phi-4 family model prices with recently published info
 2025-04-09 13:17:00 -07:00
Jacob Hagstedt P Suorra
dc9bfae053
Add user alias to API endpoint (#9859) 
Co-authored-by: Jacob Hagstedt <wcgs@novonordisk.com>
 2025-04-09 13:16:35 -07:00
Christian Owusu
d4e5da87be
Reflect key and team update in UI (#9825)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 16s
Details
Helm unit test / unit-test (push) Successful in 23s
Details 
* Reflect updates to keys in UI instantly

* Reflect updates to teams in UI instantly
 2025-04-09 07:47:16 -07:00
Marcus Hynfield
cc7d59a11e
Add service annotations to litellm-helm chart (#9840)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 15s
Details
Helm unit test / unit-test (push) Successful in 22s
Details
2025-04-08 21:42:09 -07:00
Ishaan Jaff
357f081d1c fix mldtypes dep
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 14s
Details
Helm unit test / unit-test (push) Successful in 23s
Details
2025-04-08 17:25:28 -07:00
Ishaan Jaff
9f33e9b3e8 pin ml-dtypes==0.4.0 2025-04-08 17:24:08 -07:00
Li Yang
11389535d5
chore: fix haiku cache read pricing per token (#9834) 2025-04-08 16:43:09 -07:00
Krrish Dholakia
a3ea079583 docs(gemini.md): show how to call google search via litellm 
Addresses https://github.com/BerriAI/litellm/issues/361#issuecomment-2787497217
 2025-04-08 16:41:24 -07:00
Ishaan Jaff
c403dfb615 pip install --upgrade pip wheel setuptools 2025-04-08 16:38:44 -07:00
Ishaan Jaff
8a596dbe8c pip install wheel 2025-04-08 16:27:09 -07:00
dependabot[bot]
73356b3a9f
Bump next from 14.2.25 to 14.2.26 in /ui/litellm-dashboard (#9716) 
Bumps [next](https://github.com/vercel/next.js) from 14.2.25 to 14.2.26.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v14.2.25...v14.2.26)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 14.2.26
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-04-08 14:15:19 -07:00
Ishaan Jaff
441c7275ed
test fix post call rules (#9826) 2025-04-08 13:55:37 -07:00
Ishaan Jaff
e6403b717c
[Security fix - CVE-2025-0330] - Leakage of Langfuse API keys in team exception handling (#9830) 
* fix team id exception in get team config

* test_team_info_masking

* test ref
 2025-04-08 13:55:20 -07:00
Krrish Dholakia
367f48004d build(model_prices_and_context_window.json): consistent params 2025-04-08 12:45:33 -07:00
Peter Dave Hello
6b67006b0c
Remove redundant apk update in Dockerfiles (cc #5016) (#9055)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 17s
Details
Helm unit test / unit-test (push) Successful in 18s
Details 
The `apk` commands can utilize the `--no-cache` option, making the
`update` step superfluous and ensuring the latest packages are used
without maintaining a local cache. An additional `apk update` in the
Dockerfile will just make the image larger with no benefits.
 2025-04-08 09:03:25 -07:00
Ishaan Jaff
ff3a6830a4
[Feat] LiteLLM Tag/Policy Management (#9813)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 15s
Details
Helm unit test / unit-test (push) Successful in 21s
Details 
* rendering tags on UI

* use /models for building tags

* CRUD endpoints for Tag management

* fix tag management

* working api for LIST tags

* working tag management

* refactor UI components

* fixes ui tag management

* clean up ui tag management

* fix tag management ui

* fix show allowed llms

* e2e tag controls

* stash change for rendering tags on UI

* ui working tag selector on Test Key page

* fixes for tag management

* clean up tag info

* fix code quality

* test for tag management

* ui clarify what tag routing is
 2025-04-07 21:54:24 -07:00
Krish Dholakia
ac9f03beae
Allow passing thinking param to litellm proxy via client sdk + Code QA Refactor on get_optional_params (get correct values) (#9386) 
* fix(litellm_proxy/chat/transformation.py): support 'thinking' param

Fixes https://github.com/BerriAI/litellm/issues/9380

* feat(azure/gpt_transformation.py): add azure audio model support

Closes https://github.com/BerriAI/litellm/issues/6305

* fix(utils.py): use provider_config in common functions

* fix(utils.py): add missing provider configs to get_chat_provider_config

* test: fix test

* fix: fix path

* feat(utils.py): make bedrock invoke nova config baseconfig compatible

* fix: fix linting errors

* fix(azure_ai/transformation.py): remove buggy optional param filtering for azure ai

Removes incorrect check for support tool choice when calling azure ai - prevented calling models with response_format unless on litell model cost map

* fix(amazon_cohere_transformation.py): fix bedrock invoke cohere transformation to inherit from coherechatconfig

* test: fix azure ai tool choice mapping

* fix: fix model cost map to add 'supports_tool_choice' to cohere models

* fix(get_supported_openai_params.py): check if custom llm provider in llm providers

* fix(get_supported_openai_params.py): fix llm provider in list check

* fix: fix ruff check errors

* fix: support defs when calling bedrock nova

* fix(factory.py): fix test
 2025-04-07 21:04:11 -07:00