Commit graph

21459 commits

Author SHA1 Message Date
Ishaan Jaff
690c5ac3c0 Merge branch 'main' into litellm_ui_cache_control_hook_anthropic 2025-04-14 21:16:53 -07:00
Ishaan Jaff
6cfa50d278
[Feat] Add support for cache_control_injection_points for Anthropic API, Bedrock API (#9996)
* test_anthropic_cache_control_hook_system_message

* test_anthropic_cache_control_hook.py

* should_run_prompt_management_hooks

* fix should_run_prompt_management_hooks

* test_anthropic_cache_control_hook_specific_index

* fix test

* fix linting errors

* ChatCompletionCachedContent
2025-04-14 20:50:13 -07:00
Ishaan Jaff
7fabee3eff fix get_chat_completion_prompt 2025-04-14 20:05:06 -07:00
Ishaan Jaff
903f792777 fix linting errors 2025-04-14 20:03:39 -07:00
Krish Dholakia
2ed593e052
Updated cohere v2 passthrough (#9997)
* Add cohere `/v2/chat` pass-through cost tracking support (#8235)

* feat(cohere_passthrough_handler.py): initial working commit with cohere passthrough cost tracking

* fix(v2_transformation.py): support cohere /v2/chat endpoint

* fix: fix linting errors

* fix: fix import

* fix(v2_transformation.py): fix linting error

* test: handle openai exception change
2025-04-14 19:51:01 -07:00
Ishaan Jaff
9b714ae230 fix linting errors 2025-04-14 19:46:46 -07:00
Ishaan Jaff
861a4a3ea3 fix init custom logger compatible class 2025-04-14 19:33:07 -07:00
Ishaan Jaff
b8438a3f40 update cache control settings on edit model page 2025-04-14 19:32:08 -07:00
Ishaan Jaff
a81f7300b9 fixes for using cache control on ui + backend 2025-04-14 19:21:29 -07:00
Ishaan Jaff
94c3de90bb fix entering cache control points 2025-04-14 18:54:04 -07:00
Ishaan Jaff
7649566034 fix inserting cache_control_injection_points 2025-04-14 18:42:10 -07:00
Ishaan Jaff
c0176941d0 fixes ui design 2025-04-14 18:37:33 -07:00
Ishaan Jaff
39e567416c initial commit for cache control 2025-04-14 18:21:26 -07:00
Ishaan Jaff
cb521a8b3e ChatCompletionCachedContent 2025-04-14 17:51:59 -07:00
Ishaan Jaff
a96ab87dd6 fix linting errors 2025-04-14 17:42:13 -07:00
Ishaan Jaff
47f65165d7 fix test 2025-04-14 17:18:29 -07:00
Ishaan Jaff
00b36ac60b test_anthropic_cache_control_hook_specific_index 2025-04-14 17:15:46 -07:00
Marc Klingen
db857c74d4
chore: ordering of logging & observability docs (#9994) 2025-04-14 16:49:04 -07:00
Ishaan Jaff
fd13464108 fix should_run_prompt_management_hooks 2025-04-14 16:42:34 -07:00
Ishaan Jaff
e64254b381 should_run_prompt_management_hooks 2025-04-14 16:34:08 -07:00
Ishaan Jaff
d986b5d6b1 test_anthropic_cache_control_hook.py 2025-04-14 16:31:29 -07:00
Ishaan Jaff
53ce0d434e test_anthropic_cache_control_hook_system_message 2025-04-14 16:21:59 -07:00
Emerson Gomes
a2bc0c0f36
Fix cost for Phi-4-multimodal output token (#9880)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 17s
Helm unit test / unit-test (push) Successful in 23s
2025-04-14 14:31:34 -07:00
Ishaan Jaff
24447eb0cd
fix gpt 4.1 costs (#9991) 2025-04-14 12:50:14 -07:00
Krish Dholakia
bbb7541c22
build(model_prices_and_context_window.json): add gpt-4.1 pricing (#9990)
* build(model_prices_and_context_window.json): add gpt-4.1 pricing

* build(model_prices_and_context_window.json): add gpt-4.1-mini and gpt-4.1-nano model support
2025-04-14 12:14:46 -07:00
Ishaan Jaff
64bb89c70f docs fix
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 16s
Helm unit test / unit-test (push) Successful in 22s
2025-04-12 21:20:54 -07:00
Ishaan Jaff
0e99f83cc2 team info fix default index 2025-04-12 21:06:57 -07:00
Ishaan Jaff
999a9b4ac8 bump: version 1.65.8 → 1.66.0 2025-04-12 20:45:20 -07:00
Ishaan Jaff
72c1f7e09a ui new build 2025-04-12 20:42:43 -07:00
Ishaan Jaff
89dfb42697
[UI QA checklist] (#9957)
* fix typo on UI

* fix for edit user tab

* fix for user spend

* add /team/permissions_list to management routes

* fix auth check for team member permissions

* fix team endpoints test
2025-04-12 20:41:50 -07:00
Krrish Dholakia
2ed63da5f8 docs: cleanup 2025-04-12 19:52:19 -07:00
Krish Dholakia
00e49380df
Litellm UI qa 04 12 2025 p1 (#9955)
* fix(model_info_view.tsx): cleanup text

* fix(key_management_endpoints.py): fix filtering litellm-dashboard keys for internal users

* fix(proxy_track_cost_callback.py): prevent flooding spend logs with admin endpoint errors

* test: add unit testing for logic

* test(test_auth_exception_handler.py): add more unit testing

* fix(router.py): correctly handle retrieving model info on get_model_group_info

fixes issue where model hub was showing None prices

* fix: fix linting errors
2025-04-12 19:30:48 -07:00
Krrish Dholakia
f8d52e2db9 docs: refactor order 2025-04-12 19:23:07 -07:00
Krrish Dholakia
65e18f6abe docs(index.md): update changelog with realtime api cost tracking details 2025-04-12 19:15:40 -07:00
Krrish Dholakia
44368389f4 docs(litellm_managed_files.md): cleanup doc 2025-04-12 18:24:52 -07:00
Ishaan Jaff
2394cd465e
stable release note fixes (#9954)
* docs fix

* docs metrics

* docs fix release notes

* docs 1.66.0-stable
2025-04-12 17:26:38 -07:00
Ishaan Jaff
c86e678809
[Docs] v1.66.0-stable fixes (#9953)
* add categories for spend tracking improvements

* xai reasoning usage

* docs tag management

* docs tag based routing

* [Beta] Routing based

* docs tag based routing

* docs tag routing

* docs enterprise web search
2025-04-12 16:57:25 -07:00
Ishaan Jaff
eb998ee1c0
[v1.66.0-stable] Release notes (#9952)
* release notes

* docs release notes

* docs fix release notes

* docs clean up

* docs clean up

* release notes

* docs sso tag management
2025-04-12 15:32:52 -07:00
Krish Dholakia
25d4cf1c1d
Litellm managed files docs (#9948)
Some checks failed
Helm unit test / unit-test (push) Successful in 27s
Read Version from pyproject.toml / read-version (push) Successful in 43s
Publish Prisma Migrations / publish-migrations (push) Failing after 2m17s
* docs(files_endpoints.md): add doc on litellm managed files

* refactor: separate litellm managed file docs from `/files` docs

clearer

* docs(litellm_managed_files.md): add architecture diagram explaining managed files
2025-04-12 13:02:33 -07:00
Ishaan Jaff
4e81b2cab4
[Team Member permissions] - Fixes (#9945)
* only load member permissions for non-admins

* run member permission checks on update + regenerate endpoints

* run check for /key/generate

* working test_default_member_permissions

* passing test with permissions on update delete endpoints

* test_create_permissions

* _team_key_generation_check

* fix TeamBase

* fix team endpoints

* fix api docs check
2025-04-12 11:17:51 -07:00
Krrish Dholakia
d2a462fc93 ci: handle whl 2025-04-12 11:03:21 -07:00
Krrish Dholakia
4ea9887ff6 ci: see what's in tmp dir 2025-04-12 10:59:53 -07:00
Krrish Dholakia
0853b89864 build: use new litellm-proxy-extras version 2025-04-12 10:55:47 -07:00
Krrish Dholakia
7b465c24a9 fix(config.yml): only run publish_proxy_extras check on main 2025-04-12 10:30:32 -07:00
Krish Dholakia
d004fb542f
fix(litellm_proxy_extras): add baselining db script (#9942)
* fix(litellm_proxy_extras): add baselining db script

Fixes https://github.com/BerriAI/litellm/issues/9885

* fix(prisma_client.py): fix ruff errors

* ci(config.yml): add publish_proxy_extras step

* fix(config.yml): compare contents between versions to check for changes

* fix(config.yml): fix check

* fix: install toml

* fix: update check

* fix: ensure versions in sync

* fix: fix version compare

* fix: correct the cost for 'gemini/gemini-2.5-pro-preview-03-25' (#9896)

* fix: Typo in the cost 'gemini/gemini-2.5-pro-preview-03-25', closes #9854

* chore: update in backup file as well

* Litellm add managed files db (#9930)

* fix(openai.py): ensure openai file object shows up on logs

* fix(managed_files.py): return unified file id as b64 str

allows retrieve file id to work as expected

* fix(managed_files.py): apply decoded file id transformation

* fix: add unit test for file id + decode logic

* fix: initial commit for litellm_proxy support with CRUD Endpoints

* fix(managed_files.py): support retrieve file operation

* fix(managed_files.py): support for DELETE endpoint for files

* fix(managed_files.py): retrieve file content support

supports retrieve file content api from openai

* fix: fix linting error

* test: update tests

* fix: fix linting error

* feat(managed_files.py): support reading / writing files in DB

* feat(managed_files.py): support deleting file from DB on delete

* test: update testing

* fix(spend_tracking_utils.py): ensure each file create request is logged correctly

* fix(managed_files.py): fix storing / returning managed file object from cache

* fix(files/main.py): pass litellm params to azure route

* test: fix test

* build: add new prisma migration

* build: bump requirements

* test: add more testing

* refactor: cleanup post merge w/ main

* fix: fix code qa errors

* [DB / Infra] Add new column team_member_permissions  (#9941)

* add team_member_permissions to team table

* add migration.sql file

* fix poetry lock

* fix prisma migrations

* fix poetry lock

* fix migration

* ui new build

* fix(factory.py): correct indentation for message index increment in ollama,  This fixes bug #9822 (#9943)

* fix(factory.py): correct indentation for message index increment in ollama_pt function

* test: add unit tests for ollama_pt function handling various message types

* ci: update test

* fix: fix check

* ci: see what dir looks like

* ci: more checks

* ci: fix filepath

* ci: cleanup

* ci: fix ci

---------

Co-authored-by: Nilanjan De <nilanjan.de@gmail.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dan Shaw <dan@danieljshaw.com>
2025-04-12 10:29:34 -07:00
Dan Shaw
433075a8d9
fix(factory.py): correct indentation for message index increment in ollama, This fixes bug #9822 (#9943)
* fix(factory.py): correct indentation for message index increment in ollama_pt function

* test: add unit tests for ollama_pt function handling various message types
2025-04-12 09:50:40 -07:00
Ishaan Jaff
69a3aab4c8 ui new build 2025-04-12 09:13:00 -07:00
Ishaan Jaff
fb0c3d9e18
[DB / Infra] Add new column team_member_permissions (#9941)
* add team_member_permissions to team table

* add migration.sql file

* fix poetry lock

* fix prisma migrations

* fix poetry lock

* fix migration
2025-04-12 09:06:04 -07:00
Krish Dholakia
421e0a3004
Litellm add managed files db (#9930)
* fix(openai.py): ensure openai file object shows up on logs

* fix(managed_files.py): return unified file id as b64 str

allows retrieve file id to work as expected

* fix(managed_files.py): apply decoded file id transformation

* fix: add unit test for file id + decode logic

* fix: initial commit for litellm_proxy support with CRUD Endpoints

* fix(managed_files.py): support retrieve file operation

* fix(managed_files.py): support for DELETE endpoint for files

* fix(managed_files.py): retrieve file content support

supports retrieve file content api from openai

* fix: fix linting error

* test: update tests

* fix: fix linting error

* feat(managed_files.py): support reading / writing files in DB

* feat(managed_files.py): support deleting file from DB on delete

* test: update testing

* fix(spend_tracking_utils.py): ensure each file create request is logged correctly

* fix(managed_files.py): fix storing / returning managed file object from cache

* fix(files/main.py): pass litellm params to azure route

* test: fix test

* build: add new prisma migration

* build: bump requirements

* test: add more testing

* refactor: cleanup post merge w/ main

* fix: fix code qa errors
2025-04-12 08:24:46 -07:00
Nilanjan De
93037ea4d3
fix: correct the cost for 'gemini/gemini-2.5-pro-preview-03-25' (#9896)
* fix: Typo in the cost 'gemini/gemini-2.5-pro-preview-03-25', closes #9854

* chore: update in backup file as well
2025-04-12 08:20:04 -07:00