Commit graph

21541 commits

Author SHA1 Message Date
Ishaan Jaff
779b728d34 fix base responses api tests 2025-04-19 13:18:42 -07:00
Ishaan Jaff
a508da4c8d test_basic_openai_responses_api_streaming 2025-04-19 13:11:33 -07:00
Ishaan Jaff
b61ae8dc86 fix for responses API 2025-04-19 13:09:19 -07:00
Ishaan Jaff
4e74951a7d transform_responses_api_request_to_chat_completion_request 2025-04-19 13:06:12 -07:00
Ishaan Jaff
0fec9a042c fixes for MockResponsesAPIStreamingIterator 2025-04-19 12:21:10 -07:00
Ishaan Jaff
d37ecef3f0 fix render responses completed events 2025-04-19 12:02:15 -07:00
Ishaan Jaff
e6ae53b5a9 fixes to streaming iterator 2025-04-19 11:26:40 -07:00
Ishaan Jaff
ffd6d35ba2 responses API render chunks on litellm chat ui 2025-04-19 10:43:27 -07:00
Ishaan Jaff
e4644b32b7 fix endpoint selector 2025-04-19 10:29:41 -07:00
Ishaan Jaff
df0874ef78 fix add responses API on UI 2025-04-19 10:28:01 -07:00
Ishaan Jaff
d9d5c2f577 add makeOpenAIResponsesRequest 2025-04-19 09:57:41 -07:00
Ishaan Jaff
dc67f4bc18 add makeOpenAIResponsesRequest 2025-04-19 09:57:31 -07:00
Ishaan Jaff
88b6cb6c28 add /responses API on UI 2025-04-19 09:56:29 -07:00
Krish Dholakia
2508ca71cb
Handle fireworks ai tool calling response (#10130)
* feat(fireworks_ai/chat): handle tool calling with fireworks ai correctly

Fixes https://github.com/BerriAI/litellm/issues/7209

* fix(utils.py): handle none type in message

* fix: fix model name in test

* fix(utils.py): fix validate check for openai messages

* fix: fix model returned

* fix(main.py): fix text completion routing

* test: update testing

* test: skip test - cohere having RBAC issues
2025-04-19 09:37:45 -07:00
Krrish Dholakia
b4f2b3dad1 test: update test to be more robust to usage updates 2025-04-19 09:26:26 -07:00
Ishaan Jaff
8ae2653280 fix calculated cache key for tests 2025-04-19 09:25:11 -07:00
Ishaan Jaff
97d7a5e78e fix deployment name 2025-04-19 09:23:22 -07:00
Krish Dholakia
36308a31be
Gemini-2.5-flash - support reasoning cost calc + return reasoning content (#10141)
* build(model_prices_and_context_window.json): add vertex ai gemini-2.5-flash pricing

* build(model_prices_and_context_window.json): add gemini reasoning token pricing

* fix(vertex_and_google_ai_studio_gemini.py): support counting thinking tokens for gemini

allows accurate cost calc

* fix(utils.py): add reasoning token cost calc to generic cost calc

ensures gemini-2.5-flash cost calculation is accurate

* build(model_prices_and_context_window.json): mark gemini-2.5-flash as 'supports_reasoning'

* feat(gemini/): support 'thinking' + 'reasoning_effort' params + new unit tests

allow controlling thinking effort for gemini-2.5-flash models

* test: update unit testing

* feat(vertex_and_google_ai_studio_gemini.py): return reasoning content if given in gemini response

* test: update model name

* fix: fix ruff check

* test(test_spend_management_endpoints.py): update tests to be less sensitive to new keys / updates to usage object

* fix(vertex_and_google_ai_studio_gemini.py): fix translation
2025-04-19 09:20:52 -07:00
Ishaan Jaff
db4ebe10c8 bump litellm-proxy-extras 2025-04-19 09:14:33 -07:00
Krrish Dholakia
d726e0f34c test: update testing imports 2025-04-19 09:13:16 -07:00
Krrish Dholakia
ba1b552e8b fix(common_daily_activity.py): fix python 3_8 error 2025-04-19 08:39:19 -07:00
Ishaan Jaff
49759d5678 fix get_azure_client 2025-04-19 08:33:26 -07:00
Ishaan Jaff
0a35c208d7 test assistants fixes 2025-04-19 08:09:45 -07:00
Krrish Dholakia
dee5182fc8 fix: fix linting error 2025-04-19 08:04:56 -07:00
Ishaan Jaff
a62805f98f fixes for assistans API tests 2025-04-19 07:59:53 -07:00
Ishaan Jaff
5bf76f0bb1 test fixes for azure assistants 2025-04-19 07:36:40 -07:00
Krish Dholakia
ef6ac42658
Litellm dev 04 18 2025 p2 (#10157)
* fix(proxy/_types.py): allow internal user to call api playground

* fix(new_usage.tsx): cleanup tag based usage - only show for proxy admin

not clear what tags internal user should be allowed to see

* fix(team_endpoints.py): allow internal user view spend for teams they belong to

* fix(team_endpoints.py): return team alias on `/team/daily/activity` API

allows displaying team alias on ui

* fix: fix linting error

* fix(entity_usage.tsx): allow viewing top keys by team

* fix(entity_usage.tsx): show alias, if available in breakdown

allows entity alias to be easily displayed

* Show usage by key (on all up, team, and tag usage dashboards)  (#10152)

* fix(entity_usage.tsx): allow user to select team in team usage tab

* fix(new_usage.tsx): load all tags for filtering

* fix(tag_management_endpoints.py): return dynamic tags from db on `/tag/list`

* fix(litellm_pre_call_utils.py): support x-litellm-tags even if tag based routing not enabled

* fix(new_usage.tsx): show breakdown of usage by api key on dashboard

helpful when looking at spend by team

* fix(networking.tsx): exclude litellm-dashboard team id's from calls

adds noisy ui tokens to key activity

* fix(new_usage.tsx): allow user to see activity by key on main tab

* feat(internal_user_endpoints.py): refactor to use common_daily_activity function

reuses same logic across teams/keys/tags

Allows returning team_alias in api_keys consistently

* fix(leftnav.tsx): swap old usage with new usage tab

* fix(entity_usage.tsx): show breakdown of teams in daily spend chart

* style(new_usage.tsx): show global usage tab if user is admin / has admin view

* fix(new_usage.tsx): add disclaimer for new usage dashboard

* fix(new_usage.tsx): fix linting error

* Allow filtering usage dashboard by team + tag (#10150)

* fix(entity_usage.tsx): allow user to select team in team usage tab

* fix(new_usage.tsx): load all tags for filtering

* fix(tag_management_endpoints.py): return dynamic tags from db on `/tag/list`

* fix(litellm_pre_call_utils.py): support x-litellm-tags even if tag based routing not enabled

* fix: fix linting error
2025-04-19 07:32:23 -07:00
Ishaan Jaff
b9756bf006 test_completion_azure 2025-04-19 07:24:11 -07:00
Krish Dholakia
a1879cfa35
fix(litellm-proxy-extras/utils.py): prisma migrate improvements: handle existing columns in db table (#10138) 2025-04-18 20:36:56 -07:00
Krrish Dholakia
652e1b7f0f test: update test 2025-04-18 20:36:15 -07:00
Ishaan Jaff
6de3481252 doc fix 2025-04-18 19:57:03 -07:00
Ishaan Jaff
76f00a5121 add info on litellm release 2025-04-18 19:56:29 -07:00
Ishaan Jaff
3d5022bd79
[Feat] Support for all litellm providers on Responses API (works with Codex) - Anthropic, Bedrock API, VertexAI, Ollama (#10132)
* transform request

* basic handler for LiteLLMCompletionTransformationHandler

* complete transform litellm to responses api

* fixes to test

* fix stream=True

* fix streaming iterator

* fixes for transformation

* fixes for anthropic codex support

* fix pass response_api_optional_params

* test anthropic responses api tools

* update responses types

* working codex with litellm

* add session handler

* fixes streaming iterator

* fix handler

* add litellm codex example

* fix code quality

* test fix

* docs litellm codex

* litellm codexdoc

* docs openai codex with litellm

* docs litellm openai codex

* litellm codex

* linting fixes for transforming responses API

* fix import error

* fix responses api test

* add sync iterator support for responses api
2025-04-18 19:53:59 -07:00
Krrish Dholakia
3e87ec4f16 test: replace removed fireworks ai models
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 19s
Helm unit test / unit-test (push) Successful in 24s
2025-04-18 14:23:16 -07:00
Krish Dholakia
1ea046cc61
test: update tests to new deployment model (#10142)
* test: update tests to new deployment model

* test: update model name

* test: skip cohere rbac issue test

* test: update test - replace gpt-4o model
2025-04-18 14:22:12 -07:00
Krrish Dholakia
415abfc222 test: update test 2025-04-18 13:13:58 -07:00
David Emmanuel
de3c2d14bf
Add Gemini Flash 2.5 Preview Model Price and Context Window (#10125)
* Update model_prices_and_context_window_backup.json

* Update model_prices_and_context_window.json
2025-04-18 09:44:46 -07:00
Krrish Dholakia
f7dd688035 test: handle cohere rbac issue (verified happens on calling azure directly)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 15s
Helm unit test / unit-test (push) Successful in 23s
2025-04-18 08:42:12 -07:00
Krrish Dholakia
809eb859cf fix(azure/o_series_transformation.py): fix azure o4 model routing
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 41s
Helm unit test / unit-test (push) Successful in 49s
Fixes https://github.com/BerriAI/litellm/pull/10065#issuecomment-2814015058
2025-04-17 22:58:01 -07:00
Krrish Dholakia
614d80cb1b build(model_prices_and_context_window.json): add azure gpt-4.1 pricing
ensures cost tracking for gpt-4.1 works
2025-04-17 20:09:17 -07:00
Marc Abramowitz
3c71a81100
Remove unnecessary package*.json files (#10075) 2025-04-17 20:03:56 -07:00
Ishaan Jaff
19664960eb docs azure responses API 2025-04-17 18:47:44 -07:00
Marc Abramowitz
409dde22f6
UI: Make columns resizable/hideable in Models table (#10119)
* Make columns resizable in Models table

* Make edit and delete buttons sticky on right side

* Add Columns dropdown to control which columns are shown

* Remove unnecessary dependencies

* Fix title of visibility checkboxes for Input Cost and Output Cost

* Make the Columns dropdown close if the user clicks anywhere outside of it
2025-04-17 18:12:20 -07:00
Ishaan Jaff
d3e04eac7f
[Feat] Unified Responses API - Add Azure Responses API support (#10116)
* initial commit for azure responses api support

* update get complete url

* fixes for responses API

* working azure responses API

* working responses API

* test suite for responses API

* azure responses API test suite

* fix test with complete url

* fix test refactor

* test fix metadata checks

* fix code quality check
2025-04-17 16:47:59 -07:00
Krrish Dholakia
8be8022914 docs(vertex_ai.md): document new vertex passthrough route
Some checks failed
Read Version from pyproject.toml / read-version (push) Successful in 44s
Helm unit test / unit-test (push) Successful in 51s
Publish Prisma Migrations / publish-migrations (push) Failing after 2m16s
2025-04-16 22:25:16 -07:00
Krrish Dholakia
ff81f48af3 bump: version 1.66.2 → 1.66.3 2025-04-16 22:20:10 -07:00
Krrish Dholakia
78c6d73dea build: new ui build 2025-04-16 22:11:53 -07:00
Ishaan Jaff
257e78ffb5 test fix vertex_ai/mistral-large@2407 2025-04-16 21:52:52 -07:00
Krish Dholakia
8ddaf3dfbc
fix(o_series_transformation.py): correctly map o4 to openai o_series model (#10079)
Fixes https://github.com/BerriAI/litellm/issues/10066
2025-04-16 21:51:31 -07:00
Krish Dholakia
c73a6a8d1e
Add new /vertex_ai/discovery route - enables calling AgentBuilder API routes (#10084)
* feat(llm_passthrough_endpoints.py): expose new `/vertex_ai/discovery/` endpoint

Allows calling vertex ai discovery endpoints via passthrough

 For agentbuilder api calls

* refactor(llm_passthrough_endpoints.py): use common _base_vertex_proxy_route

Prevents duplicate code

* feat(llm_passthrough_endpoints.py): add vertex endpoint specific passthrough handlers
2025-04-16 21:45:51 -07:00