Commit graph

1623 commits

Author SHA1 Message Date
Krish Dholakia
fb83567a03
Litellm new UI build (#9601)
* build: new ui build

* build: new ui build

* fix(proxy_server.py): only show user models their key can access on `/models`

* fix(model_management_endpoints.py): ensure team admin can add models

* test: update unit testing to reflect changes

* fix(model_dashboard.tsx): fix sizing on models page

* build: fix ui
2025-03-27 17:15:25 -07:00
Ishaan Jaff
a0fd508de4 DBSpendUpdateWriter 2025-03-27 16:43:18 -07:00
Ishaan Jaff
21e3b764f5 use DBSpendUpdateWriter class for managing DB spend updates 2025-03-27 16:31:23 -07:00
Krish Dholakia
11838e1c3b
Litellm fix db testing (#9593)
* ci: fix test

* test: safely change db url

* fix: print db url

* test: remove delenv
2025-03-27 14:50:41 -07:00
Krish Dholakia
63c9f59373
Allow team admins to add/update/delete models on UI + show api base and model id on request logs (#9572)
* feat(view_logs.tsx): show model id + api base in request logs

easier debugging

* fix(index.tsx): fix length of api base

easier viewing

* refactor(leftnav.tsx): show models tab to team admin

* feat(model_dashboard.tsx): add explainer for what the 'models' page is for team admin

helps them understand how they can use it

* feat(model_management_endpoints.py): restrict model add by team to just team admin

allow team admin to add models via non-team keys (e.g. ui token)

* test(test_add_update_models.py): update unit testing for new behaviour

* fix(model_dashboard.tsx): show user the models

* feat(proxy_server.py): add new query param 'user_models_only' to `/v2/model/info`

Allows user to retrieve just the models they've added

Used in UI to show internal users just the models they've added

* feat(model_dashboard.tsx): allow team admins to view their own models

* fix: allow ui user to fetch model cost map

* feat(add_model_tab.tsx): require team admins to specify team when onboarding models

* fix(_types.py): add `/v1/model/info` to info route

`/model/info` was already there

* fix(model_info_view.tsx): allow user to edit a model they created

* fix(model_management_endpoints.py): allow team admin to update team model

* feat(model_managament_endpoints.py): allow team admin to delete team models

* fix(model_management_endpoints.py): don't require team id to be set when adding a model

* fix(proxy_server.py): fix linting error

* fix: fix ui linting error

* fix(model_management_endpoints.py): ensure consistent auth checks on all model calls

* test: remove old test - function no longer exists in same form

* test: add updated mock testing
2025-03-27 12:06:31 -07:00
Krish Dholakia
c0845fec1f
Add OpenAI gpt-4o-transcribe support (#9517)
* refactor: introduce new transformation config for gpt-4o-transcribe models

* refactor: expose new transformation configs for audio transcription

* ci: fix config yml

* feat(openai/transcriptions): support provider config transformation on openai audio transcriptions

allows gpt-4o and whisper audio transformation to work as expected

* refactor: migrate fireworks ai + deepgram to new transform request pattern

* feat(openai/): working support for gpt-4o-audio-transcribe

* build(model_prices_and_context_window.json): add gpt-4o-transcribe to model cost map

* build(model_prices_and_context_window.json): specify what endpoints are supported for `/audio/transcriptions`

* fix(get_supported_openai_params.py): fix return

* refactor(deepgram/): migrate unit test to deepgram handler

* refactor: cleanup unused imports

* fix(get_supported_openai_params.py): fix linting error

* test: update test
2025-03-26 23:10:25 -07:00
Krrish Dholakia
109add7946 build(model_prices_and_context_window.json): add gemini multimodal embedding cost 2025-03-26 23:04:24 -07:00
Ishaan Jaff
b6506f7bda test_db_health_readiness_check_with_prisma_error 2025-03-26 20:56:39 -07:00
Ishaan Jaff
87f0201f84 test_handle_db_exception_with_connection_error 2025-03-26 20:44:25 -07:00
Ishaan Jaff
15c04da735 refactor tests 2025-03-26 20:00:10 -07:00
Ishaan Jaff
0155b0eba2
Merge pull request #9533 from BerriAI/litellm_stability_fixes
[Reliability Fixes] - Gracefully handle exceptions when DB is having an outage
2025-03-26 18:57:38 -07:00
Ishaan Jaff
5242c5fbab test - auth exception handler 2025-03-26 18:28:28 -07:00
Ishaan Jaff
15b1a8afb0 test_is_database_connection_error_prisma_errors 2025-03-26 18:27:39 -07:00
Ishaan Jaff
9eb9a369bb working anthropic API tests 2025-03-26 17:34:41 -07:00
Krish Dholakia
4351c77253
Support Gemini audio token cost tracking + fix openai audio input token cost tracking (#9535)
* fix(vertex_and_google_ai_studio_gemini.py): log gemini audio tokens in usage object

enables accurate cost tracking

* refactor(vertex_ai/cost_calculator.py): refactor 128k+ token cost calculation to only run if model info has it

Google has moved away from this for gemini-2.0 models

* refactor(vertex_ai/cost_calculator.py): migrate to usage object for more flexible data passthrough

* fix(llm_cost_calc/utils.py): support audio token cost tracking in generic cost per token

enables vertex ai cost tracking to work with audio tokens

* fix(llm_cost_calc/utils.py): default to total prompt tokens if text tokens field not set

* refactor(llm_cost_calc/utils.py): move openai cost tracking to generic cost per token

more consistent behaviour across providers

* test: add unit test for gemini audio token cost calculation

* ci: bump ci config

* test: fix test
2025-03-26 17:26:25 -07:00
Krrish Dholakia
04490c99d7 test: fix test 2025-03-26 17:12:09 -07:00
Krrish Dholakia
d4adc9764b test(test_db_schema_migration.py): ci/cd test to enforce schema migrations are documented in .sql files 2025-03-26 16:59:50 -07:00
Krish Dholakia
132d3f7baa
feat(prisma-migrations): add baseline db migration file (#9565)
adds initial baseline db migration file

enables future schema changes to be documented via .sql files
2025-03-26 16:22:56 -07:00
Ishaan Jaff
b7f4abd13a
Merge pull request #9542 from BerriAI/litellm_fix_vertex_ai_ft_models
[Feature]: Support for Fine-Tuned Vertex AI LLMs
2025-03-26 16:19:41 -07:00
Ishaan Jaff
4948673e35 fix test changes 2025-03-26 14:51:33 -07:00
Ishaan Jaff
044c56c439 Merge branch 'main' into litellm_stability_fixes 2025-03-26 14:41:43 -07:00
Ishaan Jaff
0aae9aa24a rename _is_model_gemini_spec_model 2025-03-26 14:28:26 -07:00
Ishaan Jaff
c38b41f65b test_get_supports_system_message 2025-03-26 14:26:08 -07:00
Ishaan Jaff
72f08bc6ea unit tests for VertexGeminiConfig 2025-03-26 14:21:35 -07:00
Ishaan Jaff
8eaf4c55c0 test_gemini_fine_tuned_model_request_consistency 2025-03-26 14:18:11 -07:00
Ishaan Jaff
da9d849348 test_gemini_fine_tuned_model_request_consistency 2025-03-26 14:10:32 -07:00
Krish Dholakia
801ecb6517
Nova Canvas complete image generation tasks (#9177) (#9525)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 17s
Helm unit test / unit-test (push) Successful in 22s
* Nova Canvas complete image generation tasks (#9177)

* add initial support for Amazon Nova Canvas model

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* adjust name to AmazonNovaCanvas and map function variables to config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* tighten model name check

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* fix quality mapping

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add premium quality in config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* support all Amazon Nova Canvas tasks

* remove unused import

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add tests for image generation tasks and fix payload

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add missing util file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* update model prices backup file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* remove image tasks other than text->image

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add color guided generation task for Nova Canvas

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* fix merge

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add nova canvas image generation documentation

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add nova canvas unit tests

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

---------

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* ci(config.yml): bump ci config

* test: fix test

---------

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: omrishiv <327609+omrishiv@users.noreply.github.com>
2025-03-26 11:28:20 -07:00
Ishaan Jaff
1822c45c0e test fix test_pick_cheapest_chat_model_from_llm_provider 2025-03-26 11:13:21 -07:00
Ishaan Jaff
baa9b34950 Merge branch 'main' into litellm_fix_vertex_ai_ft_models 2025-03-26 11:11:54 -07:00
Ishaan Jaff
bbe69a47a9 _is_model_gemini_gemini_spec_model 2025-03-26 10:53:23 -07:00
Ishaan Jaff
8c845847cd fix vertex embedding perf test 2025-03-26 10:25:50 -07:00
Ishaan Jaff
e7a8b5a809 run ci/cd again 2025-03-26 08:12:51 -07:00
Ishaan Jaff
fafbaf6302 Merge branch 'main' into litellm_stability_fixes 2025-03-26 07:16:31 -07:00
Ishaan Jaff
27c085cc56 fix util vertex 2025-03-26 00:08:16 -07:00
Ishaan Jaff
efce84815a test_gemini_fine_tuned_model_request_consistency 2025-03-25 23:54:06 -07:00
Krish Dholakia
6fd18651d1
Support litellm.api_base for vertex_ai + gemini/ across completion, embedding, image_generation (#9516)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 19s
Helm unit test / unit-test (push) Successful in 20s
* test(tests): add unit testing for litellm_proxy integration

* fix(cost_calculator.py): fix tracking cost in sdk when calling proxy

* fix(main.py): respect litellm.api_base on `vertex_ai/` and `gemini/` routes

* fix(main.py): consistently support custom api base across gemini + vertexai on embedding + completion

* feat(vertex_ai/): test

* fix: fix linting error

* test: set api base as None before starting loadtest
2025-03-25 23:46:20 -07:00
Ishaan Jaff
6e5d2b1ac7 handle failed db connections 2025-03-25 23:14:44 -07:00
Nicholas Grabar
f68cc26f15 8864 Add support for anyOf union type while handling null fields 2025-03-25 22:37:28 -07:00
Ishaan Jaff
9aec7c3878 test_create_delete_assistants 2025-03-25 22:08:06 -07:00
Ishaan Jaff
0a401ee468 test_litellm_proxy_server_config_no_general_settings 2025-03-25 19:27:15 -07:00
Ishaan Jaff
6572ba7a0e fix startup 2025-03-25 19:25:47 -07:00
Ishaan Jaff
b4e745323a add test config 2025-03-25 19:21:51 -07:00
Ishaan Jaff
9d10befa09 test_litellm_proxy_server_config_no_general_settings 2025-03-25 19:16:34 -07:00
Ishaan Jaff
53d9e33e78 fix setup toxi proxy 2025-03-25 18:59:26 -07:00
Ishaan Jaff
7b09d88680 fix setup 2025-03-25 17:52:12 -07:00
Ishaan Jaff
c6d5793bf6 add toxi proxy tests to ci/cd 2025-03-25 17:50:27 -07:00
Krish Dholakia
92883560f0
fix vertex ai multimodal embedding translation (#9471)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 20s
Helm unit test / unit-test (push) Successful in 24s
* remove data:image/jpeg;base64, prefix from base64 image input

vertex_ai's multimodal embeddings endpoint expects a raw base64 string without `data:image/jpeg;base64,` prefix.

* Add Vertex Multimodal Embedding Test

* fix(test_vertex.py): add e2e tests on multimodal embeddings

* test: unit testing

* test: remove sklearn dep

* test: update test with fixed route

* test: fix test

---------

Co-authored-by: Jonarod <jonrodd@gmail.com>
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
2025-03-24 23:23:28 -07:00
Krrish Dholakia
75994d0bf0 test: improve flaky test 2025-03-24 23:15:04 -07:00
Krish Dholakia
a619580bf8
Add vertexai topLogprobs support (#9518)
* Added support for top_logprobs in vertex gemini models

* Testing for top_logprobs feature in vertexai

* Update litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py

Co-authored-by: Tom Matthews <tomukmatthews@gmail.com>

* refactor(tests/): refactor testing to be in correct repo

---------

Co-authored-by: Aditya Thaker <adityathaker28@gmail.com>
Co-authored-by: Tom Matthews <tomukmatthews@gmail.com>
2025-03-24 22:42:38 -07:00
Ishaan Jaff
08a4ba1b7e Merge branch 'main' into litellm_exp_mcp_server 2025-03-24 19:03:56 -07:00