Commit graph

560 commits

Author SHA1 Message Date
Ishaan Jaff
01b16c36a0 Merge branch 'main' into litellm_anthropic_messages_improvements 2025-03-31 14:22:56 -07:00
Ishaan Jaff
f5c0afcf96 Merge pull request #9642 from BerriAI/litellm_mcp_improvements_expose_sse_urls
[Feat] - MCP improvements, add support for using SSE MCP servers
2025-03-29 19:37:57 -07:00
Krish Dholakia
70f993d3d7 Add gemini audio input support + handle special tokens in sagemaker response (#9640)
* fix(internal_user_endpoints.py): cleanup unused variables on beta endpoint

no team/org split on daily user endpoint

* build(model_prices_and_context_window.json): gemini-2.0-flash supports audio input

* feat(gemini/transformation.py): support passing audio input to gemini

* test: fix test

* fix(gemini/transformation.py): support audio input as a url

enables passing google cloud bucket urls

* fix(gemini/transformation.py): support explicitly passing format of file

* fix(gemini/transformation.py): expand support for inferred file types from url

* fix(sagemaker/completion/transformation.py): fix special token error when counting sagemaker tokens

* test: fix import
2025-03-29 19:23:09 -07:00
Ishaan Jaff
198fb23d0c fix listing mcp tools 2025-03-29 18:40:58 -07:00
Ishaan Jaff
55ad7cbda0 fix import errors without mcp 2025-03-29 17:44:32 -07:00
Ishaan Jaff
2dfd302a82 log MCP tool call metadata in SLP 2025-03-29 15:50:13 -07:00
Ishaan Jaff
7d00abc37f endpoints to list and call tools 2025-03-29 14:31:35 -07:00
Ishaan Jaff
6c7b42f575 REST API endpoint for MCP 2025-03-29 13:35:46 -07:00
Ishaan Jaff
061f98c570 mcp server manager 2025-03-29 12:51:16 -07:00
Krish Dholakia
d7b294dd0a build(pyproject.toml): add new dev dependencies - for type checking (#9631)
* build(pyproject.toml): add new dev dependencies - for type checking

* build: reformat files to fit black

* ci: reformat to fit black

* ci(test-litellm.yml): make tests run clear

* build(pyproject.toml): add ruff

* fix: fix ruff checks

* build(mypy/): fix mypy linting errors

* fix(hashicorp_secret_manager.py): fix passing cert for tls auth

* build(mypy/): resolve all mypy errors

* test: update test

* fix: fix black formatting

* build(pre-commit-config.yaml): use poetry run black

* fix(proxy_server.py): fix linting error

* fix: fix ruff safe representation error
2025-03-29 11:02:13 -07:00
Krish Dholakia
308a2fb195 Add bedrock latency optimized inference support (#9623)
* fix(converse_transformation.py): add performanceConfig param support on bedrock

Closes https://github.com/BerriAI/litellm/issues/7606

* fix(converse_transformation.py): refactor to use more flexible single getter for params which are separate config blocks

* test(test_main.py): add e2e mock test for bedrock performance config

* build(model_prices_and_context_window.json): add versioned multimodal embedding

* refactor(multimodal_embeddings/): migrate to config pattern

* feat(vertex_ai/multimodalembeddings): calculate usage for multimodal embedding calls

Enables cost calculation for multimodal embeddings

* feat(vertex_ai/multimodalembeddings): get usage object for embedding calls

ensures accurate cost tracking for vertexai multimodal embedding calls

* fix(embedding_handler.py): remove unused imports

* fix: fix linting errors

* fix: handle response api usage calculation

* test(test_vertex_ai_multimodal_embedding_transformation.py): update tests

* test: mark flaky test

* feat(vertex_ai/multimodal_embeddings/transformation.py): support text+image+video input

* docs(vertex.md): document sending text + image to vertex multimodal embeddings

* test: remove incorrect file

* fix(multimodal_embeddings/transformation.py): fix linting error

* style: remove unused import
2025-03-29 00:23:09 -07:00
Krish Dholakia
d58fe5a9f9 Add OpenAI gpt-4o-transcribe support (#9517)
* refactor: introduce new transformation config for gpt-4o-transcribe models

* refactor: expose new transformation configs for audio transcription

* ci: fix config yml

* feat(openai/transcriptions): support provider config transformation on openai audio transcriptions

allows gpt-4o and whisper audio transformation to work as expected

* refactor: migrate fireworks ai + deepgram to new transform request pattern

* feat(openai/): working support for gpt-4o-audio-transcribe

* build(model_prices_and_context_window.json): add gpt-4o-transcribe to model cost map

* build(model_prices_and_context_window.json): specify what endpoints are supported for `/audio/transcriptions`

* fix(get_supported_openai_params.py): fix return

* refactor(deepgram/): migrate unit test to deepgram handler

* refactor: cleanup unused imports

* fix(get_supported_openai_params.py): fix linting error

* test: update test
2025-03-26 23:10:25 -07:00
Krish Dholakia
9c083e7d2c Support Gemini audio token cost tracking + fix openai audio input token cost tracking (#9535)
* fix(vertex_and_google_ai_studio_gemini.py): log gemini audio tokens in usage object

enables accurate cost tracking

* refactor(vertex_ai/cost_calculator.py): refactor 128k+ token cost calculation to only run if model info has it

Google has moved away from this for gemini-2.0 models

* refactor(vertex_ai/cost_calculator.py): migrate to usage object for more flexible data passthrough

* fix(llm_cost_calc/utils.py): support audio token cost tracking in generic cost per token

enables vertex ai cost tracking to work with audio tokens

* fix(llm_cost_calc/utils.py): default to total prompt tokens if text tokens field not set

* refactor(llm_cost_calc/utils.py): move openai cost tracking to generic cost per token

more consistent behaviour across providers

* test: add unit test for gemini audio token cost calculation

* ci: bump ci config

* test: fix test
2025-03-26 17:26:25 -07:00
Ishaan Jaff
f450fbdfa0 fix response typing 2025-03-26 16:56:56 -07:00
Ishaan Jaff
620d49e1dc add type hints for AnthropicMessagesResponse 2025-03-26 16:53:33 -07:00
Krish Dholakia
7873080223 Nova Canvas complete image generation tasks (#9177) (#9525)
* Nova Canvas complete image generation tasks (#9177)

* add initial support for Amazon Nova Canvas model

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* adjust name to AmazonNovaCanvas and map function variables to config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* tighten model name check

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* fix quality mapping

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add premium quality in config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* support all Amazon Nova Canvas tasks

* remove unused import

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add tests for image generation tasks and fix payload

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add missing util file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* update model prices backup file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* remove image tasks other than text->image

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add color guided generation task for Nova Canvas

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* fix merge

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add nova canvas image generation documentation

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add nova canvas unit tests

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

---------

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* ci(config.yml): bump ci config

* test: fix test

---------

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: omrishiv <327609+omrishiv@users.noreply.github.com>
2025-03-26 11:28:20 -07:00
Ishaan Jaff
c6424d6246 Merge branch 'main' into litellm_exp_mcp_server 2025-03-24 19:03:56 -07:00
Ishaan Jaff
f133bb07d1 fix pydantic import error 2025-03-24 07:11:48 -07:00
Ishaan Jaff
d932206bfb Merge branch 'main' into litellm_exp_mcp_server 2025-03-22 18:51:25 -07:00
Ishaan Jaff
f03f3c3d9a test_langfuse_logging_completion 2025-03-22 18:09:04 -07:00
Ishaan Jaff
4fb3fb3dff FileSearchTool 2025-03-22 17:56:14 -07:00
Ishaan Jaff
93a2b00c93 fix StandardBuiltInToolsParams 2025-03-22 17:53:06 -07:00
Ishaan Jaff
cf01f49893 fixes for web search cost tracking 2025-03-22 16:56:32 -07:00
Ishaan Jaff
8aa13bf632 WebSearchOptions 2025-03-22 15:39:04 -07:00
Ishaan Jaff
01df7a49e5 add WebSearchOptions as supported chat completion param 2025-03-22 15:37:34 -07:00
Ishaan Jaff
02c5aebc87 search_context_cost_per_query 2025-03-22 14:52:58 -07:00
Ishaan Jaff
2d05ee2a8b fix supports_web_search 2025-03-22 14:02:51 -07:00
Ishaan Jaff
13bfe7d518 Add annotations to the delta 2025-03-22 11:38:30 -07:00
Ishaan Jaff
69da0ed3b5 feat - add openai web search 2025-03-22 10:43:35 -07:00
Ishaan Jaff
616c4db12d add litellm mcp endpoints 2025-03-20 21:12:56 -07:00
Ishaan Jaff
2581797cf7 load load_tools_from_config 2025-03-20 17:36:17 -07:00
Ishaan Jaff
63d454bb8d add MCPToolRegistry 2025-03-20 17:22:12 -07:00
Krrish Dholakia
dcbe5df98b fix(types/utils.py): support openai 'file' message type
Closes https://github.com/BerriAI/litellm/issues/9365
2025-03-19 23:13:51 -07:00
Ishaan Jaff
43eedf26f7 Merge branch 'main' into litellm_arize_dynamic_logging 2025-03-18 22:13:35 -07:00
Krrish Dholakia
3ba0df484e fix(lowest_tpm_rpm_v2.py): fix updating limits 2025-03-18 17:10:17 -07:00
Ishaan Jaff
c153fd3656 fix arize config 2025-03-18 16:54:31 -07:00
Ishaan Jaff
6468937b44 add arize_api_key to StandardCallbackDynamicParams 2025-03-18 14:46:55 -07:00
Krrish Dholakia
911b053095 fix(http_handler.py): fix typing error 2025-03-17 16:42:32 -07:00
Krrish Dholakia
b75cd3b887 feat(endpoints.py): support adding credentials by model id
Allows user to reuse existing model credentials
2025-03-14 12:32:32 -07:00
Krrish Dholakia
913dc5b73b feat(endpoints.py): enable retrieving existing credentials by model name
Enables reusing existing credentials
2025-03-14 12:02:50 -07:00
Ishaan Jaff
ad0afbb5dc Merge pull request #9222 from BerriAI/litellm_snowflake_pr_mar_13
[Feat] Add Snowflake Cortex to LiteLLM
2025-03-13 21:35:39 -07:00
Krrish Dholakia
649c41d9d1 feat(converse_transformation.py): fix type for bedrock cache usage block 2025-03-13 19:33:22 -07:00
Sunny Wan
e01d12b878 Merge branch 'BerriAI:main' into main 2025-03-13 19:37:22 -04:00
Krrish Dholakia
b51495f2da feat(converse_transformation.py): translate converse usage block with cache creation values to openai format 2025-03-13 15:49:25 -07:00
Ishaan Jaff
c3a8f140c1 working spend tracking + logging for response api 2025-03-12 17:29:25 -07:00
Krish Dholakia
103b3cb574 Merge branch 'main' into litellm_dev_03_10_2025_p3 2025-03-12 14:56:01 -07:00
Ishaan Jaff
61f4080903 fix mypy linting errors 2025-03-12 12:13:19 -07:00
Ishaan Jaff
d6ccaf1e63 Merge branch 'main' into litellm_responses_api_support 2025-03-12 12:04:12 -07:00
Ishaan Jaff
ff94836121 typing_extensions Annotated 2025-03-12 11:58:56 -07:00
Ishaan Jaff
127d084765 BaseLiteLLMOpenAIResponseObject 2025-03-12 11:39:51 -07:00