Commit graph

472 commits

Author SHA1 Message Date
Krish Dholakia
6dda1ba6dd
LiteLLM Minor Fixes & Improvements (04/02/2025) (#9725)
* Add date picker to usage tab + Add reasoning_content token tracking across all providers on streaming (#9722)

* feat(new_usage.tsx): add date picker for new usage tab

allow user to look back on their usage data

* feat(anthropic/chat/transformation.py): report reasoning tokens in completion token details

allows usage tracking on how many reasoning tokens are actually being used

* feat(streaming_chunk_builder.py): return reasoning_tokens in anthropic/openai streaming response

allows tracking reasoning_token usage across providers

* Fix update team metadata + fix bulk adding models on Ui  (#9721)

* fix(handle_add_model_submit.tsx): fix bulk adding models

* fix(team_info.tsx): fix team metadata update

Fixes https://github.com/BerriAI/litellm/issues/9689

* (v0) Unified file id - allow calling multiple providers with same file id (#9718)

* feat(files_endpoints.py): initial commit adding 'target_model_names' support

allow developer to specify all the models they want to call with the file

* feat(files_endpoints.py): return unified files endpoint

* test(test_files_endpoints.py): add validation test - if invalid purpose submitted

* feat: more updates

* feat: initial working commit of unified file id translation

* fix: additional fixes

* fix(router.py): remove model replace logic in jsonl on acreate_file

enables file upload to work for chat completion requests as well

* fix(files_endpoints.py): remove whitespace around model name

* fix(azure/handler.py): return acreate_file with correct response type

* fix: fix linting errors

* test: fix mock test to run on github actions

* fix: fix ruff errors

* fix: fix file too large error

* fix(utils.py): remove redundant var

* test: modify test to work on github actions

* test: update tests

* test: more debug logs to understand ci/cd issue

* test: fix test for respx

* test: skip mock respx test

fails on ci/cd - not clear why

* fix: fix ruff check

* fix: fix test

* fix(model_connection_test.tsx): fix linting error

* test: update unit tests
2025-04-03 11:48:52 -07:00
Krish Dholakia
8ee32291e0
Squashed commit of the following: (#9709)
commit b12a9892b7
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed Apr 2 08:09:56 2025 -0700

    fix(utils.py): don't modify openai_token_counter

commit 294de31803
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 21:22:40 2025 -0700

    fix: fix linting error

commit cb6e9fbe40
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 19:52:45 2025 -0700

    refactor: complete migration

commit bfc159172d
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 19:09:59 2025 -0700

    refactor: refactor more constants

commit 43ffb6a558
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:45:24 2025 -0700

    fix: test

commit 04dbe4310c
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:28:58 2025 -0700

    refactor: refactor: move more constants into constants.py

commit 3c26284aff
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:14:46 2025 -0700

    refactor: migrate hardcoded constants out of __init__.py

commit c11e0de69d
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:11:21 2025 -0700

    build: migrate all constants into constants.py

commit 7882bdc787
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Mar 24 18:07:37 2025 -0700

    build: initial test banning hardcoded numbers in repo
2025-04-02 21:24:54 -07:00
Krish Dholakia
23051d89dd
fix(streaming_handler.py): fix completion start time tracking (#9688)
* fix(streaming_handler.py): fix completion start time tracking

Fixes https://github.com/BerriAI/litellm/issues/9210

* feat(anthropic/chat/transformation.py): map openai 'reasoning_effort' to anthropic 'thinking' param

Fixes https://github.com/BerriAI/litellm/issues/9022

* feat: map 'reasoning_effort' to 'thinking' param across bedrock + vertex

Closes https://github.com/BerriAI/litellm/issues/9022#issuecomment-2705260808
2025-04-01 22:00:56 -07:00
Krish Dholakia
722f3ff0e6
fix(cost_calculator.py): allows checking received + sent model name when checking for cost calculation (#9669)
Fixes issue introduced by dfb838eaff (r154667517)
2025-03-31 21:29:48 -07:00
Krish Dholakia
5ad2fbcba6
Openrouter streaming fixes + Anthropic 'file' message support (#9667)
* fix(openrouter/transformation.py): Handle error in openrouter stream

Fixes https://github.com/Aider-AI/aider/issues/3550

* test(test_openrouter_chat_transformation.py): add unit tests

* feat(anthropic/chat/transformation.py): add openai 'file' message content type support

Closes https://github.com/BerriAI/litellm/issues/9463

* fix(factory.py): add bedrock converse support for openai 'file' message content type

Closes https://github.com/BerriAI/litellm/issues/9463
2025-03-31 21:22:59 -07:00
Ishaan Jaff
5df985f964
Merge pull request #9642 from BerriAI/litellm_mcp_improvements_expose_sse_urls
[Feat] - MCP improvements, add support for using SSE MCP servers
2025-03-29 19:37:57 -07:00
Ishaan Jaff
08a52f4389 log MCP tool call metadata in SLP 2025-03-29 15:50:13 -07:00
Ishaan Jaff
b7b9f9d9da working MCP tool call logging 2025-03-29 15:20:52 -07:00
Krish Dholakia
9b7ebb6a7d
build(pyproject.toml): add new dev dependencies - for type checking (#9631)
* build(pyproject.toml): add new dev dependencies - for type checking

* build: reformat files to fit black

* ci: reformat to fit black

* ci(test-litellm.yml): make tests run clear

* build(pyproject.toml): add ruff

* fix: fix ruff checks

* build(mypy/): fix mypy linting errors

* fix(hashicorp_secret_manager.py): fix passing cert for tls auth

* build(mypy/): resolve all mypy errors

* test: update test

* fix: fix black formatting

* build(pre-commit-config.yaml): use poetry run black

* fix(proxy_server.py): fix linting error

* fix: fix ruff safe representation error
2025-03-29 11:02:13 -07:00
Krish Dholakia
5ac61a7572
Add bedrock latency optimized inference support (#9623)
* fix(converse_transformation.py): add performanceConfig param support on bedrock

Closes https://github.com/BerriAI/litellm/issues/7606

* fix(converse_transformation.py): refactor to use more flexible single getter for params which are separate config blocks

* test(test_main.py): add e2e mock test for bedrock performance config

* build(model_prices_and_context_window.json): add versioned multimodal embedding

* refactor(multimodal_embeddings/): migrate to config pattern

* feat(vertex_ai/multimodalembeddings): calculate usage for multimodal embedding calls

Enables cost calculation for multimodal embeddings

* feat(vertex_ai/multimodalembeddings): get usage object for embedding calls

ensures accurate cost tracking for vertexai multimodal embedding calls

* fix(embedding_handler.py): remove unused imports

* fix: fix linting errors

* fix: handle response api usage calculation

* test(test_vertex_ai_multimodal_embedding_transformation.py): update tests

* test: mark flaky test

* feat(vertex_ai/multimodal_embeddings/transformation.py): support text+image+video input

* docs(vertex.md): document sending text + image to vertex multimodal embeddings

* test: remove incorrect file

* fix(multimodal_embeddings/transformation.py): fix linting error

* style: remove unused import
2025-03-29 00:23:09 -07:00
Nicholas Grabar
1f2bbda11d Add recursion depth to convert_anyof_null_to_nullable, constants.py. Fix recursive_detector.py raise error state 2025-03-28 13:11:19 -07:00
Krish Dholakia
c0845fec1f
Add OpenAI gpt-4o-transcribe support (#9517)
* refactor: introduce new transformation config for gpt-4o-transcribe models

* refactor: expose new transformation configs for audio transcription

* ci: fix config yml

* feat(openai/transcriptions): support provider config transformation on openai audio transcriptions

allows gpt-4o and whisper audio transformation to work as expected

* refactor: migrate fireworks ai + deepgram to new transform request pattern

* feat(openai/): working support for gpt-4o-audio-transcribe

* build(model_prices_and_context_window.json): add gpt-4o-transcribe to model cost map

* build(model_prices_and_context_window.json): specify what endpoints are supported for `/audio/transcriptions`

* fix(get_supported_openai_params.py): fix return

* refactor(deepgram/): migrate unit test to deepgram handler

* refactor: cleanup unused imports

* fix(get_supported_openai_params.py): fix linting error

* test: update test
2025-03-26 23:10:25 -07:00
Krish Dholakia
4351c77253
Support Gemini audio token cost tracking + fix openai audio input token cost tracking (#9535)
* fix(vertex_and_google_ai_studio_gemini.py): log gemini audio tokens in usage object

enables accurate cost tracking

* refactor(vertex_ai/cost_calculator.py): refactor 128k+ token cost calculation to only run if model info has it

Google has moved away from this for gemini-2.0 models

* refactor(vertex_ai/cost_calculator.py): migrate to usage object for more flexible data passthrough

* fix(llm_cost_calc/utils.py): support audio token cost tracking in generic cost per token

enables vertex ai cost tracking to work with audio tokens

* fix(llm_cost_calc/utils.py): default to total prompt tokens if text tokens field not set

* refactor(llm_cost_calc/utils.py): move openai cost tracking to generic cost per token

more consistent behaviour across providers

* test: add unit test for gemini audio token cost calculation

* ci: bump ci config

* test: fix test
2025-03-26 17:26:25 -07:00
Krrish Dholakia
6a0cf3db50 fix(litellm_logging.py): always log the api base
Fixes issue where api base missing from spend logs due to refactor
2025-03-24 13:45:39 -07:00
Ishaan Jaff
edfcff47b9 fix bug when unable to look up model info 2025-03-22 18:24:47 -07:00
Ishaan Jaff
1b9f1837cd _get_file_search_tool_call 2025-03-22 18:04:33 -07:00
Ishaan Jaff
ded612563d _get_file_search_tool_call 2025-03-22 18:02:22 -07:00
Ishaan Jaff
8766b43a4b fix StandardBuiltInToolsParams 2025-03-22 17:53:06 -07:00
Ishaan Jaff
0f36b8afc2 chat_completion_response_includes_annotations 2025-03-22 17:36:22 -07:00
Ishaan Jaff
7f807cd6ba fix code quality check 2025-03-22 17:31:47 -07:00
Ishaan Jaff
f74640cb99 test_openai_responses_api_web_search_cost_tracking 2025-03-22 17:24:29 -07:00
Ishaan Jaff
81033dc8c9 test_openai_responses_api_web_search_cost_tracking 2025-03-22 17:21:41 -07:00
Ishaan Jaff
904795443d get_cost_for_built_in_tools 2025-03-22 17:06:34 -07:00
Ishaan Jaff
475dfaa156 fixes for web search cost tracking 2025-03-22 16:56:32 -07:00
Ishaan Jaff
bfe3132bb6 StandardBuiltInToolCostTracking 2025-03-22 16:06:47 -07:00
Ishaan Jaff
10da287225 initialize_standard_built_in_tools_params 2025-03-22 16:03:53 -07:00
Ishaan Jaff
792a2d6115 test_is_chunk_non_empty_with_annotations 2025-03-22 11:41:53 -07:00
Ishaan Jaff
dc5bf10009 feat - add openai web search 2025-03-22 10:43:35 -07:00
Ishaan Jaff
9bc5aaee64 fix ModelParamHelper 2025-03-21 21:35:49 -07:00
Ishaan Jaff
c63217e4d4 _get_litellm_supported_transcription_kwargs 2025-03-21 19:20:13 -07:00
Hammad Saeed
732ab55cc8 fix(model_param_helper.py): update _get_litellm_supported_transcription_kwargs() to use proper annotations from TranscriptionCreateParamsNonStreaming & `TranscriptionCreateParamsStreaming 2025-03-21 16:55:35 -07:00
Hammad Saeed
3b9dd9be36 fix(model_param_helper.py): change TranscriptionCreateParams.__annotations__ to TranscriptionCreateParams.__dict__ to clean logging error // handle typeddict 2025-03-21 16:30:46 -07:00
Ishaan Jaff
dae5c24b55 get_chat_completion_prompt 2025-03-19 20:50:15 -07:00
Ishaan Jaff
a49bd460e0 get_custom_logger_for_prompt_management 2025-03-19 17:38:24 -07:00
Ishaan Jaff
805542005a CustomPromptManagement 2025-03-19 16:38:59 -07:00
Ishaan Jaff
730f61f476 get_custom_loggers_for_type 2025-03-19 16:36:41 -07:00
Ishaan Jaff
f5ef0c3cb7 fix code quality checks 2025-03-18 22:34:43 -07:00
Ishaan Jaff
0f2e095b6b _arize_otel_logger 2025-03-18 22:19:51 -07:00
Ishaan Jaff
57e5c94360 Merge branch 'main' into litellm_arize_dynamic_logging 2025-03-18 22:13:35 -07:00
Krrish Dholakia
a34cc2031d fix(response_metadata.py): log the litellm_model_name
make it easier to track the model sent to the provider
2025-03-18 17:46:33 -07:00
Krrish Dholakia
453729b6c9 fix(litellm_logging.py): fix update 2025-03-18 17:18:58 -07:00
Ishaan Jaff
f4b7b49e9a fix arize logging 2025-03-18 15:51:58 -07:00
Krrish Dholakia
2c094c2356 fix(get_llm_provider.py): Fixes https://github.com/BerriAI/litellm/issues/9291 2025-03-18 10:48:28 -07:00
Krish Dholakia
cd5024f3b1
Merge pull request #9333 from BerriAI/litellm_dev_03_17_2025_p2
fix(ollama/completions/transformation.py): pass prompt, untemplated o…
2025-03-17 21:48:30 -07:00
Krrish Dholakia
22faf7d232 fix(ollama/completions/transformation.py): pass prompt, untemplated on /completions request
Fixes https://github.com/BerriAI/litellm/issues/6900
2025-03-17 18:35:44 -07:00
Krrish Dholakia
c4b2e0ae3d fix(streaming_handler.py): support logging complete streaming response on cache hit 2025-03-17 18:10:39 -07:00
Krrish Dholakia
dd9e79adbd fix(streaming_handler.py): emit deep copy of completed chunk 2025-03-17 17:26:21 -07:00
Krrish Dholakia
a5b497667c fix(logging_utils.py): revert change 2025-03-16 21:04:41 -07:00
Krrish Dholakia
a99251a4ab fix(streaming_handler.py): raise stop iteration post-finish reason 2025-03-16 20:40:41 -07:00
Krrish Dholakia
bde9ae8a95 fix(litellm_logging.py): remove unused import 2025-03-16 20:24:27 -07:00