Commit graph

41 commits

Author SHA1 Message Date
Krish Dholakia
d58fe5a9f9 Add OpenAI gpt-4o-transcribe support (#9517)
* refactor: introduce new transformation config for gpt-4o-transcribe models

* refactor: expose new transformation configs for audio transcription

* ci: fix config yml

* feat(openai/transcriptions): support provider config transformation on openai audio transcriptions

allows gpt-4o and whisper audio transformation to work as expected

* refactor: migrate fireworks ai + deepgram to new transform request pattern

* feat(openai/): working support for gpt-4o-audio-transcribe

* build(model_prices_and_context_window.json): add gpt-4o-transcribe to model cost map

* build(model_prices_and_context_window.json): specify what endpoints are supported for `/audio/transcriptions`

* fix(get_supported_openai_params.py): fix return

* refactor(deepgram/): migrate unit test to deepgram handler

* refactor: cleanup unused imports

* fix(get_supported_openai_params.py): fix linting error

* test: update test
2025-03-26 23:10:25 -07:00
Ishaan Jaff
f77fcbcad3 rename _is_model_gemini_spec_model 2025-03-26 14:28:26 -07:00
Ishaan Jaff
64c2a46647 test_get_supports_system_message 2025-03-26 14:26:08 -07:00
Ishaan Jaff
4e7b099c38 unit tests for VertexGeminiConfig 2025-03-26 14:21:35 -07:00
Krish Dholakia
6a40d50bc5 fix vertex ai multimodal embedding translation (#9471)
* remove data:image/jpeg;base64, prefix from base64 image input

vertex_ai's multimodal embeddings endpoint expects a raw base64 string without `data:image/jpeg;base64,` prefix.

* Add Vertex Multimodal Embedding Test

* fix(test_vertex.py): add e2e tests on multimodal embeddings

* test: unit testing

* test: remove sklearn dep

* test: update test with fixed route

* test: fix test

---------

Co-authored-by: Jonarod <jonrodd@gmail.com>
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
2025-03-24 23:23:28 -07:00
Krish Dholakia
71848566c5 Add vertexai topLogprobs support (#9518)
* Added support for top_logprobs in vertex gemini models

* Testing for top_logprobs feature in vertexai

* Update litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py

Co-authored-by: Tom Matthews <tomukmatthews@gmail.com>

* refactor(tests/): refactor testing to be in correct repo

---------

Co-authored-by: Aditya Thaker <adityathaker28@gmail.com>
Co-authored-by: Tom Matthews <tomukmatthews@gmail.com>
2025-03-24 22:42:38 -07:00
Krrish Dholakia
860ddec3c5 test: migrate testing 2025-03-22 12:48:53 -07:00
Krrish Dholakia
6c51bdebe7 test: add unit testing 2025-03-21 10:35:36 -07:00
Ishaan Jaff
baf859f580 test_prepare_fake_stream_request 2025-03-20 14:50:00 -07:00
Ishaan Jaff
c15e38a148 Merge branch 'main' into litellm_fix_ssl_verify 2025-03-19 21:03:06 -07:00
Krrish Dholakia
db3a65d52a fix(common_utils.py): handle cris only model
Fixes https://github.com/BerriAI/litellm/issues/9161#issuecomment-2734905153
2025-03-18 23:35:43 -07:00
Ishaan Jaff
55e669d7d8 get_openai_client_cache_key 2025-03-18 18:35:50 -07:00
Ishaan Jaff
bb8400a350 test_openai_client_reuse 2025-03-18 18:13:36 -07:00
Ishaan Jaff
4307316f4d test openai common utils 2025-03-18 18:08:24 -07:00
Ishaan Jaff
caca5a1b58 Union[TranscriptionResponse, Coroutine[Any, Any, TranscriptionResponse]]: 2025-03-18 14:23:14 -07:00
Ishaan Jaff
3261c66b39 initialize_azure_sdk_client 2025-03-18 12:27:17 -07:00
Krrish Dholakia
e1860c3d9d test: refactor testing to handle routing correctly 2025-03-18 12:24:12 -07:00
Krrish Dholakia
425585f25c fix(http_handler.py): support reading ssl security level from env var
Allows user to specify lower security settings
2025-03-17 15:48:31 -07:00
Krish Dholakia
834d826490 Merge pull request #9274 from BerriAI/litellm_contributor_rebase_branch
Litellm contributor rebase branch
2025-03-14 21:57:49 -07:00
Krrish Dholakia
649c41d9d1 feat(converse_transformation.py): fix type for bedrock cache usage block 2025-03-13 19:33:22 -07:00
Krrish Dholakia
09eb748e13 test(tests/litellm): add unit test for transform usage function 2025-03-13 15:56:55 -07:00
Krish Dholakia
cb7cbdff8f Merge pull request #9123 from omrishiv/8911-fix-model-encoding
Fixes bedrock modelId encoding for Inference Profiles
2025-03-13 10:42:32 -07:00
sven
fb43ef1bb7 add test code 2025-03-13 14:00:12 +09:00
Ishaan Jaff
9895c94bda test_ensure_initialize_azure_sdk_client_always_used 2025-03-12 18:51:19 -07:00
Krish Dholakia
103b3cb574 Merge branch 'main' into litellm_dev_03_10_2025_p3 2025-03-12 14:56:01 -07:00
Ishaan Jaff
f75022ab59 test_validate_environment 2025-03-12 12:57:40 -07:00
Krrish Dholakia
af1de16117 fix(azure/completions): migrate completions endpoint to support base azure llm class
enables consistent auth logic across all azure calls
2025-03-11 18:47:30 -07:00
Krrish Dholakia
9a98942e87 fix: fix merge conflicts 2025-03-11 18:41:41 -07:00
Krrish Dholakia
f06811ca40 test: fix test 2025-03-11 18:13:27 -07:00
Krrish Dholakia
2f262ed9b4 refactor(azure.py): refactor to have client init work across all endpoints 2025-03-11 17:27:24 -07:00
Krrish Dholakia
1516240bab refactor(batches/main.py): working refactored azure client init on batches 2025-03-11 14:36:38 -07:00
Krrish Dholakia
9855e46208 refactor(azure/audio_transcriptions.py): support client init with common logic 2025-03-11 14:24:12 -07:00
Krrish Dholakia
4f4507ccc0 refactor(azure.py): working azure client init on audio speech endpoint 2025-03-11 14:19:45 -07:00
Krrish Dholakia
858d9005a2 refactor(azure.py): refactor acompletion to use base azure sdk client 2025-03-11 13:59:13 -07:00
Krrish Dholakia
d68218a0fe test(test_azure_common_utils.py): add unit testing for common azure client params function 2025-03-11 12:24:08 -07:00
omrishiv
45bb27c146 update test
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
2025-03-10 20:34:52 -07:00
omrishiv
e91d0fa205 add test
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
2025-03-10 20:28:55 -07:00
Teja Vishwanadha
048ff931be support bytes.IO for audio transcription (#9071) 2025-03-08 08:47:15 -08:00
Ishaan Jaff
c83486f39c (UI + Backend) Fix Adding Azure, Azure AI Studio models on LiteLLM (#8856)
* fix Azure_AI_Studio

* fix flow for adding custom model names

* fix _should_use_api_key_header

* handle custom model name change

* test_azure_ai_request_format

* Azure AI Foundry (Studio)

* fix _should_use_api_key_header
2025-02-26 18:45:02 -08:00
Krish Dholakia
30a4f2abc2 Add cost tracking for rerank via bedrock (#8691)
* feat(bedrock/rerank): infer model region if model given as arn

* test: add unit testing to ensure bedrock region name inferred from arn on rerank

* feat(bedrock/rerank/transformation.py): include search units for bedrock rerank result

Resolves https://github.com/BerriAI/litellm/issues/7258#issuecomment-2671557137

* test(test_bedrock_completion.py): add testing for bedrock cohere rerank

* feat(cost_calculator.py): refactor rerank cost tracking to support bedrock cost tracking

* build(model_prices_and_context_window.json): add amazon.rerank model to model cost map

* fix(cost_calculator.py): bedrock/common_utils.py

get base model from model w/ arn -> handles rerank model

* build(model_prices_and_context_window.json): add bedrock cohere rerank pricing

* feat(bedrock/rerank): migrate bedrock config to basererank config

* Revert "feat(bedrock/rerank): migrate bedrock config to basererank config"

This reverts commit 84fae1f167.

* test: add testing to ensure large doc / queries are correctly counted

* Revert "test: add testing to ensure large doc / queries are correctly counted"

This reverts commit 4337f1657e.

* fix(migrate-jina-ai-to-rerank-config): enables cost tracking

* refactor(jina_ai/): finish migrating jina ai to base rerank config

enables cost tracking

* fix(jina_ai/rerank): e2e jina ai rerank cost tracking

* fix: cleanup dead code

* fix: fix python3.8 compatibility error

* test: fix test

* test: add e2e testing for azure ai rerank

* fix: fix linting error

* test: mark cohere as flaky
2025-02-20 21:00:18 -08:00
Ishaan Jaff
f940392971 (Observability) - Add more detailed dd tracing on Proxy Auth, Bedrock Auth (#8693)
* add dd tracer

* fix dd tracing

* add @tracer.wrap() on def user_api_key_auth

* add async_function_with_retries

* remove dead code

* add tracer.wrap on base aws llm

* add tracer.wrap on base aws llm

* fix print verbose

* fix dd tracing

* trace base aws llm

* fix test base aws llm

* fix converse transform

* test base aws llm

* BASE_AWS_LLM_PATH

* BASE_AWS_LLM_PATH

* test dd tracing
2025-02-20 18:00:41 -08:00