Krish Dholakia
d58fe5a9f9
Add OpenAI gpt-4o-transcribe support ( #9517 )
...
* refactor: introduce new transformation config for gpt-4o-transcribe models
* refactor: expose new transformation configs for audio transcription
* ci: fix config yml
* feat(openai/transcriptions): support provider config transformation on openai audio transcriptions
allows gpt-4o and whisper audio transformation to work as expected
* refactor: migrate fireworks ai + deepgram to new transform request pattern
* feat(openai/): working support for gpt-4o-audio-transcribe
* build(model_prices_and_context_window.json): add gpt-4o-transcribe to model cost map
* build(model_prices_and_context_window.json): specify what endpoints are supported for `/audio/transcriptions`
* fix(get_supported_openai_params.py): fix return
* refactor(deepgram/): migrate unit test to deepgram handler
* refactor: cleanup unused imports
* fix(get_supported_openai_params.py): fix linting error
* test: update test
2025-03-26 23:10:25 -07:00
Krish Dholakia
9c083e7d2c
Support Gemini audio token cost tracking + fix openai audio input token cost tracking ( #9535 )
...
* fix(vertex_and_google_ai_studio_gemini.py): log gemini audio tokens in usage object
enables accurate cost tracking
* refactor(vertex_ai/cost_calculator.py): refactor 128k+ token cost calculation to only run if model info has it
Google has moved away from this for gemini-2.0 models
* refactor(vertex_ai/cost_calculator.py): migrate to usage object for more flexible data passthrough
* fix(llm_cost_calc/utils.py): support audio token cost tracking in generic cost per token
enables vertex ai cost tracking to work with audio tokens
* fix(llm_cost_calc/utils.py): default to total prompt tokens if text tokens field not set
* refactor(llm_cost_calc/utils.py): move openai cost tracking to generic cost per token
more consistent behaviour across providers
* test: add unit test for gemini audio token cost calculation
* ci: bump ci config
* test: fix test
2025-03-26 17:26:25 -07:00
Ishaan Jaff
147548f151
Merge pull request #9542 from BerriAI/litellm_fix_vertex_ai_ft_models
...
[Feature]: Support for Fine-Tuned Vertex AI LLMs
2025-03-26 16:19:41 -07:00
Ishaan Jaff
f77fcbcad3
rename _is_model_gemini_spec_model
2025-03-26 14:28:26 -07:00
Ishaan Jaff
826deea6fb
test_gemini_fine_tuned_model_request_consistency
2025-03-26 14:18:11 -07:00
Ishaan Jaff
e291c10b4f
_get_model_name_from_gemini_spec_model
2025-03-26 12:16:18 -07:00
Krish Dholakia
7873080223
Nova Canvas complete image generation tasks ( #9177 ) ( #9525 )
...
* Nova Canvas complete image generation tasks (#9177 )
* add initial support for Amazon Nova Canvas model
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* adjust name to AmazonNovaCanvas and map function variables to config
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* tighten model name check
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* fix quality mapping
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add premium quality in config
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* support all Amazon Nova Canvas tasks
* remove unused import
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add tests for image generation tasks and fix payload
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add missing util file
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* update model prices backup file
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* remove image tasks other than text->image
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add color guided generation task for Nova Canvas
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* fix merge
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add nova canvas image generation documentation
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add nova canvas unit tests
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
---------
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* ci(config.yml): bump ci config
* test: fix test
---------
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: omrishiv <327609+omrishiv@users.noreply.github.com>
2025-03-26 11:28:20 -07:00
Ishaan Jaff
654536b561
rename _is_model_gemini_spec_model
2025-03-26 11:14:51 -07:00
Ishaan Jaff
0767a3cc88
Merge branch 'main' into litellm_fix_vertex_ai_ft_models
2025-03-26 11:11:54 -07:00
Ishaan Jaff
166ec84c01
undo code changes
2025-03-26 10:57:08 -07:00
Ishaan Jaff
172c9d5689
_is_model_gemini_gemini_spec_model
2025-03-26 10:53:23 -07:00
Ishaan Jaff
59a773920e
_transform_request_body
2025-03-26 00:05:45 -07:00
Krish Dholakia
1ab5d49132
Support litellm.api_base for vertex_ai + gemini/ across completion, embedding, image_generation ( #9516 )
...
* test(tests): add unit testing for litellm_proxy integration
* fix(cost_calculator.py): fix tracking cost in sdk when calling proxy
* fix(main.py): respect litellm.api_base on `vertex_ai/` and `gemini/` routes
* fix(main.py): consistently support custom api base across gemini + vertexai on embedding + completion
* feat(vertex_ai/): test
* fix: fix linting error
* test: set api base as None before starting loadtest
2025-03-25 23:46:20 -07:00
Krish Dholakia
6a40d50bc5
fix vertex ai multimodal embedding translation ( #9471 )
...
* remove data:image/jpeg;base64, prefix from base64 image input
vertex_ai's multimodal embeddings endpoint expects a raw base64 string without `data:image/jpeg;base64,` prefix.
* Add Vertex Multimodal Embedding Test
* fix(test_vertex.py): add e2e tests on multimodal embeddings
* test: unit testing
* test: remove sklearn dep
* test: update test with fixed route
* test: fix test
---------
Co-authored-by: Jonarod <jonrodd@gmail.com>
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
2025-03-24 23:23:28 -07:00
Krish Dholakia
71848566c5
Add vertexai topLogprobs support ( #9518 )
...
* Added support for top_logprobs in vertex gemini models
* Testing for top_logprobs feature in vertexai
* Update litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py
Co-authored-by: Tom Matthews <tomukmatthews@gmail.com>
* refactor(tests/): refactor testing to be in correct repo
---------
Co-authored-by: Aditya Thaker <adityathaker28@gmail.com>
Co-authored-by: Tom Matthews <tomukmatthews@gmail.com>
2025-03-24 22:42:38 -07:00
Krrish Dholakia
017a9b882a
fix(invoke_handler.py): remove hard code
2025-03-24 17:58:26 -07:00
Krrish Dholakia
851c993063
fix(vertex_ai/common_utils.py): fix handling constructed url with default vertex config
2025-03-22 11:32:01 -07:00
Krrish Dholakia
e99a22b9ac
refactor(llm_passthrough_endpoints.py): refactor vertex passthrough to use common llm passthrough handler.py
2025-03-22 10:42:46 -07:00
Krrish Dholakia
b6b6d66882
test: mock sagemaker tests
2025-03-21 16:21:18 -07:00
Krrish Dholakia
f6981606f7
fix: fix linting error
2025-03-21 12:20:21 -07:00
Krrish Dholakia
b3b93dbd14
fix(anthropic/chat/transformation.py): correctly update response_format to tool call transformation
...
Fixes https://github.com/BerriAI/litellm/issues/9411
2025-03-21 10:20:21 -07:00
Ishaan Jaff
6feb83eb51
Merge pull request #9419 from BerriAI/litellm_streaming_o1_pro
...
[Feat] OpenAI o1-pro Responses API streaming support
2025-03-20 21:54:43 -07:00
Krish Dholakia
bc03378fef
Merge pull request #9260 from Grizzly-jobs/fix/voyage-ai-token-usage-tracking
...
fix: VoyageAI `prompt_token` always empty
2025-03-20 14:00:51 -07:00
Ishaan Jaff
3088204ac2
fix code quality checks
2025-03-20 13:57:35 -07:00
Krish Dholakia
f5f92bf6ae
Merge pull request #9366 from JamesGuthrie/jg/vertex-output-dimensionality
...
fix: VertexAI outputDimensionality configuration
2025-03-20 13:55:33 -07:00
Ishaan Jaff
6d4cf6581d
MockResponsesAPIStreamingIterator
2025-03-20 12:30:09 -07:00
Ishaan Jaff
435a89dd79
transform_responses_api_request
2025-03-20 12:28:55 -07:00
Ishaan Jaff
6608770e64
add fake_stream to llm http handler
2025-03-20 09:55:59 -07:00
Ishaan Jaff
1567e52185
add should_fake_stream
2025-03-20 09:54:26 -07:00
Krrish Dholakia
b228456b67
feat(azure/gpt_transformation.py): add azure audio model support
...
Closes https://github.com/BerriAI/litellm/issues/6305
2025-03-19 22:57:49 -07:00
Ishaan Jaff
08cb68c8fb
fix import hashlib
2025-03-19 21:08:19 -07:00
Ishaan Jaff
c15e38a148
Merge branch 'main' into litellm_fix_ssl_verify
2025-03-19 21:03:06 -07:00
James Guthrie
4044bca614
fix: VertexAI outputDimensionality configuration
...
VertexAI's API documentation [1] is an absolute mess. In it, they
describe the parameter to configure output dimensionality as
`output_dimensionality`. In the API example, they switch to using snake
case `outputDimensionality`, which is the correct variant.
[1]: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#generative-ai-get-text-embedding-drest
2025-03-19 11:07:36 +01:00
Krish Dholakia
887648a364
Merge pull request #9363 from BerriAI/litellm_dev_03_18_2025_p3
...
fix(common_utils.py): handle cris only model
2025-03-18 23:36:12 -07:00
Krrish Dholakia
db3a65d52a
fix(common_utils.py): handle cris only model
...
Fixes https://github.com/BerriAI/litellm/issues/9161#issuecomment-2734905153
2025-03-18 23:35:43 -07:00
Ishaan Jaff
55e669d7d8
get_openai_client_cache_key
2025-03-18 18:35:50 -07:00
Ishaan Jaff
4bac8f53a5
fix common utils
2025-03-18 17:59:46 -07:00
Ishaan Jaff
9f31177a20
use common caching logic for openai/azure clients
2025-03-18 17:57:03 -07:00
Ishaan Jaff
ef91a0c72b
use common logic for re-using openai clients
2025-03-18 17:56:32 -07:00
Ishaan Jaff
caca5a1b58
Union[TranscriptionResponse, Coroutine[Any, Any, TranscriptionResponse]]:
2025-03-18 14:23:14 -07:00
Ishaan Jaff
7e2d383885
fix code quality
2025-03-18 12:58:59 -07:00
Ishaan Jaff
0173ca3ad3
test_azure_instruct
2025-03-18 12:56:11 -07:00
Ishaan Jaff
94ac105c23
fix test_ensure_initialize_azure_sdk_client_always_used
2025-03-18 12:46:55 -07:00
Ishaan Jaff
70e80d2149
fix azure chat logic
2025-03-18 12:42:24 -07:00
Ishaan Jaff
069d14ad91
test_azure_embedding_max_retries_0
2025-03-18 12:35:34 -07:00
Ishaan Jaff
b3bc8a3231
:test_completion_azure_ad_toke
2025-03-18 12:25:32 -07:00
Ishaan Jaff
57bb599b47
fix azure embedding test
2025-03-18 12:19:12 -07:00
Ishaan Jaff
9d23214e87
fix amebedding issue on ssl azure
2025-03-18 11:42:11 -07:00
Ishaan Jaff
40736c3263
fix linting error
2025-03-18 11:38:31 -07:00
Ishaan Jaff
e3752cd32a
fix common utils
2025-03-18 11:04:02 -07:00
