Ishaan Jaff
39d391d8e7
Optional[Dict]
2025-03-12 12:29:13 -07:00
Ishaan Jaff
342741ede1
Merge branch 'main' into litellm_responses_api_support
2025-03-12 12:04:12 -07:00
Ishaan Jaff
181072e15d
fix code quality checks
2025-03-12 11:47:10 -07:00
Ishaan Jaff
047879c004
add aresponses
2025-03-12 09:22:44 -07:00
Ishaan Jaff
4ff6e41c15
ResponsesAPIStreamEvents
2025-03-11 23:42:35 -07:00
Ishaan Jaff
278b6fb5f6
add debug logging
2025-03-11 23:13:10 -07:00
Krish Dholakia
a7e0e7283e
Merge pull request #9166 from BerriAI/litellm_dev_03_11_2025_p2
...
Litellm dev 03 11 2025 p2
2025-03-11 22:51:20 -07:00
Krish Dholakia
8c0bf06c87
Merge branch 'main' into litellm_dev_contributor_prs_03_10_2025_p1
2025-03-11 22:50:02 -07:00
Krrish Dholakia
b8d590da0c
fix(azure/audio_transcriptions.py): support azure cost tracking
...
extract content time and log correctly as duration
2025-03-11 22:25:13 -07:00
Ishaan Jaff
ddb819da45
fix order of imports
2025-03-11 22:04:24 -07:00
Krrish Dholakia
92d85555fe
fix(invoke_handler.py): fix converse chunk parsing to only return empty dict on tool use
...
Fixes https://github.com/BerriAI/litellm/issues/9127
2025-03-11 22:04:17 -07:00
Krrish Dholakia
1051478e95
fix(azure_ai/): fix transformation to handle when models don't support tool_choice
Read Version from pyproject.toml / read-version (push) Successful in 15s
Helm unit test / unit-test (push) Successful in 23s
2025-03-11 21:42:24 -07:00
Ishaan Jaff
8fa313ab07
add async streaming support
2025-03-11 20:00:42 -07:00
Ishaan Jaff
cb270887a7
ResponsesAPIStreamingIterator
2025-03-11 19:49:09 -07:00
Ishaan Jaff
f32968409e
working basic openai response api request
2025-03-11 17:37:19 -07:00
Ishaan Jaff
b3999b4c75
transform_response_api_response
2025-03-11 17:03:31 -07:00
Ishaan Jaff
d3575f0a31
working async_response_api_handler
2025-03-11 16:58:43 -07:00
Ishaan Jaff
dff8308e92
add transform_response_api_response
2025-03-11 16:53:18 -07:00
Ishaan Jaff
0f8de3d0a5
add transform_request for OpenAI responses API
2025-03-11 16:33:26 -07:00
Ishaan Jaff
c7e8534df4
BaseResponsesAPIConfig
2025-03-11 16:14:14 -07:00
Ishaan Jaff
980354b78b
add validate_environment, get_complete_url
2025-03-11 16:12:36 -07:00
Ishaan Jaff
2c6774e3ee
get_optional_params_responses_api
2025-03-11 16:00:49 -07:00
Ishaan Jaff
4d55212c62
add BaseResponsesAPIConfig
2025-03-11 15:57:53 -07:00
Ishaan Jaff
401a52e694
working transform
2025-03-11 15:24:42 -07:00
Ishaan Jaff
368f1de2e1
add OpenAIResponsesAPIConfig
2025-03-11 15:10:34 -07:00
Ishaan Jaff
6462466e02
add OpenAIResponsesAPIConfig
2025-03-11 15:09:42 -07:00
Krish Dholakia
9610c844c7
Merge pull request #9109 from BerriAI/litellm_dev_03_10_2025_p1_v2
...
Read Version from pyproject.toml / read-version (push) Successful in 12s
Return `code`, `param` and `type` on openai bad request error
2025-03-10 22:38:16 -07:00
Krish Dholakia
86a5926e26
Merge pull request #9113 from BerriAI/litellm_dev_03_10_2025_p2
...
fix(base_invoke_transformation.py): support extra_headers on bedrock …
2025-03-10 22:19:10 -07:00
Ishaan Jaff
24b8dcff1f
get_complete_url
2025-03-10 20:45:58 -07:00
Krrish Dholakia
68bd05ac24
fix(base_invoke_transformation.py): support extra_headers on bedrock invoke route
...
Fixes https://github.com/BerriAI/litellm/issues/9106
2025-03-10 16:13:11 -07:00
Ishaan Jaff
94667e1cf0
Merge pull request #8386 from minwhoo/triton-completions-streaming-fix
...
Fix triton streaming completions bug
2025-03-10 16:07:19 -07:00
Krrish Dholakia
bfbe26b91d
feat(azure.py): add azure bad request error support
2025-03-10 15:59:06 -07:00
Ishaan Jaff
ea058ab4ea
Merge pull request #8746 from niinpatel/patch-1
...
fix missing comma
2025-03-10 15:57:32 -07:00
Krrish Dholakia
5f87dc229a
feat(openai.py): bubble all error information back to client
2025-03-10 15:27:43 -07:00
Krrish Dholakia
c1ec82fbd5
refactor: instrument body param to bubble up on exception
2025-03-10 15:21:04 -07:00
omrishiv
0674491386
add support for Amazon Nova Canvas model ( #7838 )
...
* add initial support for Amazon Nova Canvas model
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* adjust name to AmazonNovaCanvas and map function variables to config
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* tighten model name check
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* fix quality mapping
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add premium quality in config
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* support all Amazon Nova Canvas tasks
* remove unused import
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add tests for image generation tasks and fix payload
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add missing util file
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* update model prices backup file
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* remove image tasks other than text->image
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
---------
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-03-10 08:02:00 -07:00
Krish Dholakia
f899b828cf
Support openrouter reasoning_content on streaming ( #9094 )
...
* feat(convert_dict_to_response.py): support openrouter format of reasoning content
* fix(transformation.py): fix openrouter streaming with reasoning content
Fixes https://github.com/BerriAI/litellm/issues/8193#issuecomment-270892962
* fix: fix type error
2025-03-09 20:03:59 -07:00
Krish Dholakia
e00d4fb18c
Litellm dev 03 08 2025 p3 ( #9089 )
...
* feat(ollama_chat.py): pass down http client to ollama_chat
enables easier testing
* fix(factory.py): fix passing images to ollama's `/api/generate` endpoint
Fixes https://github.com/BerriAI/litellm/issues/6683
* fix(factory.py): fix ollama pt to handle templating correctly
2025-03-09 18:20:56 -07:00
Teja Vishwanadha
8c049dfffc
support bytes.IO for audio transcription ( #9071 )
2025-03-08 08:47:15 -08:00
Ishaan Jaff
e2d612efd9
Bug fix - String data: stripped from entire content in streamed Gemini responses ( #9070 )
...
* _strip_sse_data_from_chunk
* use _strip_sse_data_from_chunk
* use _strip_sse_data_from_chunk
* use _strip_sse_data_from_chunk
* _strip_sse_data_from_chunk
* test_strip_sse_data_from_chunk
* _strip_sse_data_from_chunk
* testing
* _strip_sse_data_from_chunk
2025-03-07 21:06:39 -08:00
Ishaan Jaff
f47987e673
(Refactor) /v1/messages to follow simpler logic for Anthropic API spec ( #9013 )
...
* anthropic_messages_handler v0
* fix /messages
* working messages with router methods
* test_anthropic_messages_handler_litellm_router_non_streaming
* test_anthropic_messages_litellm_router_non_streaming_with_logging
* AnthropicMessagesConfig
* _handle_anthropic_messages_response_logging
* working with /v1/messages endpoint
* working /v1/messages endpoint
* refactor to use router factory function
* use aanthropic_messages
* use BaseConfig for Anthropic /v1/messages
* track api key, team on /v1/messages endpoint
* fix get_logging_payload
* BaseAnthropicMessagesTest
* align test config
* test_anthropic_messages_with_thinking
* test_anthropic_streaming_with_thinking
* fix - display anthropic url for debugging
* test_bad_request_error_handling
* test_anthropic_messages_router_streaming_with_bad_request
* fix ProxyException
* test_bad_request_error_handling_streaming
* use provider_specific_header
* test_anthropic_messages_with_extra_headers
* test_anthropic_messages_to_wildcard_model
* fix gcs pub sub test
* standard_logging_payload
* fix unit testing for anthopic /v1/messages support
* fix pass through anthropic messages api
* delete dead code
* fix anthropic pass through response
* revert change to spend tracking utils
* fix get_litellm_metadata_from_kwargs
* fix spend logs payload json
* proxy_pass_through_endpoint_tests
* TestAnthropicPassthroughBasic
* fix pass through tests
* test_async_vertex_proxy_route_api_key_auth
* _handle_anthropic_messages_response_logging
* vertex_credentials
* test_set_default_vertex_config
* test_anthropic_messages_litellm_router_non_streaming_with_logging
* test_ageneric_api_call_with_fallbacks_basic
* test__aadapter_completion
2025-03-06 00:43:08 -08:00
Krish Dholakia
744e10b0f0
Litellm dev 03 05 2025 p3 ( #9023 )
...
* fix(invoke_handler.py): fix converse streaming - return signature + ensure consistency with anthropic api response
* build(model_prices_and_context_window.json): fix anthropic api claude-3-7 max output tokens
with beta header this is 128k
Resolves https://github.com/BerriAI/litellm/issues/8964
* feat(handler.py): handle new anthropic 'thinking_delta' block on streaming
Fixes https://github.com/BerriAI/litellm/issues/8825
2025-03-05 22:31:39 -08:00
Krish Dholakia
f6535ae6ad
Support format param for specifying image type ( #9019 )
...
* fix(transformation.py): support a 'format' parameter for image's
allow user to specify mime type
* fix: pass mimetype via 'format' param
* feat(gemini/chat/transformation.py): support 'format' param for gemini
* fix(factory.py): support 'format' param on sync bedrock converse calls
* feat(bedrock/converse_transformation.py): support 'format' param for bedrock async calls
* refactor(factory.py): move to supporting 'format' param in base helper
ensures consistency in param support
* feat(gpt_transformation.py): filter out 'format' param
don't send invalid param to openai
* fix(gpt_transformation.py): fix translation
* fix: fix translation error
2025-03-05 19:52:53 -08:00
Krish Dholakia
ec4f665e29
Return signature on anthropic streaming + migrate to signature field instead of signature_delta [MINOR bump] ( #9021 )
...
* Fix missing signature_delta in thinking blocks when streaming from Claude 3.7 (#8797 )
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* test: update test to enforce signature found
* feat(refactor-signature-param-to-be-'signature'-instead-of-'signature_delta'): keeps it in sync with anthropic
* fix: fix linting error
---------
Co-authored-by: Martin Krasser <krasserm@googlemail.com>
2025-03-05 19:33:54 -08:00
Krish Dholakia
c69ec66dc5
fix(base_aws_llm.py): remove region name before sending in args ( #8998 )
...
Read Version from pyproject.toml / read-version (push) Successful in 12s
* fix(base_aws_llm.py): remove region name before sending in args
* fix(base_aws_llm.py): fix optional param pop position
* fix: fix linting error
2025-03-04 23:05:28 -08:00
Krish Dholakia
5e386c28b2
Litellm dev 03 04 2025 p3 ( #8997 )
...
* fix(core_helpers.py): handle litellm_metadata instead of 'metadata'
* feat(batches/): ensure batches logs are written to db
makes batches response dict compatible
* fix(cost_calculator.py): handle batch response being a dictionary
* fix(batches/main.py): modify retrieve endpoints to use @client decorator
enables logging to work on retrieve call
* fix(batches/main.py): fix retrieve batch response type to be 'dict' compatible
* fix(spend_tracking_utils.py): send unique uuid for retrieve batch call type
create batch and retrieve batch share the same id
* fix(spend_tracking_utils.py): prevent duplicate retrieve batch calls from being double counted
* refactor(batches/): refactor cost tracking for batches - do it on retrieve, and within the established litellm_logging pipeline
ensures cost is always logged to db
* fix: fix linting errors
* fix: fix linting error
2025-03-04 21:58:03 -08:00
Krish Dholakia
f1a44d1fdc
fix(common_utils.py): handle $id in response schema when calling vert… ( #8991 )
...
* fix(common_utils.py): handle $id in response schema when calling vertex ai
Fixes issue where `$id` present in response_schema was not accepted by vertex ai
* test(test_vertex.py): add unit test to ensure $id stripped out of vertex schema
2025-03-04 21:19:50 -08:00
Krrish Dholakia
a63eb58f1b
fix(anthropic/chat/transformation.py): fix headers to be a set
...
avoid duplicates
2025-03-02 08:36:43 -08:00
Krrish Dholakia
4418e6dd14
build: merge branch
2025-03-02 08:31:57 -08:00
Krish Dholakia
c84b489d58
Fix bedrock passing response_format: {"type": "text"} ( #8900 )
...
* fix(converse_transformation.py): ignore type: text, value in response_format
no-op for bedrock
* fix(converse_transformation.py): handle adding response format value to tools
* fix(base_invoke_transformation.py): fix 'get_bedrock_invoke_provider' to handle cross-region-inferencing models
* test(test_bedrock_completion.py): add unit testing for bedrock invoke provider logic
* test: update test
* fix(exception_mapping_utils.py): add context window exceeded error handling for databricks provider route
* fix(fireworks_ai/): support passing tools + response_format together
* fix: cleanup
* fix(base_invoke_transformation.py): fix imports
2025-02-28 20:09:59 -08:00
