Ishaan Jaff
401a52e694
working transform
2025-03-11 15:24:42 -07:00
Ishaan Jaff
368f1de2e1
add OpenAIResponsesAPIConfig
2025-03-11 15:10:34 -07:00
Ishaan Jaff
6462466e02
add OpenAIResponsesAPIConfig
2025-03-11 15:09:42 -07:00
Krrish Dholakia
d99d60a182
refactor(batches/main.py): working refactored azure client init on batches
2025-03-11 14:36:38 -07:00
Krrish Dholakia
af71e14d79
refactor(azure/audio_transcriptions.py): support client init with common logic
2025-03-11 14:24:12 -07:00
Krrish Dholakia
2c2404dac9
refactor(azure.py): working client init logic in azure image generation
2025-03-11 14:22:25 -07:00
Krrish Dholakia
152bc67d22
refactor(azure.py): working azure client init on audio speech endpoint
2025-03-11 14:19:45 -07:00
Krrish Dholakia
f7d9cce536
refactor(azure.py): refactor acompletion to use base azure sdk client
2025-03-11 13:59:13 -07:00
Krrish Dholakia
b58edb7fa1
test(test_azure_common_utils.py): add unit testing for common azure client params function
2025-03-11 12:24:08 -07:00
Krrish Dholakia
69839b3720
refactor(azure/common_utils.py): refactor azure client param logic
...
create common util for azure client param logic
2025-03-11 12:14:50 -07:00
omrishiv
e2adbae9f8
Merge branch 'main' into 8911-fix-model-encoding
2025-03-11 08:28:33 -07:00
Sunny Wan
a775c9ca13
removed handler and refactored to deepseek/chat format
2025-03-11 02:00:52 -04:00
Krish Dholakia
9610c844c7
Merge pull request #9109 from BerriAI/litellm_dev_03_10_2025_p1_v2
...
Read Version from pyproject.toml / read-version (push) Successful in 12s
Return `code`, `param` and `type` on openai bad request error
2025-03-10 22:38:16 -07:00
Krish Dholakia
86a5926e26
Merge pull request #9113 from BerriAI/litellm_dev_03_10_2025_p2
...
fix(base_invoke_transformation.py): support extra_headers on bedrock …
2025-03-10 22:19:10 -07:00
Krrish Dholakia
49792e8cc4
fix: fix linting error
2025-03-10 21:51:55 -07:00
Ishaan Jaff
24b8dcff1f
get_complete_url
2025-03-10 20:45:58 -07:00
omrishiv
12e730885b
encode bedrock model id
...
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
2025-03-10 20:14:35 -07:00
Krrish Dholakia
68bd05ac24
fix(base_invoke_transformation.py): support extra_headers on bedrock invoke route
...
Fixes https://github.com/BerriAI/litellm/issues/9106
2025-03-10 16:13:11 -07:00
Ishaan Jaff
94667e1cf0
Merge pull request #8386 from minwhoo/triton-completions-streaming-fix
...
Fix triton streaming completions bug
2025-03-10 16:07:19 -07:00
Krrish Dholakia
bfbe26b91d
feat(azure.py): add azure bad request error support
2025-03-10 15:59:06 -07:00
Ishaan Jaff
ea058ab4ea
Merge pull request #8746 from niinpatel/patch-1
...
fix missing comma
2025-03-10 15:57:32 -07:00
Krrish Dholakia
5f87dc229a
feat(openai.py): bubble all error information back to client
2025-03-10 15:27:43 -07:00
Krrish Dholakia
c1ec82fbd5
refactor: instrument body param to bubble up on exception
2025-03-10 15:21:04 -07:00
omrishiv
0674491386
add support for Amazon Nova Canvas model ( #7838 )
...
* add initial support for Amazon Nova Canvas model
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* adjust name to AmazonNovaCanvas and map function variables to config
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* tighten model name check
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* fix quality mapping
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add premium quality in config
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* support all Amazon Nova Canvas tasks
* remove unused import
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add tests for image generation tasks and fix payload
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add missing util file
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* update model prices backup file
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* remove image tasks other than text->image
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
---------
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-03-10 08:02:00 -07:00
Krish Dholakia
f899b828cf
Support openrouter reasoning_content on streaming ( #9094 )
...
* feat(convert_dict_to_response.py): support openrouter format of reasoning content
* fix(transformation.py): fix openrouter streaming with reasoning content
Fixes https://github.com/BerriAI/litellm/issues/8193#issuecomment-270892962
* fix: fix type error
2025-03-09 20:03:59 -07:00
Krish Dholakia
e00d4fb18c
Litellm dev 03 08 2025 p3 ( #9089 )
...
* feat(ollama_chat.py): pass down http client to ollama_chat
enables easier testing
* fix(factory.py): fix passing images to ollama's `/api/generate` endpoint
Fixes https://github.com/BerriAI/litellm/issues/6683
* fix(factory.py): fix ollama pt to handle templating correctly
2025-03-09 18:20:56 -07:00
Teja Vishwanadha
8c049dfffc
support bytes.IO for audio transcription ( #9071 )
2025-03-08 08:47:15 -08:00
Ishaan Jaff
e2d612efd9
Bug fix - String data: stripped from entire content in streamed Gemini responses ( #9070 )
...
* _strip_sse_data_from_chunk
* use _strip_sse_data_from_chunk
* use _strip_sse_data_from_chunk
* use _strip_sse_data_from_chunk
* _strip_sse_data_from_chunk
* test_strip_sse_data_from_chunk
* _strip_sse_data_from_chunk
* testing
* _strip_sse_data_from_chunk
2025-03-07 21:06:39 -08:00
Ishaan Jaff
f47987e673
(Refactor) /v1/messages to follow simpler logic for Anthropic API spec ( #9013 )
...
* anthropic_messages_handler v0
* fix /messages
* working messages with router methods
* test_anthropic_messages_handler_litellm_router_non_streaming
* test_anthropic_messages_litellm_router_non_streaming_with_logging
* AnthropicMessagesConfig
* _handle_anthropic_messages_response_logging
* working with /v1/messages endpoint
* working /v1/messages endpoint
* refactor to use router factory function
* use aanthropic_messages
* use BaseConfig for Anthropic /v1/messages
* track api key, team on /v1/messages endpoint
* fix get_logging_payload
* BaseAnthropicMessagesTest
* align test config
* test_anthropic_messages_with_thinking
* test_anthropic_streaming_with_thinking
* fix - display anthropic url for debugging
* test_bad_request_error_handling
* test_anthropic_messages_router_streaming_with_bad_request
* fix ProxyException
* test_bad_request_error_handling_streaming
* use provider_specific_header
* test_anthropic_messages_with_extra_headers
* test_anthropic_messages_to_wildcard_model
* fix gcs pub sub test
* standard_logging_payload
* fix unit testing for anthopic /v1/messages support
* fix pass through anthropic messages api
* delete dead code
* fix anthropic pass through response
* revert change to spend tracking utils
* fix get_litellm_metadata_from_kwargs
* fix spend logs payload json
* proxy_pass_through_endpoint_tests
* TestAnthropicPassthroughBasic
* fix pass through tests
* test_async_vertex_proxy_route_api_key_auth
* _handle_anthropic_messages_response_logging
* vertex_credentials
* test_set_default_vertex_config
* test_anthropic_messages_litellm_router_non_streaming_with_logging
* test_ageneric_api_call_with_fallbacks_basic
* test__aadapter_completion
2025-03-06 00:43:08 -08:00
Krish Dholakia
744e10b0f0
Litellm dev 03 05 2025 p3 ( #9023 )
...
* fix(invoke_handler.py): fix converse streaming - return signature + ensure consistency with anthropic api response
* build(model_prices_and_context_window.json): fix anthropic api claude-3-7 max output tokens
with beta header this is 128k
Resolves https://github.com/BerriAI/litellm/issues/8964
* feat(handler.py): handle new anthropic 'thinking_delta' block on streaming
Fixes https://github.com/BerriAI/litellm/issues/8825
2025-03-05 22:31:39 -08:00
Krish Dholakia
f6535ae6ad
Support format param for specifying image type ( #9019 )
...
* fix(transformation.py): support a 'format' parameter for image's
allow user to specify mime type
* fix: pass mimetype via 'format' param
* feat(gemini/chat/transformation.py): support 'format' param for gemini
* fix(factory.py): support 'format' param on sync bedrock converse calls
* feat(bedrock/converse_transformation.py): support 'format' param for bedrock async calls
* refactor(factory.py): move to supporting 'format' param in base helper
ensures consistency in param support
* feat(gpt_transformation.py): filter out 'format' param
don't send invalid param to openai
* fix(gpt_transformation.py): fix translation
* fix: fix translation error
2025-03-05 19:52:53 -08:00
Krish Dholakia
ec4f665e29
Return signature on anthropic streaming + migrate to signature field instead of signature_delta [MINOR bump] ( #9021 )
...
* Fix missing signature_delta in thinking blocks when streaming from Claude 3.7 (#8797 )
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* test: update test to enforce signature found
* feat(refactor-signature-param-to-be-'signature'-instead-of-'signature_delta'): keeps it in sync with anthropic
* fix: fix linting error
---------
Co-authored-by: Martin Krasser <krasserm@googlemail.com>
2025-03-05 19:33:54 -08:00
Krish Dholakia
c69ec66dc5
fix(base_aws_llm.py): remove region name before sending in args ( #8998 )
...
Read Version from pyproject.toml / read-version (push) Successful in 12s
* fix(base_aws_llm.py): remove region name before sending in args
* fix(base_aws_llm.py): fix optional param pop position
* fix: fix linting error
2025-03-04 23:05:28 -08:00
Krish Dholakia
5e386c28b2
Litellm dev 03 04 2025 p3 ( #8997 )
...
* fix(core_helpers.py): handle litellm_metadata instead of 'metadata'
* feat(batches/): ensure batches logs are written to db
makes batches response dict compatible
* fix(cost_calculator.py): handle batch response being a dictionary
* fix(batches/main.py): modify retrieve endpoints to use @client decorator
enables logging to work on retrieve call
* fix(batches/main.py): fix retrieve batch response type to be 'dict' compatible
* fix(spend_tracking_utils.py): send unique uuid for retrieve batch call type
create batch and retrieve batch share the same id
* fix(spend_tracking_utils.py): prevent duplicate retrieve batch calls from being double counted
* refactor(batches/): refactor cost tracking for batches - do it on retrieve, and within the established litellm_logging pipeline
ensures cost is always logged to db
* fix: fix linting errors
* fix: fix linting error
2025-03-04 21:58:03 -08:00
Krish Dholakia
f1a44d1fdc
fix(common_utils.py): handle $id in response schema when calling vert… ( #8991 )
...
* fix(common_utils.py): handle $id in response schema when calling vertex ai
Fixes issue where `$id` present in response_schema was not accepted by vertex ai
* test(test_vertex.py): add unit test to ensure $id stripped out of vertex schema
2025-03-04 21:19:50 -08:00
Sunny Wan
162ea295e9
[CHORE] Removed old code
2025-03-03 01:45:13 -05:00
Sunny Wan
61ee71745a
[CHORE] Added proper typing
2025-03-03 01:42:48 -05:00
Sunny Wan
b87704cc34
[CHORE] Fixed some style issues and leaks
2025-03-03 01:34:44 -05:00
Sunny Wan
fd090c8043
[FEAT] Added snowflake completion provider
2025-03-03 01:20:00 -05:00
Krrish Dholakia
a63eb58f1b
fix(anthropic/chat/transformation.py): fix headers to be a set
...
avoid duplicates
2025-03-02 08:36:43 -08:00
Krrish Dholakia
4418e6dd14
build: merge branch
2025-03-02 08:31:57 -08:00
Krish Dholakia
c84b489d58
Fix bedrock passing response_format: {"type": "text"} ( #8900 )
...
* fix(converse_transformation.py): ignore type: text, value in response_format
no-op for bedrock
* fix(converse_transformation.py): handle adding response format value to tools
* fix(base_invoke_transformation.py): fix 'get_bedrock_invoke_provider' to handle cross-region-inferencing models
* test(test_bedrock_completion.py): add unit testing for bedrock invoke provider logic
* test: update test
* fix(exception_mapping_utils.py): add context window exceeded error handling for databricks provider route
* fix(fireworks_ai/): support passing tools + response_format together
* fix: cleanup
* fix(base_invoke_transformation.py): fix imports
2025-02-28 20:09:59 -08:00
Krish Dholakia
c8dc4f3eec
converse_transformation: pass 'description' if set in response_format ( #8907 )
...
* test(test_bedrock_completion.py): e2e test ensuring tool description is passed in
* fix(converse_transformation.py): pass description, if set
* fix(transformation.py): Fixes https://github.com/BerriAI/litellm/issues/8767#issuecomment-2689887663
2025-02-28 18:47:07 -08:00
Krish Dholakia
a65bfab697
Fix calling claude via invoke route + response_format support for claude on invoke route ( #8908 )
...
* fix(anthropic_claude3_transformation.py): fix amazon anthropic claude 3 tool calling transformation on invoke route
move to using anthropic config as base
* fix(utils.py): expose anthropic config via providerconfigmanager
* fix(llm_http_handler.py): support json mode on async completion calls
* fix(invoke_handler/make_call): support json mode for anthropic called via bedrock invoke
* fix(anthropic/): handle 'response_format: {"type": "text"}` + migrate amazon claude 3 invoke config to inherit from anthropic config
Prevents error when passing in 'response_format: {"type": "text"}
* test: fix test
* fix(utils.py): fix base invoke provider check
* fix(anthropic_claude3_transformation.py): don't pass 'stream' param
* fix: fix linting errors
* fix(converse_transformation.py): handle response_format type=text for converse
2025-02-28 17:56:26 -08:00
Krish Dholakia
8f86959c32
Litellm dev 02 27 2025 p6 ( #8891 )
...
Read Version from pyproject.toml / read-version (push) Successful in 13s
* fix(http_parsing_utils.py): orjson can throw errors on some emoji's in text, default to json.loads
* fix(sagemaker/handler.py): support passing model id on async streaming
* fix(litellm_pre_call_utils.py): Fixes https://github.com/BerriAI/litellm/issues/7237
2025-02-28 14:34:17 -08:00
Tomáš Dvořák
b2eb2365b9
fix: ollama chat async stream error propagation ( #8870 )
...
Ref: #8868
2025-02-28 08:11:56 -08:00
Jorge Piedrahita Ortiz
ad1357dd0e
update sambanova docs ( #8875 )
2025-02-27 20:23:33 -08:00
Ishaan Jaff
047d1b1208
(Bug Fix) - Accurate token counting for /anthropic/ API Routes on LiteLLM Proxy ( #8880 )
...
* fix _create_anthropic_response_logging_payload
* fix - pass through don't create standard logging payload
* fix logged key hash
* test_init_kwargs_for_pass_through_endpoint_basic
* test_unit_test_anthropic_pass_through
* fix anthropic pass through logging handler
* test_stream_token_counting_anthropic_with_include_usage
* convert_str_chunk_to_generic_chunk
* _build_complete_streaming_response
* test_anthropic_basic_completion_with_headers
* test_anthropic_streaming_with_headers
* improve test for pass through token counting
2025-02-27 15:43:03 -08:00
Krrish Dholakia
2b13fe5b6c
fix(sagemaker/completion/handler.py): fix typo
...
Read Version from pyproject.toml / read-version (push) Successful in 12s
Fixes https://github.com/BerriAI/litellm/issues/8863
2025-02-26 23:55:19 -08:00
Krish Dholakia
88eedb22b9
vertex ai anthropic thinking param support ( #8853 )
...
* fix(vertex_llm_base.py): handle credentials passed in as dictionary
* fix(router.py): support vertex credentials as json dict
* test(test_vertex.py): allows easier testing
mock anthropic thinking response for vertex ai
* test(vertex_ai_partner_models/): don't remove "@" from model
breaks anthropic cost calculation
* test: move testing
* fix: fix linting error
* fix: fix linting error
* fix(vertex_ai_partner_models/main.py): split @ for codestral model
* test: fix test
* fix: fix stripping "@" on mistral models
* fix: fix test
* test: fix test
2025-02-26 21:37:18 -08:00
