Krish Dholakia
217681eb5e
Litellm dev 04 22 2025 p1 ( #10206 )
...
* fix(openai.py): initial commit adding generic event type for openai responses api streaming
Ensures handling for undocumented event types - e.g. "response.reasoning_summary_part.added"
* fix(transformation.py): handle unknown openai response type
* fix(datadog_llm_observability.py): handle dict[str, any] -> dict[str, str] conversion
Fixes https://github.com/BerriAI/litellm/issues/9494
* test: add more unit testing
* test: add unit test
* fix(common_utils.py): fix message with content list
* test: update testing
2025-04-22 23:58:43 -07:00
Ishaan Jaff
868cdd0226
[Feat] Add Support for DELETE /v1/responses/{response_id} on OpenAI, Azure OpenAI ( #10205 )
...
* add transform_delete_response_api_request to base responses config
* add transform_delete_response_api_request
* add delete_response_api_handler
* fixes for deleting responses, response API
* add adelete_responses
* add async test_basic_openai_responses_delete_endpoint
* test_basic_openai_responses_delete_endpoint
* working delete for streaming on responses API
* fixes azure transformation
* TestAnthropicResponsesAPITest
* fix code check
* fix linting
* fixes for get_complete_url
* test_basic_openai_responses_streaming_delete_endpoint
* streaming fixes
2025-04-22 18:27:03 -07:00
Ishaan Jaff
0c2f705417
[Feat] Add Responses API - Routing Affinity logic for sessions ( #10193 )
...
* test for test_responses_api_routing_with_previous_response_id
* test_responses_api_routing_with_previous_response_id
* add ResponsesApiDeploymentCheck
* ResponsesApiDeploymentCheck
* ResponsesApiDeploymentCheck
* fix ResponsesApiDeploymentCheck
* test_responses_api_routing_with_previous_response_id
* ResponsesApiDeploymentCheck
* test_responses_api_deployment_check.py
* docs routing affinity
* simplify ResponsesApiDeploymentCheck
* test response id
* fix code quality check
2025-04-21 20:00:27 -07:00
Ishaan Jaff
f39d917886
[Docs] Responses API ( #10172 )
...
* docs litellm responses api
* doc fix
* docs responses API
* add get_supported_openai_params for LiteLLMCompletionResponsesConfig
* add Supported Responses API Parameters
2025-04-19 17:10:45 -07:00
Ishaan Jaff
0717369ae6
[Feat] Expose Responses API on LiteLLM UI Test Key Page ( #10166 )
...
* add /responses API on UI
* add makeOpenAIResponsesRequest
* add makeOpenAIResponsesRequest
* fix add responses API on UI
* fix endpoint selector
* responses API render chunks on litellm chat ui
* fixes to streaming iterator
* fix render responses completed events
* fixes for MockResponsesAPIStreamingIterator
* transform_responses_api_request_to_chat_completion_request
* fix for responses API
* test_basic_openai_responses_api_streaming
* fix base responses api tests
2025-04-19 13:18:54 -07:00
Ishaan Jaff
3d5022bd79
[Feat] Support for all litellm providers on Responses API (works with Codex) - Anthropic, Bedrock API, VertexAI, Ollama ( #10132 )
...
* transform request
* basic handler for LiteLLMCompletionTransformationHandler
* complete transform litellm to responses api
* fixes to test
* fix stream=True
* fix streaming iterator
* fixes for transformation
* fixes for anthropic codex support
* fix pass response_api_optional_params
* test anthropic responses api tools
* update responses types
* working codex with litellm
* add session handler
* fixes streaming iterator
* fix handler
* add litellm codex example
* fix code quality
* test fix
* docs litellm codex
* litellm codexdoc
* docs openai codex with litellm
* docs litellm openai codex
* litellm codex
* linting fixes for transforming responses API
* fix import error
* fix responses api test
* add sync iterator support for responses api
2025-04-18 19:53:59 -07:00
Ishaan Jaff
d3e04eac7f
[Feat] Unified Responses API - Add Azure Responses API support ( #10116 )
...
* initial commit for azure responses api support
* update get complete url
* fixes for responses API
* working azure responses API
* working responses API
* test suite for responses API
* azure responses API test suite
* fix test with complete url
* fix test refactor
* test fix metadata checks
* fix code quality check
2025-04-17 16:47:59 -07:00
Krish Dholakia
9b7ebb6a7d
build(pyproject.toml): add new dev dependencies - for type checking ( #9631 )
...
* build(pyproject.toml): add new dev dependencies - for type checking
* build: reformat files to fit black
* ci: reformat to fit black
* ci(test-litellm.yml): make tests run clear
* build(pyproject.toml): add ruff
* fix: fix ruff checks
* build(mypy/): fix mypy linting errors
* fix(hashicorp_secret_manager.py): fix passing cert for tls auth
* build(mypy/): resolve all mypy errors
* test: update test
* fix: fix black formatting
* build(pre-commit-config.yaml): use poetry run black
* fix(proxy_server.py): fix linting error
* fix: fix ruff safe representation error
2025-03-29 11:02:13 -07:00
Krish Dholakia
5ac61a7572
Add bedrock latency optimized inference support ( #9623 )
...
* fix(converse_transformation.py): add performanceConfig param support on bedrock
Closes https://github.com/BerriAI/litellm/issues/7606
* fix(converse_transformation.py): refactor to use more flexible single getter for params which are separate config blocks
* test(test_main.py): add e2e mock test for bedrock performance config
* build(model_prices_and_context_window.json): add versioned multimodal embedding
* refactor(multimodal_embeddings/): migrate to config pattern
* feat(vertex_ai/multimodalembeddings): calculate usage for multimodal embedding calls
Enables cost calculation for multimodal embeddings
* feat(vertex_ai/multimodalembeddings): get usage object for embedding calls
ensures accurate cost tracking for vertexai multimodal embedding calls
* fix(embedding_handler.py): remove unused imports
* fix: fix linting errors
* fix: handle response api usage calculation
* test(test_vertex_ai_multimodal_embedding_transformation.py): update tests
* test: mark flaky test
* feat(vertex_ai/multimodal_embeddings/transformation.py): support text+image+video input
* docs(vertex.md): document sending text + image to vertex multimodal embeddings
* test: remove incorrect file
* fix(multimodal_embeddings/transformation.py): fix linting error
* style: remove unused import
2025-03-29 00:23:09 -07:00
Ishaan Jaff
af8e917eb0
MockResponsesAPIStreamingIterator
2025-03-20 12:25:58 -07:00
Ishaan Jaff
c64f834272
add AsyncMockResponsesAPIStreamingIterator
2025-03-20 12:06:36 -07:00
Ishaan Jaff
0cd671785d
add fake_stream to llm http handler
2025-03-20 09:55:59 -07:00
Ishaan Jaff
15d618f5b1
Add exception mapping for responses API
2025-03-13 15:57:58 -07:00
Ishaan Jaff
de473bee4b
fix mypy linting errors
2025-03-12 12:13:19 -07:00
Ishaan Jaff
1f7c21fd1b
remove infinit loop for streaming
2025-03-12 11:55:17 -07:00
Ishaan Jaff
181072e15d
fix code quality checks
2025-03-12 11:47:10 -07:00
Ishaan Jaff
d6351c3433
test_basic_openai_responses_api
2025-03-12 10:07:03 -07:00
Ishaan Jaff
35e9bba154
_handle_logging_completed_response
2025-03-12 09:42:48 -07:00
Ishaan Jaff
d6a49f6b66
explictly pass params to partial func
2025-03-12 09:37:38 -07:00
Ishaan Jaff
ffa4978f8a
ResponsesAPIRequestUtils
2025-03-12 09:36:08 -07:00
Ishaan Jaff
58acf23c3e
STREAM_SSE_DONE_STRING
2025-03-12 09:33:28 -07:00
Ishaan Jaff
aa250088b2
re-use base_llm_http_handler
2025-03-12 09:31:34 -07:00
Ishaan Jaff
047879c004
add aresponses
2025-03-12 09:22:44 -07:00
Ishaan Jaff
e4cda0a1b7
add SyncResponsesAPIStreamingIterator
2025-03-12 09:18:25 -07:00
Ishaan Jaff
fde75a068a
working streaming logging
2025-03-12 00:02:39 -07:00
Ishaan Jaff
b790f0a5c6
log input of response API
2025-03-11 22:34:18 -07:00
Ishaan Jaff
51dc24a405
_transform_response_api_usage_to_chat_usage
2025-03-11 22:26:44 -07:00
Ishaan Jaff
20e3398476
fix typing for aresponses
2025-03-11 22:03:48 -07:00
Ishaan Jaff
24cb83b0e4
Response API cost tracking
2025-03-11 22:02:14 -07:00
Ishaan Jaff
8fa313ab07
add async streaming support
2025-03-11 20:00:42 -07:00
Ishaan Jaff
aa40cb5b26
working ResponsesAPIStreamingIterator
2025-03-11 19:47:43 -07:00
Ishaan Jaff
f32968409e
working basic openai response api request
2025-03-11 17:37:19 -07:00
Ishaan Jaff
2c6774e3ee
get_optional_params_responses_api
2025-03-11 16:00:49 -07:00
Ishaan Jaff
8c4331638e
add aysnc aresponses
2025-03-11 14:33:50 -07:00
Ishaan Jaff
4b1b87eb67
openai reasoning initial types
2025-03-11 14:28:47 -07:00
