phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-26 19:24:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

refactor: make bedrock image transformation requests async (#7840)

Browse source 
* refactor: initial commit for using separate sync vs. async transformation routes for bedrock

ensures no blocking calls e.g. when converting image url to b64

* perf(converse_transformation.py): make bedrock converse transformation async

asyncify's the bedrock message transformation - useful for handling image urls for bedrock

* fix(converse_handler.py): fix logging for async streaming

* style: cleanup unused imports
This commit is contained in:
Krish Dholakia 2025-01-17 20:14:15 -08:00 • committed by GitHub
parent 32c8933935
commit 2b58f16fda
8 changed files with 266 additions and 93 deletions
Download patch file Download diff file Expand all files Collapse all files

2
tests/local_testing/test_streaming.py
View file

@ -1387,7 +1387,7 @@ async def test_completion_replicate_llama3_streaming(sync_mode):
[
# ["bedrock/ai21.jamba-instruct-v1:0", "us-east-1"],
# ["bedrock/cohere.command-r-plus-v1:0", None],
# ["anthropic.claude-3-sonnet-20240229-v1:0", None],
["anthropic.claude-3-sonnet-20240229-v1:0", None],
# ["anthropic.claude-instant-v1", None],
# ["mistral.mistral-7b-instruct-v0:2", None],
["bedrock/amazon.titan-tg1-large", None],