phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-27 03:34:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix vertex ai multimodal embedding translation (#9471)

Browse source 
* remove data:image/jpeg;base64, prefix from base64 image input

vertex_ai's multimodal embeddings endpoint expects a raw base64 string without `data:image/jpeg;base64,` prefix.

* Add Vertex Multimodal Embedding Test

* fix(test_vertex.py): add e2e tests on multimodal embeddings

* test: unit testing

* test: remove sklearn dep

* test: update test with fixed route

* test: fix test

---------

Co-authored-by: Jonarod <jonrodd@gmail.com>
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
This commit is contained in:
Krish Dholakia 2025-03-24 23:23:28 -07:00 • committed by GitHub
parent 4a91e2116d
commit 6a40d50bc5
6 changed files with 149 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

10
litellm/llms/vertex_ai/multimodal_embeddings/embedding_handler.py
View file

@ -226,7 +226,15 @@ class VertexMultimodalEmbedding(VertexLLM):
else:
return Instance(image=InstanceImage(gcsUri=input_element))
elif is_base64_encoded(s=input_element):
return Instance(image=InstanceImage(bytesBase64Encoded=input_element))
return Instance(
image=InstanceImage(
bytesBase64Encoded=(
input_element.split(",")[1]
if "," in input_element
else input_element
)
)
)
else:
return Instance(text=input_element)