phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-25 10:44:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

(feat) use @google-cloud/vertexai js sdk with litellm (#6873)

Browse source 
* stash gemini JS test

* add vertex js sdj example

* handle vertex pass through separately

* tes vertex JS sdk

* fix vertex_proxy_route

* use PassThroughStreamingHandler

* fix PassThroughStreamingHandler

* use common _create_vertex_response_logging_payload_for_generate_content

* test vertex js

* add working vertex jest tests

* move basic bass through test

* use good name for test

* test vertex

* test_chunk_processor_yields_raw_bytes

* unit tests for streaming

* test_convert_raw_bytes_to_str_lines

* run unit tests 1st

* simplify local

* docs add usage example for js

* use get_litellm_virtual_key

* add unit tests for vertex pass through
This commit is contained in:
Ishaan Jaff 2024-11-22 16:50:10 -08:00 • committed by GitHub
parent 5930c42e74
commit b2b3e40d13
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
14 changed files with 680 additions and 89 deletions
Download patch file Download diff file Expand all files Collapse all files

6
litellm/proxy/pass_through_endpoints/pass_through_endpoints.py
View file

@ -36,7 +36,7 @@ from litellm.proxy._types import (
from litellm.proxy.auth.user_api_key_auth import user_api_key_auth
from litellm.secret_managers.main import get_secret_str
from .streaming_handler import chunk_processor
from .streaming_handler import PassThroughStreamingHandler
from .success_handler import PassThroughEndpointLogging
from .types import EndpointType, PassthroughStandardLoggingPayload
@ -448,7 +448,7 @@ async def pass_through_request( # noqa: PLR0915
)
return StreamingResponse(
chunk_processor(
PassThroughStreamingHandler.chunk_processor(
response=response,
request_body=_parsed_body,
litellm_logging_obj=logging_obj,
@ -491,7 +491,7 @@ async def pass_through_request( # noqa: PLR0915
)
return StreamingResponse(
chunk_processor(
PassThroughStreamingHandler.chunk_processor(
response=response,
request_body=_parsed_body,
litellm_logging_obj=logging_obj,