phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-25 10:44:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

(feat) Add usage tracking for streaming /anthropic passthrough routes (#6842)

Browse source 
* use 1 file for AnthropicPassthroughLoggingHandler

* add support for anthropic streaming usage tracking

* ci/cd run again

* fix - add real streaming for anthropic pass through

* remove unused function stream_response

* working anthropic streaming logging

* fix code quality

* fix use 1 file for vertex success handler

* use helper for _handle_logging_vertex_collected_chunks

* enforce vertex streaming to use sse for streaming

* test test_basic_vertex_ai_pass_through_streaming_with_spendlog

* fix type hints

* add comment

* fix linting

* add pass through logging unit testing
This commit is contained in:
Ishaan Jaff 2024-11-21 19:36:03 -08:00 • committed by GitHub
parent 920f4c9f82
commit b8af46e1a2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
12 changed files with 688 additions and 295 deletions
Download patch file Download diff file Expand all files Collapse all files

7
litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py
View file

@ -178,8 +178,11 @@ async def anthropic_proxy_route(
## check for streaming
is_streaming_request = False
if "stream" in str(updated_url):
is_streaming_request = True
# anthropic is streaming when 'stream' = True is in the body
if request.method == "POST":
_request_body = await request.json()
if _request_body.get("stream"):
is_streaming_request = True
## CREATE PASS-THROUGH
endpoint_func = create_pass_through_route(