phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

allow configuring httpx hooks for AsyncHTTPHandler (#6290) (#6415)

Browse source 
* allow configuring httpx hooks for AsyncHTTPHandler (#6290)

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* Fixes and minor improvements for Helm Chart (#6402)

* reckoner hack

* fix default

* add extracontainers option

* revert chart

* fix extracontainers

* fix deployment

* remove init container

* update docs

* add helm lint to deploy step

* change name

* (refactor) prometheus async_log_success_event to be under 100 LOC  (#6416)

* unit testig for prometheus

* unit testing for success metrics

* use 1 helper for _increment_token_metrics

* use helper for _increment_remaining_budget_metrics

* use _increment_remaining_budget_metrics

* use _increment_top_level_request_and_spend_metrics

* use helper for _set_latency_metrics

* remove noqa violation

* fix test prometheus

* test prometheus

* unit testing for all prometheus helper functions

* fix prom unit tests

* fix unit tests prometheus

* fix unit test prom

* (refactor) router - use static methods for client init utils  (#6420)

* use InitalizeOpenAISDKClient

* use InitalizeOpenAISDKClient static method

* fix  # noqa: PLR0915

* (code cleanup) remove unused and undocumented logging integrations - litedebugger, berrispend  (#6406)

* code cleanup remove unused and undocumented code files

* fix unused logging integrations cleanup

* update chart version

* add circleci tests

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>

* fix: fix linting error

* fix(http_handler.py): fix linting error

---------

Co-authored-by: Alejandro Rodríguez <alejorro70@gmail.com>
Co-authored-by: Robert Brennan <accounts@rbren.io>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
This commit is contained in:
Krish Dholakia 2024-10-24 22:00:24 -07:00 • committed by GitHub
parent 1cd1d23fdf
commit cc8dd80209
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 60 additions and 86 deletions
Download patch file Download diff file Expand all files Collapse all files

25
tests/local_testing/test_utils.py
View file

@ -15,6 +15,7 @@ sys.path.insert(
import pytest
import litellm
from litellm.llms.custom_httpx.http_handler import AsyncHTTPHandler, headers
from litellm.proxy.utils import (
_duration_in_seconds,
_extract_from_regex,
@ -830,6 +831,29 @@ def test_is_base64_encoded():
assert is_base64_encoded(s=base64_image) is True
@mock.patch("httpx.AsyncClient")
@mock.patch.dict(os.environ, {"SSL_VERIFY": "/certificate.pem", "SSL_CERTIFICATE": "/client.pem"}, clear=True)
def test_async_http_handler(mock_async_client):
import httpx
timeout = 120
event_hooks = {"request": [lambda r: r]}
concurrent_limit = 2
AsyncHTTPHandler(timeout, event_hooks, concurrent_limit)
mock_async_client.assert_called_with(
cert="/client.pem",
event_hooks=event_hooks,
headers=headers,
limits=httpx.Limits(
max_connections=concurrent_limit,
max_keepalive_connections=concurrent_limit,
),
timeout=timeout,
verify="/certificate.pem",
)
@pytest.mark.parametrize(
"model, expected_bool", [("gpt-3.5-turbo", False), ("gpt-4o-audio-preview", True)]
)
@ -842,3 +866,4 @@ def test_supports_audio_input(model, expected_bool):
supports_pc = supports_audio_input(model=model)
assert supports_pc == expected_bool