phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

(docs + testing) Correctly document the timeout value used by litellm proxy is 6000 seconds + add to best practices for prod (#6339)

Browse source 
* fix docs use documented timeout

* document request timeout

* add test for litellm.request_timeout

* add test for checking value of timeout
This commit is contained in:
Ishaan Jaff 2024-10-23 14:09:35 +05:30 • committed by GitHub
parent 64c3d3210c
commit 807e9dcea8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 54 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

20
tests/local_testing/test_proxy_server.py
View file

@ -173,6 +173,26 @@ def test_chat_completion(mock_acompletion, client_no_auth):
pytest.fail(f"LiteLLM Proxy test failed. Exception - {str(e)}")
def test_get_settings_request_timeout(client_no_auth):
"""
When no timeout is set, it should use the litellm.request_timeout value
"""
# Set a known value for litellm.request_timeout
import litellm
# Make a GET request to /settings
response = client_no_auth.get("/settings")
# Check if the request was successful
assert response.status_code == 200
# Parse the JSON response
settings = response.json()
print("settings", settings)
assert settings["litellm.request_timeout"] == litellm.request_timeout
@pytest.mark.parametrize(
"litellm_key_header_name",
["x-litellm-key", None],