phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

Merge pull request #4652 from msabramo/shorter-success_callbacks-in-health-readiness-response

Browse source 
Shorter success callbacks from `/health/readiness`
This commit is contained in:
Ishaan Jaff 2024-07-11 09:57:52 -07:00 • committed by GitHub
commit b4f8c7304f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 33 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

30
docs/my-website/docs/proxy/health.md
View file

@ -119,30 +119,32 @@ Unprotected endpoint for checking if proxy is ready to accept requests
Example Request: Example Request:
```bash ```bash
curl --location 'http://0.0.0.0:4000/health/readiness' curl http://0.0.0.0:4000/health/readiness
``` ```
Example Response: Example Response:
*If proxy connected to a database*
```json ```json
{ {
"status": "healthy", "status": "connected",
"db": "connected", "db": "connected",
"litellm_version":"1.19.2", "cache": null,
"litellm_version": "1.40.21",
"success_callbacks": [
"langfuse",
"_PROXY_track_cost_callback",
"response_taking_too_long_callback",
"_PROXY_MaxParallelRequestsHandler",
"_PROXY_MaxBudgetLimiter",
"_PROXY_CacheControlCheck",
"ServiceLogging"
],
"last_updated": "2024-07-10T18:59:10.616968"
} }
``` ```
*If proxy not connected to a database* If the proxy is not connected to a database, then the `"db"` field will be `"Not
connected"` instead of `"connected"` and the `"last_updated"` field will not be present.
```json
{
"status": "healthy",
"db": "Not connected",
"litellm_version":"1.19.2",
}
```
## `/health/liveliness` ## `/health/liveliness`

17
litellm/proxy/health_endpoints/_health_endpoints.py
View file

@ -406,6 +406,19 @@ async def active_callbacks():
} }
def callback_name(callback):
if isinstance(callback, str):
return callback
try:
return callback.__name__
except AttributeError:
try:
return callback.__class__.__name__
except AttributeError:
return str(callback)
@router.get( @router.get(
"/health/readiness", "/health/readiness",
tags=["health"], tags=["health"],
@ -424,8 +437,8 @@ async def health_readiness():
try: try:
# this was returning a JSON of the values in some of the callbacks # this was returning a JSON of the values in some of the callbacks
# all we need is the callback name, hence we do str(callback) # all we need is the callback name, hence we do str(callback)
success_callback_names = [str(x) for x in litellm.success_callback] success_callback_names = [callback_name(x) for x in litellm.success_callback]
except: except AttributeError:
# don't let this block the /health/readiness response, if we can't convert to str -> return litellm.success_callback # don't let this block the /health/readiness response, if we can't convert to str -> return litellm.success_callback
success_callback_names = litellm.success_callback success_callback_names = litellm.success_callback