Merge pull request #4652 from msabramo/shorter-success_callbacks-in-health-readiness-response

Shorter success callbacks from `/health/readiness`
This commit is contained in:
Ishaan Jaff 2024-07-11 09:57:52 -07:00 committed by GitHub
commit b4f8c7304f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 33 additions and 18 deletions

View file

@ -118,31 +118,33 @@ Unprotected endpoint for checking if proxy is ready to accept requests
Example Request:
```bash
curl --location 'http://0.0.0.0:4000/health/readiness'
```bash
curl http://0.0.0.0:4000/health/readiness
```
Example Response:
*If proxy connected to a database*
```json
{
"status": "healthy",
"db": "connected",
"litellm_version":"1.19.2",
"status": "connected",
"db": "connected",
"cache": null,
"litellm_version": "1.40.21",
"success_callbacks": [
"langfuse",
"_PROXY_track_cost_callback",
"response_taking_too_long_callback",
"_PROXY_MaxParallelRequestsHandler",
"_PROXY_MaxBudgetLimiter",
"_PROXY_CacheControlCheck",
"ServiceLogging"
],
"last_updated": "2024-07-10T18:59:10.616968"
}
```
*If proxy not connected to a database*
```json
{
"status": "healthy",
"db": "Not connected",
"litellm_version":"1.19.2",
}
```
If the proxy is not connected to a database, then the `"db"` field will be `"Not
connected"` instead of `"connected"` and the `"last_updated"` field will not be present.
## `/health/liveliness`

View file

@ -406,6 +406,19 @@ async def active_callbacks():
}
def callback_name(callback):
if isinstance(callback, str):
return callback
try:
return callback.__name__
except AttributeError:
try:
return callback.__class__.__name__
except AttributeError:
return str(callback)
@router.get(
"/health/readiness",
tags=["health"],
@ -424,8 +437,8 @@ async def health_readiness():
try:
# this was returning a JSON of the values in some of the callbacks
# all we need is the callback name, hence we do str(callback)
success_callback_names = [str(x) for x in litellm.success_callback]
except:
success_callback_names = [callback_name(x) for x in litellm.success_callback]
except AttributeError:
# don't let this block the /health/readiness response, if we can't convert to str -> return litellm.success_callback
success_callback_names = litellm.success_callback