Ishaan Jaff
d1ff7a5f45
allow proxy to startup on DB unavailable
2025-03-26 19:50:57 -07:00
Ishaan Jaff
8afef3908b
fix code quality
2025-03-14 21:06:28 -07:00
Ishaan Jaff
dede5125bb
test_model_connection
2025-03-14 18:33:49 -07:00
Ishaan Jaff
b7f13031e3
fix endpoint_data
2025-03-14 17:21:01 -07:00
Ishaan Jaff
f62acaa7eb
fix endpoint
2025-03-14 12:26:17 -07:00
Ishaan Jaff
912c169ef9
add health/test_connection
2025-03-14 11:30:24 -07:00
Krish Dholakia
fe460f19f5
Add datadog health check support + fix bedrock converse cost tracking w/ region name specified ( #7958 )
...
* fix(bedrock/converse_handler.py): fix bedrock region name on async calls
* fix(utils.py): fix split model handling
Fixes bedrock cost calculation when region name is given
* feat(_health_endpoints.py): support health checking datadog integration
Closes https://github.com/BerriAI/litellm/issues/7921
2025-01-23 22:17:09 -08:00
Ishaan Jaff
62a1cdec47
(code quality) run ruff rule to ban unused imports ( #7313 )
...
* remove unused imports
* fix AmazonConverseConfig
* fix test
* fix import
* ruff check fixes
* test fixes
* fix testing
* fix imports
2024-12-19 12:33:42 -08:00
Ishaan Jaff
a74a07e459
(fix) Langfuse key based logging ( #6372 )
...
* langfuse use helper for get_langfuse_logging_config
* fix get_langfuse_logger_for_request
* fix import
* fix get_langfuse_logger_for_request
* test_get_langfuse_logger_for_request_with_dynamic_params
* unit testing for test_get_langfuse_logger_for_request_with_no_dynamic_params
* parameterized langfuse testing
* fix langfuse test
* fix langfuse logging
* fix test_aaalangfuse_logging_metadata
* fix langfuse log metadata test
* fix langfuse logger
* use create_langfuse_logger_from_credentials
* fix test_get_langfuse_logger_for_request_with_no_dynamic_params
* fix correct langfuse/ folder structure
* use static methods for langfuse logger
* add commment on langfuse handler
* fix linting error
* add unit testing for langfuse logging
* fix linting
* fix failure handler langfuse
2024-10-23 18:24:22 +05:30
Ishaan Jaff
06e0dd9b98
(docs + testing) Correctly document the timeout value used by litellm proxy is 6000 seconds + add to best practices for prod ( #6339 )
...
* fix docs use documented timeout
* document request timeout
* add test for litellm.request_timeout
* add test for checking value of timeout
2024-10-23 14:09:35 +05:30
Ishaan Jaff
0c5a47c404
(code quality) add ruff check PLR0915 for too-many-statements ( #6309 )
...
* ruff add PLR0915
* add noqa for PLR0915
* fix noqa
* add # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* add # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
2024-10-18 15:36:49 +05:30
Ishaan Jaff
ba56e37244
(refactor) caching use LLMCachingHandler for async_get_cache and set_cache ( #6208 )
...
* use folder for caching
* fix importing caching
* fix clickhouse pyright
* fix linting
* fix correctly pass kwargs and args
* fix test case for embedding
* fix linting
* fix embedding caching logic
* fix refactor handle utils.py
* fix test_embedding_caching_azure_individual_items_reordered
2024-10-14 16:34:01 +05:30
Krish Dholakia
94a05ca5d0
Litellm ruff linting enforcement ( #5992 )
...
* ci(config.yml): add a 'check_code_quality' step
Addresses https://github.com/BerriAI/litellm/issues/5991
* ci(config.yml): check why circle ci doesn't pick up this test
* ci(config.yml): fix to run 'check_code_quality' tests
* fix(__init__.py): fix unprotected import
* fix(__init__.py): don't remove unused imports
* build(ruff.toml): update ruff.toml to ignore unused imports
* fix: fix: ruff + pyright - fix linting + type-checking errors
* fix: fix linting errors
* fix(lago.py): fix module init error
* fix: fix linting errors
* ci(config.yml): cd into correct dir for checks
* fix(proxy_server.py): fix linting error
* fix(utils.py): fix bare except
causes ruff linting errors
* fix: ruff - fix remaining linting errors
* fix(clickhouse.py): use standard logging object
* fix(__init__.py): fix unprotected import
* fix: ruff - fix linting errors
* fix: fix linting errors
* ci(config.yml): cleanup code qa step (formatting handled in local_testing)
* fix(_health_endpoints.py): fix ruff linting errors
* ci(config.yml): just use ruff in check_code_quality pipeline for now
* build(custom_guardrail.py): include missing file
* style(embedding_handler.py): fix ruff check
2024-10-01 19:44:20 -04:00
Ishaan Jaff
7dd44f8586
(feat proxy slack alerting) - allow opting in to getting key / internal user alerts ( #5990 )
...
* define all slack alert types
* use correct type hints for alert type
* use correct defaults on slack alerting
* add readme for slack alerting
* fix linting error
* update readme
* docs all alert types
* update slack alerting docs
* fix slack alerting docs
* handle new testing dir structure
* fix config for testing
* fix testing folder related imports
* fix /tests import errors
* fix import stream_chunk_testdata
* docs alert types
* fix test test_langfuse_trace_id
* fix type checks for slack alerting
* fix outage alerting test slack
2024-10-01 10:49:22 -07:00
Ishaan Jaff
0d027b22fd
[Feat-Proxy] Slack Alerting - allow using os.environ/ vars for alert to webhook url ( #5726 )
...
* allow using os.environ for slack urls
* use env vars for webhook urls
* fix types for get_secret
* fix linting
* fix linting
* fix linting
* linting fixes
* linting fix
* docs alerting slack
* fix get data
2024-09-16 18:03:37 -07:00
Ishaan Jaff
00832bb8f2
fix email health checks
2024-08-06 15:59:26 -07:00
Krrish Dholakia
e60bec92ac
feat(ui): add ability to enable traceloop + langsmith via ui
2024-07-31 21:40:29 -07:00
Krrish Dholakia
c5e38b3bec
feat(ui): add braintrust logging to ui
2024-07-31 21:15:19 -07:00
Ishaan Jaff
3375351323
Merge pull request #4781 from msabramo/liveness-alias-of-liveliness
...
Alias `/health/liveliness` as `/health/liveness`
2024-07-18 22:20:09 -07:00
Marc Abramowitz
692080c457
Alias /health/liveliness as /health/liveness
...
The latter is the more common term in Kubernetes, so it's nice to support that.
2024-07-18 14:29:32 -07:00
Florian Greinacher
d83a22b392
feat(proxy): support hiding health check details
2024-07-18 17:21:12 +02:00
Marc Abramowitz
9d2cfe6933
Shorter success callbacks from /health/readiness
...
Before:
```shell
$ curl -sSL http://0.0.0.0:4000/health/readiness | jq '.success_callbacks'
[
"langfuse",
"<function _PROXY_track_cost_callback at 0x12fc14b80>",
"<bound method SlackAlerting.response_taking_too_long_callback of <litellm.integrations.slack_alerting.SlackAlerting object at 0x12cedb740>>",
"<litellm.proxy.hooks.parallel_request_limiter._PROXY_MaxParallelRequestsHandler object at 0x12cedb8f0>",
"<litellm.proxy.hooks.max_budget_limiter._PROXY_MaxBudgetLimiter object at 0x12cedb830>",
"<litellm.proxy.hooks.cache_control_check._PROXY_CacheControlCheck object at 0x12ca101d0>",
"<litellm._service_logger.ServiceLogging object at 0x13a6d8c50>"
]
```
After:
```shell
$ curl -sSL http://0.0.0.0:4000/health/readiness | jq '.success_callbacks'
[
"langfuse",
"_PROXY_track_cost_callback",
"response_taking_too_long_callback",
"_PROXY_MaxParallelRequestsHandler",
"_PROXY_MaxBudgetLimiter",
"_PROXY_CacheControlCheck",
"ServiceLogging"
]
```
2024-07-10 18:45:42 -07:00
Ishaan Jaff
170150e02e
use ProxyErrorTypes,
2024-07-08 12:47:53 -07:00
Ishaan Jaff
69c32eaa0e
add options for /health/readiness and liveliness
2024-06-19 12:13:35 -07:00
Ishaan Jaff
c4a8177f89
add health/readiness OPTIONS
2024-06-19 12:00:48 -07:00
Ishaan Jaff
0ae86abd56
feat - support OPTIONS for health endpoints
2024-06-19 09:55:38 -07:00
Ishaan Jaff
abb544190a
fix - refactor key endpoints
2024-06-15 14:13:32 -07:00
