Ishaan Jaff
901d6fe7b7
add operational metrics for pod lock manager v2 arch
2025-04-04 16:41:07 -07:00
Ishaan Jaff
bde88b3ba6
fix type error
2025-04-04 16:34:43 -07:00
Ishaan Jaff
e3b788ea29
fix test
2025-04-02 21:58:35 -07:00
Ishaan Jaff
bcf42fd82d
linting fix prometheus services
2025-04-02 21:19:05 -07:00
Ishaan Jaff
80fb4ece97
prom emit size of DB TX queues for observability
2025-04-02 18:39:29 -07:00
Ishaan Jaff
05b30e28db
clean up service metrics
2025-04-02 17:50:41 -07:00
Krish Dholakia
4e69711411
Litellm dev 01 07 2025 p1 ( #7618 )
...
* fix(main.py): pass custom llm provider on litellm logging provider update
* fix(cost_calculator.py): don't append provider name to return model if existing llm provider
Fixes https://github.com/BerriAI/litellm/issues/7607
* fix(prometheus_services.py): fix prometheus system health error logging
Fixes https://github.com/BerriAI/litellm/issues/7611
2025-01-07 21:22:31 -08:00
Ishaan Jaff
c7f14e936a
(code quality) run ruff rule to ban unused imports ( #7313 )
...
* remove unused imports
* fix AmazonConverseConfig
* fix test
* fix import
* ruff check fixes
* test fixes
* fix testing
* fix imports
2024-12-19 12:33:42 -08:00
Krish Dholakia
b82add11ba
LITELLM: Remove requests library usage ( #7235 )
...
* fix(generic_api_callback.py): remove requests lib usage
* fix(budget_manager.py): remove requests lib usgae
* fix(main.py): cleanup requests lib usage
* fix(utils.py): remove requests lib usage
* fix(argilla.py): fix argilla test
* fix(athina.py): replace 'requests' lib usage with litellm module
* fix(greenscale.py): replace 'requests' lib usage with httpx
* fix: remove unused 'requests' lib import + replace usage in some places
* fix(prompt_layer.py): remove 'requests' lib usage from prompt layer
* fix(ollama_chat.py): remove 'requests' lib usage
* fix(baseten.py): replace 'requests' lib usage
* fix(codestral/): replace 'requests' lib usage
* fix(predibase/): replace 'requests' lib usage
* refactor: cleanup unused 'requests' lib imports
* fix(oobabooga.py): cleanup 'requests' lib usage
* fix(invoke_handler.py): remove unused 'requests' lib usage
* refactor: cleanup unused 'requests' lib import
* fix: fix linting errors
* refactor(ollama/): move ollama to using base llm http handler
removes 'requests' lib dep for ollama integration
* fix(ollama_chat.py): fix linting errors
* fix(ollama/completion/transformation.py): convert non-jpeg/png image to jpeg/png before passing to ollama
2024-12-17 12:50:04 -08:00
Ishaan Jaff
eb47117800
(feat) log error class, function_name on prometheus service failure hook + only log DB related failures on DB service hook ( #6650 )
...
* log error on prometheus service failure hook
* use a more accurate function name for wrapper that handles logging db metrics
* fix log_db_metrics
* test_log_db_metrics_failure_error_types
* fix linting
* fix auth checks
2024-11-07 17:01:18 -08:00
Ishaan Jaff
f05bdd4074
(fix) PrometheusServicesLogger _get_metric should return metric in Registry ( #6486 )
...
* fix logging DB fails on prometheus
* unit testing log to otel wrapper
* unit testing for service logger + prometheus
* use LATENCY buckets for service logging
* fix service logging
* fix _get_metric in prom services logger
* add clear doc string
* unit testing for prom service logger
2024-10-29 21:29:19 +05:30
Ishaan Jaff
69b1bc1f1e
(fix) Prometheus - Log Postgres DB latency, status on prometheus ( #6484 )
...
* fix logging DB fails on prometheus
* unit testing log to otel wrapper
* unit testing for service logger + prometheus
* use LATENCY buckets for service logging
* fix service logging
2024-10-29 12:17:35 +05:30
Krish Dholakia
d57be47b0f
Litellm ruff linting enforcement ( #5992 )
...
* ci(config.yml): add a 'check_code_quality' step
Addresses https://github.com/BerriAI/litellm/issues/5991
* ci(config.yml): check why circle ci doesn't pick up this test
* ci(config.yml): fix to run 'check_code_quality' tests
* fix(__init__.py): fix unprotected import
* fix(__init__.py): don't remove unused imports
* build(ruff.toml): update ruff.toml to ignore unused imports
* fix: fix: ruff + pyright - fix linting + type-checking errors
* fix: fix linting errors
* fix(lago.py): fix module init error
* fix: fix linting errors
* ci(config.yml): cd into correct dir for checks
* fix(proxy_server.py): fix linting error
* fix(utils.py): fix bare except
causes ruff linting errors
* fix: ruff - fix remaining linting errors
* fix(clickhouse.py): use standard logging object
* fix(__init__.py): fix unprotected import
* fix: ruff - fix linting errors
* fix: fix linting errors
* ci(config.yml): cleanup code qa step (formatting handled in local_testing)
* fix(_health_endpoints.py): fix ruff linting errors
* ci(config.yml): just use ruff in check_code_quality pipeline for now
* build(custom_guardrail.py): include missing file
* style(embedding_handler.py): fix ruff check
2024-10-01 19:44:20 -04:00
Krrish Dholakia
4a3b084961
feat(bedrock_httpx.py): moves to using httpx client for bedrock cohere calls
2024-05-11 13:43:08 -07:00
Krrish Dholakia
6575143460
feat(proxy_server.py): return litellm version in response headers
2024-05-08 16:00:08 -07:00
Krrish Dholakia
161e836427
fix(utils.py): fix 'no-cache': true when caching is turned on
2024-04-23 12:58:30 -07:00
Krrish Dholakia
9379e3d047
fix(lowest_tpm_rpm_v2.py): use a combined tpm+rpm query in async get cache, to reduce redis client calls in high traffic
2024-04-20 16:13:11 -07:00
Krish Dholakia
6d9f0f1839
Merge branch 'main' into litellm_ssl_caching_fix
2024-04-19 17:20:27 -07:00
Krrish Dholakia
0d9c96bebf
test(test_prometheus_services.py): fix testing to handle caching ping in init
2024-04-19 16:15:29 -07:00
Krrish Dholakia
01a1a8f731
fix(caching.py): dual cache async_batch_get_cache fix + testing
...
this fixes a bug in usage-based-routing-v2 which was caused b/c of how the result was being returned from dual cache async_batch_get_cache. it also adds unit testing for that function (and it's sync equivalent)
2024-04-19 15:03:25 -07:00
Krrish Dholakia
0f95a824c4
feat(prometheus_services.py): emit proxy latency for successful llm api requests
...
uses prometheus histogram for this
2024-04-18 16:04:35 -07:00
Krrish Dholakia
67d0d5e356
fix(prometheus_services.py): add better import error statement
2024-04-13 19:03:32 -07:00
Krrish Dholakia
bef24cd4ab
fix(caching.py): don't decode a string
2024-04-13 18:48:03 -07:00
Krrish Dholakia
9f42d15713
feat(prometheus_services.py): track when redis calls fail
2024-04-13 18:31:35 -07:00
Krrish Dholakia
4e81acf2c6
feat(prometheus_services.py): monitor health of proxy adjacent services (redis / postgres / etc.)
2024-04-13 18:15:02 -07:00
