Krish Dholakia
4f8a3fd4cf
redis otel tracing + async support for latency routing ( #6452 )
...
* docs(exception_mapping.md): add missing exception types
Fixes https://github.com/Aider-AI/aider/issues/2120#issuecomment-2438971183
* fix(main.py): register custom model pricing with specific key
Ensure custom model pricing is registered to the specific model+provider key combination
* test: make testing more robust for custom pricing
* fix(redis_cache.py): instrument otel logging for sync redis calls
ensures complete coverage for all redis cache calls
* refactor: pass parent_otel_span for redis caching calls in router
allows for more observability into what calls are causing latency issues
* test: update tests with new params
* refactor: ensure e2e otel tracing for router
* refactor(router.py): add more otel tracing acrosss router
catch all latency issues for router requests
* fix: fix linting error
* fix(router.py): fix linting error
* fix: fix test
* test: fix tests
* fix(dual_cache.py): pass ttl to redis cache
* fix: fix param
2024-10-28 21:52:12 -07:00
Krish Dholakia
d59f8f952d
perf: remove 'always_read_redis' - adding +830ms on each llm call ( #6414 )
...
* perf: remove 'always_read_redis' - adding +830ms on each llm call
* test: cleanup codestral tests - backend api unavailable
2024-10-24 17:48:36 -07:00
Ishaan Jaff
b75019c1a5
(feat) Arize - Allow using Arize HTTP endpoint ( #6364 )
...
* arize use helper for get_arize_opentelemetry_config
* use helper to get Arize OTEL config
* arize add helpers for arize
* docs allow using arize http endpoint
* fix importing OTEL for Arize
* use static methods for ArizeLogger
* fix ArizeLogger tests
2024-10-23 09:38:35 +05:30
Ishaan Jaff
610974b4fc
(code quality) add ruff check PLR0915 for too-many-statements ( #6309 )
...
* ruff add PLR0915
* add noqa for PLR0915
* fix noqa
* add # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* add # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
2024-10-18 15:36:49 +05:30
Ishaan Jaff
4554bf760c
(testing) add test coverage for LLM OTEL logging ( #6227 )
...
* add test coverage for OTEL logging
* test_async_otel_callback
* test test_awesome_otel_with_message_logging_off
* fix otel testing
* add otel testing
* otel testing
* otel testing
* otel testing coverage
* otel add testing
2024-10-17 16:34:04 +05:30
Ishaan Jaff
846bb4cb91
(refactor) OTEL - use safe_set_attribute for setting attributes ( #6226 )
...
* otel - use safe_set_attribute for setting attributes
* fix OTEL only use safe_set_attribute
2024-10-15 13:39:29 +05:30
Ali Waleed
7ec414a3cf
Feat: Add Langtrace integration ( #5341 )
...
* Feat: Add Langtrace integration
* add langtrace service name
* fix timestamps for traces
* add tests
* Discard Callback + use existing otel logger
* cleanup
* remove print statments
* remove callback
* add docs
* docs
* add logging docs
* format logging
* remove emoji and add litellm proxy example
* format logging
* format `logging.md`
* add langtrace docs to logging.md
* sync conflict
2024-10-11 19:19:53 +05:30
Krish Dholakia
d57be47b0f
Litellm ruff linting enforcement ( #5992 )
...
* ci(config.yml): add a 'check_code_quality' step
Addresses https://github.com/BerriAI/litellm/issues/5991
* ci(config.yml): check why circle ci doesn't pick up this test
* ci(config.yml): fix to run 'check_code_quality' tests
* fix(__init__.py): fix unprotected import
* fix(__init__.py): don't remove unused imports
* build(ruff.toml): update ruff.toml to ignore unused imports
* fix: fix: ruff + pyright - fix linting + type-checking errors
* fix: fix linting errors
* fix(lago.py): fix module init error
* fix: fix linting errors
* ci(config.yml): cd into correct dir for checks
* fix(proxy_server.py): fix linting error
* fix(utils.py): fix bare except
causes ruff linting errors
* fix: ruff - fix remaining linting errors
* fix(clickhouse.py): use standard logging object
* fix(__init__.py): fix unprotected import
* fix: ruff - fix linting errors
* fix: fix linting errors
* ci(config.yml): cleanup code qa step (formatting handled in local_testing)
* fix(_health_endpoints.py): fix ruff linting errors
* ci(config.yml): just use ruff in check_code_quality pipeline for now
* build(custom_guardrail.py): include missing file
* style(embedding_handler.py): fix ruff check
2024-10-01 19:44:20 -04:00
Ishaan Jaff
b468c58fe5
chore - handle case when otel metadata field value is None
2024-09-24 22:24:12 -07:00
Ishaan Jaff
3ccdb42d26
[Fix] OTEL - Don't log messages when callback settings disable message logging ( #5875 )
...
* fix otel dont log messages
* otel fix redis failure hook logging
2024-09-24 18:29:52 -07:00
Ishaan Jaff
91e58d9049
[Feat] Add proxy level prometheus metrics ( #5789 )
...
* add Proxy Level Tracking Metrics doc
* update service logger
* prometheus - track litellm_proxy_failed_requests_metric
* use REQUESTED_MODEL
* fix prom request_data
2024-09-19 17:13:07 -07:00
Ishaan Jaff
3376f151c6
fix otel use sensible defaults
2024-09-11 14:24:04 -07:00
Ishaan Jaff
479b12be09
Merge branch 'main' into litellm_allow_turning_off_message_logging_for_callbacks
2024-09-09 21:59:36 -07:00
Ishaan Jaff
f742d6162f
fix otel defaults
2024-09-09 16:18:55 -07:00
Ishaan Jaff
b36f964217
fix init custom logger when init OTEL runs
2024-09-09 16:03:39 -07:00
Ishaan Jaff
356ad9b22b
fix otel set max_queue_size, max_queue_size
2024-09-06 17:31:43 -07:00
Ishaan Jaff
dcb50243e7
fix otel max batch size
2024-09-06 17:12:01 -07:00
Ishaan Jaff
31a5cb1885
fix otel type
2024-09-05 19:04:56 -07:00
Ishaan Jaff
6e4098fa26
otel log service errors
2024-08-05 20:21:26 -07:00
Ishaan Jaff
05a2a4462e
otel log event_metadata
2024-08-05 20:01:58 -07:00
Ishaan Jaff
96af712605
fix _handle_failure for otel
2024-08-01 11:12:31 -07:00
Ishaan Jaff
19fb5cc11c
use common helpers for writing to otel
2024-07-27 11:40:39 -07:00
Ishaan Jaff
2a89486948
move _get_parent_otel_span_from_kwargs to otel.py
2024-07-27 11:12:13 -07:00
Krrish Dholakia
fe0b55f2ca
fix(utils.py): fix cache hits for streaming
...
Fixes https://github.com/BerriAI/litellm/issues/4109
2024-07-26 19:04:08 -07:00
Krrish Dholakia
548e4f53f8
feat(redact_messages.py): allow remove sensitive key information before passing to logging integration
2024-07-22 20:58:02 -07:00
Ishaan Jaff
0436eba2fa
otel - log to arize ai
2024-07-22 13:40:42 -07:00
Krrish Dholakia
fd743aaefd
feat(opentelemetry.py): support logging call metadata to otel
2024-07-12 15:41:34 -07:00
Ishaan Jaff
02ab3cb73d
test- otel span recording
2024-07-11 08:47:16 -07:00
Ishaan Jaff
c2631ce860
Merge pull request #4422 from BerriAI/litellm_otel_env_name
...
[Feat] OTEL allow setting deployment environment
2024-07-06 15:23:07 -07:00
Ishaan Jaff
c91b254e25
fix OTEL TracerProvider
2024-07-06 15:15:49 -07:00
Krrish Dholakia
7618ec43b3
fix(proxy_server.py): support langfuse logging for rejected requests on /v1/chat/completions
2024-07-05 13:07:09 -07:00
Ishaan Jaff
174d6eaf13
OTEL allow setting deployment environment
2024-06-26 11:50:16 -07:00
Krrish Dholakia
a028600932
feat(dynamic_rate_limiter.py): update cache with active project
2024-06-21 20:25:40 -07:00
Ishaan Jaff
253b504199
fix - otel fixes
2024-06-17 19:15:14 -07:00
Ishaan Jaff
427f2ee1fa
feat - otel log /team exceptions
2024-06-11 16:27:07 -07:00
Ishaan Jaff
13cb49688a
feat - log management endpoint logs to otel
2024-06-11 16:09:11 -07:00
Ishaan Jaff
7ce1289548
feat - use traceparent to proporgate OTEL traces
2024-06-11 13:57:31 -07:00
Ishaan Jaff
1af3d34d8b
fix OTEL semantic conventions does not exist
2024-06-11 12:28:38 -07:00
Ishaan Jaff
39bbc5d8ac
Merge pull request #4086 from BerriAI/litellm_sdk_tool_calling_fic
...
[Fix] Litellm sdk - allow ChatCompletionMessageToolCall, and Function to be used as dict
2024-06-08 20:48:54 -07:00
Ishaan Jaff
0d3add56a6
Merge pull request #4078 from BerriAI/litellm_low_raw_request_response_otel
...
[FEAT]- OTEL Log raw LLM request/response on OTEL
2024-06-08 19:49:28 -07:00
Ishaan Jaff
0253c2b213
feat - allow ChatCompletionMessageToolCall, and Function to be used as dict
2024-06-08 19:47:31 -07:00
Ishaan Jaff
5b73bcfe31
fix - otel set tools attribute
2024-06-08 19:40:29 -07:00
Ishaan Jaff
8b187eb66d
fix - show custom llm provider on OTEL span
2024-06-08 19:13:10 -07:00
Ishaan Jaff
090f350169
minor fix - _debug_otel
2024-06-08 18:16:35 -07:00
Ishaan Jaff
dcc63c7d67
fix otel don't log raw request / response when turn_off_message_logging
2024-06-08 14:37:06 -07:00
Ishaan Jaff
9478c98fff
otel - use correct raw request name
2024-06-08 14:22:44 -07:00
Ishaan Jaff
1ff99bf9b5
fix otel - handle vision images content
2024-06-08 14:12:00 -07:00
Ishaan Jaff
259540900b
feat - set span attributes OTEL with raw request / response
2024-06-08 14:01:59 -07:00
Ishaan Jaff
62411d2870
feat - low raw request on OTEL
2024-06-08 11:35:17 -07:00
Ishaan Jaff
17833b117d
feat - OTEL log litellm request / response
2024-06-08 09:58:52 -07:00
