phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-26 19:24:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
18230 commits 2490 branches 894 tags 548 MiB
Commit graph

73 commits

Author SHA1 Message Date
Krish Dholakia
e7ce45236a Litellm perf improvements 3 (#6573) 
* perf: move writing key to cache, to background task

* perf(litellm_pre_call_utils.py): add otel tracing for pre-call utils

adds 200ms on calls with pgdb connected

* fix(litellm_pre_call_utils.py'): rename call_type to actual call used

* perf(proxy_server.py): remove db logic from _get_config_from_file

was causing db calls to occur on every llm request, if team_id was set on key

* fix(auth_checks.py): add check for reducing db calls if user/team id does not exist in db

reduces latency/call by ~100ms

* fix(proxy_server.py): minor fix on existing_settings not incl alerting

* fix(exception_mapping_utils.py): map databricks exception string

* fix(auth_checks.py): fix auth check logic

* test: correctly mark flaky test

* fix(utils.py): handle auth token error for tokenizers.from_pretrained
 2024-11-05 03:51:26 +05:30
Krish Dholakia
976926e231 LiteLLM Minor Fixes & Improvements (11/01/2024) (#6551) 
* fix: add lm_studio support

* fix(cohere_transformation.py): fix transformation logic for azure cohere embedding model name

Fixes https://github.com/BerriAI/litellm/issues/6540

* fix(utils.py): require base64 str to begin with `data:`

Fixes https://github.com/BerriAI/litellm/issues/6541

* fix: cleanup tests

* docs(guardrails.md): fix typo

* fix(opentelemetry.py): move to `.exception` and update 'response_obj' value to handle 'None' case

Fixes https://github.com/BerriAI/litellm/issues/6510

* fix: fix linting noqa placement
 2024-11-02 02:09:31 +05:30
Krish Dholakia
e712a2090b redis otel tracing + async support for latency routing (#6452) 
* docs(exception_mapping.md): add missing exception types

Fixes https://github.com/Aider-AI/aider/issues/2120#issuecomment-2438971183

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* refactor: pass parent_otel_span for redis caching calls in router

allows for more observability into what calls are causing latency issues

* test: update tests with new params

* refactor: ensure e2e otel tracing for router

* refactor(router.py): add more otel tracing acrosss router

catch all latency issues for router requests

* fix: fix linting error

* fix(router.py): fix linting error

* fix: fix test

* test: fix tests

* fix(dual_cache.py): pass ttl to redis cache

* fix: fix param
 2024-10-28 21:52:12 -07:00
Krish Dholakia
da292e48b2 perf: remove 'always_read_redis' - adding +830ms on each llm call (#6414) 
* perf: remove 'always_read_redis' - adding +830ms on each llm call

* test: cleanup codestral tests - backend api unavailable
 2024-10-24 17:48:36 -07:00
Ishaan Jaff
300ac43025 (feat) Arize - Allow using Arize HTTP endpoint (#6364) 
* arize use helper for get_arize_opentelemetry_config

* use helper to get Arize OTEL config

* arize add helpers for arize

* docs allow using arize http endpoint

* fix importing OTEL for Arize

* use static methods for ArizeLogger

* fix ArizeLogger tests
 2024-10-23 09:38:35 +05:30
Ishaan Jaff
0c5a47c404 (code quality) add ruff check PLR0915 for too-many-statements (#6309) 
* ruff add PLR0915

* add noqa for PLR0915

* fix noqa

* add # noqa: PLR0915

* # noqa: PLR0915

* # noqa: PLR0915

* # noqa: PLR0915

* add # noqa: PLR0915

* # noqa: PLR0915

* # noqa: PLR0915

* # noqa: PLR0915

* # noqa: PLR0915
 2024-10-18 15:36:49 +05:30
Ishaan Jaff
12054ad8c6 (testing) add test coverage for LLM OTEL logging (#6227) 
* add test coverage for OTEL logging

* test_async_otel_callback

* test test_awesome_otel_with_message_logging_off

* fix otel testing

* add otel testing

* otel testing

* otel testing

* otel testing coverage

* otel add testing
 2024-10-17 16:34:04 +05:30
Ishaan Jaff
700a87204a (refactor) OTEL - use safe_set_attribute for setting attributes (#6226) 
* otel - use safe_set_attribute for setting attributes

* fix OTEL only use safe_set_attribute
 2024-10-15 13:39:29 +05:30
Ali Waleed
f3a24d22d5 Feat: Add Langtrace integration (#5341) 
* Feat: Add Langtrace integration

* add langtrace service name

* fix timestamps for traces

* add tests

* Discard Callback + use existing otel logger

* cleanup

* remove print statments

* remove callback

* add docs

* docs

* add logging docs

* format logging

* remove emoji and add litellm proxy example

* format logging

* format `logging.md`

* add langtrace docs to logging.md

* sync conflict
 2024-10-11 19:19:53 +05:30
Krish Dholakia
94a05ca5d0 Litellm ruff linting enforcement (#5992) 
* ci(config.yml): add a 'check_code_quality' step

Addresses https://github.com/BerriAI/litellm/issues/5991

* ci(config.yml): check why circle ci doesn't pick up this test

* ci(config.yml): fix to run 'check_code_quality' tests

* fix(__init__.py): fix unprotected import

* fix(__init__.py): don't remove unused imports

* build(ruff.toml): update ruff.toml to ignore unused imports

* fix: fix: ruff + pyright - fix linting + type-checking errors

* fix: fix linting errors

* fix(lago.py): fix module init error

* fix: fix linting errors

* ci(config.yml): cd into correct dir for checks

* fix(proxy_server.py): fix linting error

* fix(utils.py): fix bare except

causes ruff linting errors

* fix: ruff - fix remaining linting errors

* fix(clickhouse.py): use standard logging object

* fix(__init__.py): fix unprotected import

* fix: ruff - fix linting errors

* fix: fix linting errors

* ci(config.yml): cleanup code qa step (formatting handled in local_testing)

* fix(_health_endpoints.py): fix ruff linting errors

* ci(config.yml): just use ruff in check_code_quality pipeline for now

* build(custom_guardrail.py): include missing file

* style(embedding_handler.py): fix ruff check
 2024-10-01 19:44:20 -04:00
Ishaan Jaff
276a141719 chore - handle case when otel metadata field value is None 2024-09-24 22:24:12 -07:00
Ishaan Jaff
26d8097779 [Fix] OTEL - Don't log messages when callback settings disable message logging (#5875) 
* fix otel dont log messages

* otel fix redis failure hook logging
 2024-09-24 18:29:52 -07:00
Ishaan Jaff
fd6cc10922 [Feat] Add proxy level prometheus metrics (#5789) 
* add Proxy Level Tracking Metrics doc

* update service logger

* prometheus - track litellm_proxy_failed_requests_metric

* use REQUESTED_MODEL

* fix prom request_data
 2024-09-19 17:13:07 -07:00
Ishaan Jaff
c72c8c0383 fix otel use sensible defaults 2024-09-11 14:24:04 -07:00
Ishaan Jaff
02325f33d7 Merge branch 'main' into litellm_allow_turning_off_message_logging_for_callbacks 2024-09-09 21:59:36 -07:00
Ishaan Jaff
2e021a6203 fix otel defaults 2024-09-09 16:18:55 -07:00
Ishaan Jaff
0f0a5c1758 fix init custom logger when init OTEL runs 2024-09-09 16:03:39 -07:00
Ishaan Jaff
22fe7705e7 fix otel set max_queue_size, max_queue_size 2024-09-06 17:31:43 -07:00
Ishaan Jaff
5c7fbe0be2 fix otel max batch size 2024-09-06 17:12:01 -07:00
Ishaan Jaff
4bc0d297f7 fix otel type 2024-09-05 19:04:56 -07:00
Ishaan Jaff
8da555a2d9 otel log service errors 2024-08-05 20:21:26 -07:00
Ishaan Jaff
774807d2c4 otel log event_metadata 2024-08-05 20:01:58 -07:00
Ishaan Jaff
291df6b41d fix _handle_failure for otel 2024-08-01 11:12:31 -07:00
Ishaan Jaff
aade38760d use common helpers for writing to otel 2024-07-27 11:40:39 -07:00
Ishaan Jaff
40f9e67be4 move _get_parent_otel_span_from_kwargs to otel.py 2024-07-27 11:12:13 -07:00
Krrish Dholakia
1562cba823 fix(utils.py): fix cache hits for streaming 
Fixes https://github.com/BerriAI/litellm/issues/4109
 2024-07-26 19:04:08 -07:00
Krrish Dholakia
8c005d8134 feat(redact_messages.py): allow remove sensitive key information before passing to logging integration 2024-07-22 20:58:02 -07:00
Ishaan Jaff
464a4edfb2 otel - log to arize ai 2024-07-22 13:40:42 -07:00
Krrish Dholakia
d2a0977af7 feat(opentelemetry.py): support logging call metadata to otel 2024-07-12 15:41:34 -07:00
Ishaan Jaff
97da1252f1 test- otel span recording 2024-07-11 08:47:16 -07:00
Ishaan Jaff
35d4adb147 Merge pull request #4422 from BerriAI/litellm_otel_env_name 
[Feat] OTEL allow setting deployment environment
 2024-07-06 15:23:07 -07:00
Ishaan Jaff
9738aaebf6 fix OTEL TracerProvider 2024-07-06 15:15:49 -07:00
Krrish Dholakia
56410cfcd0 fix(proxy_server.py): support langfuse logging for rejected requests on /v1/chat/completions 2024-07-05 13:07:09 -07:00
Ishaan Jaff
4f409e647f OTEL allow setting deployment environment 2024-06-26 11:50:16 -07:00
Krrish Dholakia
0430807178 feat(dynamic_rate_limiter.py): update cache with active project 2024-06-21 20:25:40 -07:00
Ishaan Jaff
b343911d59 fix - otel fixes 2024-06-17 19:15:14 -07:00
Ishaan Jaff
4859244e67 feat - otel log /team exceptions 2024-06-11 16:27:07 -07:00
Ishaan Jaff
9d78e1bd5f feat - log management endpoint logs to otel 2024-06-11 16:09:11 -07:00
Ishaan Jaff
2d5904c9a5 feat - use traceparent to proporgate OTEL traces 2024-06-11 13:57:31 -07:00
Ishaan Jaff
6c19b76edc fix OTEL semantic conventions does not exist 2024-06-11 12:28:38 -07:00
Ishaan Jaff
ef9349e6a2 Merge pull request #4086 from BerriAI/litellm_sdk_tool_calling_fic 
[Fix] Litellm sdk - allow ChatCompletionMessageToolCall, and Function to be used as dict
 2024-06-08 20:48:54 -07:00
Ishaan Jaff
8c5802d506 Merge pull request #4078 from BerriAI/litellm_low_raw_request_response_otel 
[FEAT]- OTEL Log raw LLM request/response on OTEL
 2024-06-08 19:49:28 -07:00
Ishaan Jaff
af61eff8e3 feat - allow ChatCompletionMessageToolCall, and Function to be used as dict 2024-06-08 19:47:31 -07:00
Ishaan Jaff
21ac0efaae fix - otel set tools attribute 2024-06-08 19:40:29 -07:00
Ishaan Jaff
638c4acbb2 fix - show custom llm provider on OTEL span 2024-06-08 19:13:10 -07:00
Ishaan Jaff
0d83ffb4e2 minor fix - _debug_otel 2024-06-08 18:16:35 -07:00
Ishaan Jaff
e7f6c5d651 fix otel don't log raw request / response when turn_off_message_logging 2024-06-08 14:37:06 -07:00
Ishaan Jaff
22fa537d98 otel - use correct raw request name 2024-06-08 14:22:44 -07:00
Ishaan Jaff
4d7ff7c79b fix otel - handle vision images content 2024-06-08 14:12:00 -07:00
Ishaan Jaff
7f86dc859e feat - set span attributes OTEL with raw request / response 2024-06-08 14:01:59 -07:00