Commit graph

74 commits

Author SHA1 Message Date
Ishaan Jaff
0c5a47c404 (code quality) add ruff check PLR0915 for too-many-statements (#6309)
* ruff add PLR0915

* add noqa for PLR0915

* fix noqa

* add # noqa: PLR0915

* # noqa: PLR0915

* # noqa: PLR0915

* # noqa: PLR0915

* add # noqa: PLR0915

* # noqa: PLR0915

* # noqa: PLR0915

* # noqa: PLR0915

* # noqa: PLR0915
2024-10-18 15:36:49 +05:30
Krish Dholakia
17fa7c17ec LiteLLM Minor Fixes & Improvements (10/10/2024) (#6158)
* refactor(vertex_ai_partner_models/anthropic): refactor anthropic to use partner model logic

* fix(vertex_ai/): support passing custom api base to partner models

Fixes https://github.com/BerriAI/litellm/issues/4317

* fix(proxy_server.py): Fix prometheus premium user check logic

* docs(prometheus.md): update quick start docs

* fix(custom_llm.py): support passing dynamic api key + api base

* fix(realtime_api/main.py): Add request/response logging for realtime api endpoints

Closes https://github.com/BerriAI/litellm/issues/6081

* feat(openai/realtime): add openai realtime api logging

Closes https://github.com/BerriAI/litellm/issues/6081

* fix(realtime_streaming.py): fix linting errors

* fix(realtime_streaming.py): fix linting errors

* fix: fix linting errors

* fix pattern match router

* Add literalai in the sidebar observability category (#6163)

* fix: add literalai in the sidebar

* fix: typo

* update (#6160)

* Feat: Add Langtrace integration (#5341)

* Feat: Add Langtrace integration

* add langtrace service name

* fix timestamps for traces

* add tests

* Discard Callback + use existing otel logger

* cleanup

* remove print statments

* remove callback

* add docs

* docs

* add logging docs

* format logging

* remove emoji and add litellm proxy example

* format logging

* format `logging.md`

* add langtrace docs to logging.md

* sync conflict

* docs fix

* (perf) move s3 logging to Batch logging + async [94% faster perf under 100 RPS on 1 litellm instance] (#6165)

* fix move s3 to use customLogger

* add basic s3 logging test

* add s3 to custom logger compatible

* use batch logger for s3

* s3 set flush interval and batch size

* fix s3 logging

* add notes on s3 logging

* fix s3 logging

* add basic s3 logging test

* fix s3 type errors

* add test for sync logging on s3

* fix: fix to debug log

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Willy Douhard <willy.douhard@gmail.com>
Co-authored-by: yujonglee <yujonglee.dev@gmail.com>
Co-authored-by: Ali Waleed <ali@scale3labs.com>
2024-10-11 23:04:36 -07:00
Ishaan Jaff
dfe9c97e1b (code clean up) use a folder for gcs bucket logging + add readme in folder (#6080)
* refactor gcs bucket

* add readme
2024-10-05 16:58:10 +05:30
Krish Dholakia
94a05ca5d0 Litellm ruff linting enforcement (#5992)
* ci(config.yml): add a 'check_code_quality' step

Addresses https://github.com/BerriAI/litellm/issues/5991

* ci(config.yml): check why circle ci doesn't pick up this test

* ci(config.yml): fix to run 'check_code_quality' tests

* fix(__init__.py): fix unprotected import

* fix(__init__.py): don't remove unused imports

* build(ruff.toml): update ruff.toml to ignore unused imports

* fix: fix: ruff + pyright - fix linting + type-checking errors

* fix: fix linting errors

* fix(lago.py): fix module init error

* fix: fix linting errors

* ci(config.yml): cd into correct dir for checks

* fix(proxy_server.py): fix linting error

* fix(utils.py): fix bare except

causes ruff linting errors

* fix: ruff - fix remaining linting errors

* fix(clickhouse.py): use standard logging object

* fix(__init__.py): fix unprotected import

* fix: ruff - fix linting errors

* fix: fix linting errors

* ci(config.yml): cleanup code qa step (formatting handled in local_testing)

* fix(_health_endpoints.py): fix ruff linting errors

* ci(config.yml): just use ruff in check_code_quality pipeline for now

* build(custom_guardrail.py): include missing file

* style(embedding_handler.py): fix ruff check
2024-10-01 19:44:20 -04:00
Krish Dholakia
dc060857a7 LiteLLM Minor Fixes & Improvements (09/17/2024) (#5742)
* fix(proxy_server.py): use default azure credentials to support azure non-client secret kms

* fix(langsmith.py): raise error if credentials missing

* feat(langsmith.py): support error logging for langsmith + standard logging payload

Fixes https://github.com/BerriAI/litellm/issues/5738

* Fix hardcoding of schema in view check (#5749)

* fix - deal with case when check view exists returns None (#5740)

* Revert "fix - deal with case when check view exists returns None (#5740)" (#5741)

This reverts commit 535228159b.

* test(test_router_debug_logs.py): move to mock response

* Fix hardcoding of schema

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>

* fix(proxy_server.py): allow admin to disable ui via `DISABLE_ADMIN_UI` flag

* fix(router.py): fix default model name value

Fixes 55db19a1e4 (r1763712148)

* fix(utils.py): fix unbound variable error

* feat(rerank/main.py): add azure ai rerank endpoints

Closes https://github.com/BerriAI/litellm/issues/5667

* feat(secret_detection.py): Allow configuring secret detection params

Allows admin to control what plugins to run for secret detection. Prevents overzealous secret detection.

* docs(secret_detection.md): add secret detection guardrail docs

* fix: fix linting errors

* fix - deal with case when check view exists returns None (#5740)

* Revert "fix - deal with case when check view exists returns None (#5740)" (#5741)

This reverts commit 535228159b.

* Litellm fix router testing (#5748)

* test: fix testing - azure changed content policy error logic

* test: fix tests to use mock responses

* test(test_image_generation.py): handle api instability

* test(test_image_generation.py): handle azure api instability

* fix(utils.py): fix unbounded variable error

* fix(utils.py): fix unbounded variable error

* test: refactor test to use mock response

* test: mark flaky azure tests

* Bump next from 14.1.1 to 14.2.10 in /ui/litellm-dashboard (#5753)

Bumps [next](https://github.com/vercel/next.js) from 14.1.1 to 14.2.10.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v14.1.1...v14.2.10)

---
updated-dependencies:
- dependency-name: next
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [Fix] o1-mini causes pydantic warnings on `reasoning_tokens`  (#5754)

* add requester_metadata in standard logging payload

* log requester_metadata in metadata

* use StandardLoggingPayload for logging

* docs StandardLoggingPayload

* fix import

* include standard logging object in failure

* add test for requester metadata

* handle completion_tokens_details

* add test for completion_tokens_details

* [Feat-Proxy-DataDog] Log Redis, Postgres Failure events on DataDog  (#5750)

* dd - start tracking redis status on dd

* add async_service_succes_hook / failure hook in custom logger

* add async_service_failure_hook

* log service failures on dd

* fix import error

* add test for redis errors / warning

* [Fix] Router/ Proxy - Tag Based routing, raise correct error when no deployments found and tag filtering is on  (#5745)

* fix tag routing - raise correct error when no model with tag based routing

* fix error string from tag based routing

* test router tag based routing

* raise 401 error when no tags avialable for deploymen

* linting fix

* [Feat] Log Request metadata on gcs bucket logging (#5743)

* add requester_metadata in standard logging payload

* log requester_metadata in metadata

* use StandardLoggingPayload for logging

* docs StandardLoggingPayload

* fix import

* include standard logging object in failure

* add test for requester metadata

* fix(litellm_logging.py): fix logging message

* fix(rerank_api/main.py): fix linting errors

* fix(custom_guardrails.py): maintain backwards compatibility for older guardrails

* fix(rerank_api/main.py): fix cost tracking for rerank endpoints

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: steffen-sbt <148480574+steffen-sbt@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-17 23:00:04 -07:00
Ishaan Jaff
7e8af27527 fix otel test 2024-09-09 16:20:47 -07:00
Ishaan Jaff
e07b2ce6ea use callback_settings when intializing otel 2024-09-09 16:05:48 -07:00
Ishaan Jaff
3369b4e41a support using "callbacks" for prometheus 2024-09-09 08:26:03 -07:00
Ishaan Jaff
c60125d7be add gcs bucket base 2024-08-30 10:41:39 -07:00
Ishaan Jaff
ce1a0a93b5 use helper to get_config_file_contents_from_gcs 2024-08-30 10:26:42 -07:00
Ishaan Jaff
378182cba2 show all error types on swagger 2024-08-29 18:50:41 -07:00
Ishaan Jaff
b522ade709 docs semantic caching qdrant 2024-08-21 13:03:41 -07:00
Ishaan Jaff
cdbd245c3d working lakera ai during call hook 2024-08-20 14:39:04 -07:00
Ishaan Jaff
4665487558 rename lakera ai 2024-08-20 13:44:39 -07:00
Ishaan Jaff
7bffe2a0fa rename Aporia Guardrail 2024-08-20 13:42:55 -07:00
Ishaan Jaff
cf0d2baee8 fix importing _ENTERPRISE_Aporia 2024-08-19 20:39:24 -07:00
Ishaan Jaff
0920c5ece1 fix aporia typo 2024-08-19 18:34:45 -07:00
Ishaan Jaff
f16e0472c2 feat - return applied guardrails in response headers 2024-08-19 11:56:20 -07:00
Ishaan Jaff
ec671b491d fix proxy all models test 2024-08-17 15:54:51 -07:00
Ishaan Jaff
9f6630912d feat return rmng tokens for model for api key 2024-08-17 12:35:10 -07:00
Ishaan Jaff
a62277a6aa feat - use commong helper for getting model group 2024-08-17 10:46:04 -07:00
Ishaan Jaff
e79a81522e fix ImportError 2024-08-15 13:02:44 -07:00
Ishaan Jaff
4a89a30f12 fix - don't require boto3 on the cli 2024-08-15 12:36:38 -07:00
Ishaan Jaff
11345c4689 fix use s3 get_credentials to get boto3 creds 2024-08-13 21:18:06 -07:00
Ishaan Jaff
702f6bca00 comment on using boto3 2024-08-13 20:28:40 -07:00
Ishaan Jaff
b39cd3b9b4 add helper to load config from s3 2024-08-13 20:16:54 -07:00
Krish Dholakia
0044fd0041 Merge branch 'main' into litellm_support_lakera_config_thresholds 2024-08-06 22:47:13 -07:00
Ishaan Jaff
7bcb088ac0 clean up unused func 2024-08-06 20:58:51 -07:00
Ishaan Jaff
53451834bd Merge pull request #5077 from BerriAI/litellm_fix_ui_emails
[Fix] Fix testing emails through Admin UI
2024-08-06 16:41:01 -07:00
Ishaan Jaff
6a46899521 add debug statements on docker 2024-08-06 15:29:49 -07:00
Krrish Dholakia
ff28823025 fix(encrypt_decrypt_utils.py): add helper line, explaining why there might be a key decryption error 2024-08-06 11:35:40 -07:00
Ishaan Jaff
d3d13e40be use prints 2024-08-06 11:07:12 -07:00
Ishaan Jaff
51f5c2d08f use safe init_verbose_loggers 2024-08-05 20:56:48 -07:00
Ishaan Jaff
8ad43780cf add debugging utils to print when connecting to prisma 2024-08-05 19:19:28 -07:00
Ishaan Jaff
357ca893f8 build ui on custom path 2024-08-05 16:34:37 -07:00
Ishaan Jaff
cc0e790863 log file_size_in_mb in metadata 2024-07-29 08:00:28 -07:00
Krish Dholakia
ce022ca4dd Merge branch 'main' into litellm_parallel_requests 2024-07-24 19:25:56 -07:00
Krrish Dholakia
34a8875e8e fix(init_callbacks.py): fix presidio optional param 2024-07-22 22:31:17 -07:00
Krrish Dholakia
0c8196b3c7 feat(lakera_ai.py): control running prompt injection between pre-call and in_parallel 2024-07-22 20:04:42 -07:00
Ishaan Jaff
3bbb4e8f1d fix checking if _known_custom_logger_compatible_callbacks 2024-07-22 15:43:43 -07:00
Krrish Dholakia
af2055c2b7 feat(aporio_ai.py): support aporio ai prompt injection for chat completion requests
Closes https://github.com/BerriAI/litellm/issues/2950
2024-07-17 16:38:47 -07:00
Ishaan Jaff
b954ffe9bf fix set default value for max_file_size_mb 2024-07-16 20:34:56 -07:00
Ishaan Jaff
470642c92c fix linting 2024-07-15 20:02:41 -07:00
Ishaan Jaff
e2096badbc fix error from max file size 2024-07-15 19:57:33 -07:00
Ishaan Jaff
df6995f067 check_file_size_under_limit 2024-07-15 19:38:08 -07:00
Ishaan Jaff
a9f77c2af3 fix show debugging utils on in mem usage 2024-07-15 10:05:57 -07:00
Krish Dholakia
808653ced6 Merge pull request #4696 from BerriAI/litellm_guardrail_logging_only
Allow setting `logging_only` in guardrails config
2024-07-13 21:50:43 -07:00
Ishaan Jaff
e3e649ea55 Merge pull request #4704 from BerriAI/litellm_debug_mem
[Debug-Utils] Add some useful memory usage debugging utils
2024-07-13 18:44:40 -07:00
Ishaan Jaff
668070a2bb feat - return size of in memory cache 2024-07-13 18:22:44 -07:00
Ishaan Jaff
4c7cc2208f debug mem issues show growth 2024-07-13 18:05:19 -07:00