Commit graph

70 commits

Author SHA1 Message Date
Krish Dholakia
98c335acd0
LiteLLM Minor Fixes & Improvements (09/17/2024) (#5742)
* fix(proxy_server.py): use default azure credentials to support azure non-client secret kms

* fix(langsmith.py): raise error if credentials missing

* feat(langsmith.py): support error logging for langsmith + standard logging payload

Fixes https://github.com/BerriAI/litellm/issues/5738

* Fix hardcoding of schema in view check (#5749)

* fix - deal with case when check view exists returns None (#5740)

* Revert "fix - deal with case when check view exists returns None (#5740)" (#5741)

This reverts commit 535228159b.

* test(test_router_debug_logs.py): move to mock response

* Fix hardcoding of schema

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>

* fix(proxy_server.py): allow admin to disable ui via `DISABLE_ADMIN_UI` flag

* fix(router.py): fix default model name value

Fixes 55db19a1e4 (r1763712148)

* fix(utils.py): fix unbound variable error

* feat(rerank/main.py): add azure ai rerank endpoints

Closes https://github.com/BerriAI/litellm/issues/5667

* feat(secret_detection.py): Allow configuring secret detection params

Allows admin to control what plugins to run for secret detection. Prevents overzealous secret detection.

* docs(secret_detection.md): add secret detection guardrail docs

* fix: fix linting errors

* fix - deal with case when check view exists returns None (#5740)

* Revert "fix - deal with case when check view exists returns None (#5740)" (#5741)

This reverts commit 535228159b.

* Litellm fix router testing (#5748)

* test: fix testing - azure changed content policy error logic

* test: fix tests to use mock responses

* test(test_image_generation.py): handle api instability

* test(test_image_generation.py): handle azure api instability

* fix(utils.py): fix unbounded variable error

* fix(utils.py): fix unbounded variable error

* test: refactor test to use mock response

* test: mark flaky azure tests

* Bump next from 14.1.1 to 14.2.10 in /ui/litellm-dashboard (#5753)

Bumps [next](https://github.com/vercel/next.js) from 14.1.1 to 14.2.10.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v14.1.1...v14.2.10)

---
updated-dependencies:
- dependency-name: next
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [Fix] o1-mini causes pydantic warnings on `reasoning_tokens`  (#5754)

* add requester_metadata in standard logging payload

* log requester_metadata in metadata

* use StandardLoggingPayload for logging

* docs StandardLoggingPayload

* fix import

* include standard logging object in failure

* add test for requester metadata

* handle completion_tokens_details

* add test for completion_tokens_details

* [Feat-Proxy-DataDog] Log Redis, Postgres Failure events on DataDog  (#5750)

* dd - start tracking redis status on dd

* add async_service_succes_hook / failure hook in custom logger

* add async_service_failure_hook

* log service failures on dd

* fix import error

* add test for redis errors / warning

* [Fix] Router/ Proxy - Tag Based routing, raise correct error when no deployments found and tag filtering is on  (#5745)

* fix tag routing - raise correct error when no model with tag based routing

* fix error string from tag based routing

* test router tag based routing

* raise 401 error when no tags avialable for deploymen

* linting fix

* [Feat] Log Request metadata on gcs bucket logging (#5743)

* add requester_metadata in standard logging payload

* log requester_metadata in metadata

* use StandardLoggingPayload for logging

* docs StandardLoggingPayload

* fix import

* include standard logging object in failure

* add test for requester metadata

* fix(litellm_logging.py): fix logging message

* fix(rerank_api/main.py): fix linting errors

* fix(custom_guardrails.py): maintain backwards compatibility for older guardrails

* fix(rerank_api/main.py): fix cost tracking for rerank endpoints

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: steffen-sbt <148480574+steffen-sbt@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-17 23:00:04 -07:00
Ishaan Jaff
2efdd2a6a4 mark test as flaky 2024-09-14 19:32:22 -07:00
Ishaan Jaff
9a3873b9ed mark flaky test as flaky 2024-08-31 15:02:56 -07:00
Krish Dholakia
dd7b008161
fix: Minor LiteLLM Fixes + Improvements (29/08/2024) (#5436)
* fix(model_checks.py): support returning wildcard models on `/v1/models`

Fixes https://github.com/BerriAI/litellm/issues/4903

* fix(bedrock_httpx.py): support calling bedrock via api_base

Closes https://github.com/BerriAI/litellm/pull/4587

* fix(litellm_logging.py): only leave last 4 char of gemini key unmasked

Fixes https://github.com/BerriAI/litellm/issues/5433

* feat(router.py): support setting 'weight' param for models on router

Closes https://github.com/BerriAI/litellm/issues/5410

* test(test_bedrock_completion.py): add unit test for custom api base

* fix(model_checks.py): handle no "/" in model
2024-08-29 22:40:25 -07:00
Krish Dholakia
8ce1e49fbe
fix(utils.py): correctly log streaming cache hits (#5417) (#5426)
Fixes https://github.com/BerriAI/litellm/issues/5401
2024-08-28 22:50:33 -07:00
Krrish Dholakia
67eb58d9f8 test(test_custom_callback_input.py): skip flaky ci/cd test 2024-08-22 09:19:10 -07:00
Krrish Dholakia
cf4574dd65 test(test_custom_callback_input.py): fix test 2024-08-22 08:54:59 -07:00
Krrish Dholakia
8a05ce77e9 feat(litellm_logging.py): add 'saved_cache_cost' to standard logging payload (s3) 2024-08-21 16:58:07 -07:00
Krrish Dholakia
0ce476a7f6 test(test_custom_callback.py): add test for message redaction to standard logging object 2024-08-19 17:06:41 -07:00
Krrish Dholakia
a92dcdd2d6 fix(litellm_logging.py): fix price information logging to s3 2024-08-16 16:42:38 -07:00
Krrish Dholakia
178139f18d feat(litellm_logging.py): support logging model price information to s3 logs 2024-08-16 16:21:34 -07:00
Krrish Dholakia
f6dba82882 feat(litellm_logging.py): cleanup payload + add response cost to logged payload 2024-08-15 17:53:25 -07:00
Krrish Dholakia
cda50e5d47 fix(s3.py): fix s3 logging payload to have valid json values
Previously pydantic objects were being stringified, making them unparsable
2024-08-15 17:09:02 -07:00
Krrish Dholakia
a26b23a3f4 fix(litellm_logging.py): fix calling success callback w/ stream_options true
Fixes https://github.com/BerriAI/litellm/issues/5118
2024-08-09 18:20:42 -07:00
Krish Dholakia
e6963217ba
Merge branch 'main' into litellm_parallel_requests 2024-07-24 19:25:56 -07:00
Krrish Dholakia
9d10881f3d test: re-run ci/cd 2024-07-23 07:55:42 -07:00
Krrish Dholakia
1a83935aa4 fix(proxy/utils.py): add stronger typing for litellm params in failure call logging 2024-07-22 21:31:39 -07:00
Krrish Dholakia
a4b0740c4a fix(test_custom_callback_input.py): fix unit test for embedding logging 2024-07-15 21:47:05 -07:00
Krrish Dholakia
d136f2b8a7 fix(litellm_logging.py): fix circular reference 2024-07-15 21:28:33 -07:00
Ishaan Jaff
4aa98dcbe9 test_amazing_sync_embedding 2024-07-15 20:53:13 -07:00
Ishaan Jaff
4baa48ba4b fix test_sync_embedding 2024-07-15 20:51:29 -07:00
Krrish Dholakia
f5b3cc6c02 fix(litellm_logging.py): fix condition check
Fixes https://github.com/BerriAI/litellm/issues/4633
2024-07-12 09:22:19 -07:00
Krrish Dholakia
beddb58852 test(test_custom_callback_input.py): fix custom callback input test 2024-06-07 16:13:30 -07:00
Krrish Dholakia
f73b6033fd fix(test_custom_callbacks_input.py): unit tests for 'turn_off_message_logging'
ensure no raw request is logged either
2024-06-07 15:39:15 -07:00
Krrish Dholakia
2b3da449c8 feat(lowest_latency.py): route by time to first token, for streaming requests (if available)
Closes https://github.com/BerriAI/litellm/issues/3574
2024-05-21 13:08:17 -07:00
Krrish Dholakia
21f2ba6f1f fix(bedrock_httpx.py): logging fixes 2024-05-16 23:20:51 -07:00
Krrish Dholakia
29aefaa7d7 fix(utils.py): fix default values 2024-05-01 22:24:37 -07:00
Krrish Dholakia
6234982fd9 test(test_custom_callback_input.py): fix assert 2024-04-06 18:48:33 -07:00
Krrish Dholakia
6110d32b1c feat(proxy/utils.py): return api base for request hanging alerts 2024-04-06 15:58:53 -07:00
Ishaan Jaff
1ba1c84d83 skip vertex test for this PR 2024-04-05 09:06:15 -07:00
Krrish Dholakia
0d949d71ab fix(main.py): support text completion input being a list of strings
addresses - https://github.com/BerriAI/litellm/issues/2792, https://github.com/BerriAI/litellm/issues/2777
2024-04-02 08:50:16 -07:00
Krrish Dholakia
364526d0bc test: fix test 2024-03-30 20:22:48 -07:00
Krrish Dholakia
bd75498913 fix(utils.py): log success event for streaming 2024-03-25 19:03:10 -07:00
Krrish Dholakia
8d2d51b625 fix(utils.py): fix model name checking 2024-03-09 18:22:26 -08:00
Krrish Dholakia
478307d4cf fix(bedrock.py): support anthropic messages api on bedrock (claude-3) 2024-03-04 17:15:47 -08:00
Krrish Dholakia
4c951d20bc test: removing aws tests - account suspended - pending their approval 2024-02-28 13:46:20 -08:00
Krrish Dholakia
a042092faa test: removing bedrock claude-v1 testing - bedrock removed this 2024-02-28 11:08:17 -08:00
Krrish Dholakia
e48fff47dd test(test_custom_callback_input.py): assert async success called only once during vertex ai streaming 2024-02-26 09:18:46 -08:00
Krrish Dholakia
2d62dee712 fix(utils.py): enable streaming cache logging 2024-02-21 21:10:58 -08:00
Krrish Dholakia
d2d9e63176 test(test_custom_callback_input.py): fix test 2024-02-21 20:32:39 -08:00
Krrish Dholakia
f8b233b653 fix(utils.py): support streaming cached response logging 2024-02-21 17:53:14 -08:00
Krrish Dholakia
9cbdbda8e4 test(test_custom_callback_input.py): fix image gen callback test 2024-02-16 16:29:26 -08:00
Krrish Dholakia
f57483ea70 fix(utils.py): support image gen logging to langfuse 2024-02-16 16:12:52 -08:00
Krish Dholakia
9ab59045a3
Merge branch 'main' into litellm_embedding_caching_updates 2024-02-03 18:08:47 -08:00
Krrish Dholakia
618d216c11 fix(openai.py): fix image gen logging 2024-01-26 21:04:06 -08:00
Krrish Dholakia
a299ac2328 fix(utils.py): enable cost tracking for image gen models on proxy 2024-01-26 20:51:13 -08:00
Krrish Dholakia
09ec6d6458 fix(utils.py): fix sagemaker async logging for sync streaming
https://github.com/BerriAI/litellm/issues/1592
2024-01-25 12:49:45 -08:00
Krrish Dholakia
42a2a2fe94 test(test_custom_callback_input.py): fix assert 2024-01-22 20:39:14 -08:00
Krrish Dholakia
3e8c8ef507 fix(openai.py): fix linting issue 2024-01-22 18:20:15 -08:00
Krrish Dholakia
e917d0eee6 feat(utils.py): emit response cost as part of logs 2024-01-22 15:53:04 -08:00