Commit graph

333 commits

Author SHA1 Message Date
Ishaan Jaff
2449d258cf
(docs) add 1k rps load test doc (#6059)
* docs 1k rps load test

* docs load testing

* docs load testing litellm

* docs load testing

* clean up load test doc

* docs prom metrics for load testing

* docs using prometheus on load testing

* doc load testing with prometheus
2024-10-04 16:56:34 +05:30
Krrish Dholakia
793593e735 docs(realtime.md): add new /v1/realtime endpoint 2024-10-03 22:44:02 -04:00
Krrish Dholakia
121b493fe8 docs(code_quality.md): add doc on litellm code qa 2024-10-02 11:20:15 -04:00
Krrish Dholakia
7630680690 docs(response_headers.md): add response headers to docs 2024-09-28 23:33:50 -07:00
Ishaan Jaff
b817974c8e docs clean up langfuse.md 2024-09-28 18:59:02 -07:00
Ishaan Jaff
bbf4db79c1 docs - show correct rpm - > tpm conversion for Azure 2024-09-27 17:18:55 -07:00
Ishaan Jaff
4bdeefd7e4
docs service accounts (#5900) 2024-09-25 15:46:13 -07:00
Ishaan Jaff
4ec4d02474
[Feat-Router] Allow setting which environment to use a model on (#5892)
* add check deployment_is_active_for_environment

* add test for test_init_router_with_supported_environments

* show good example config for environments

* docs clean up config.yaml

* docs cleanup

* docs configs

* docs specfic env
2024-09-25 10:12:06 -07:00
Krrish Dholakia
16c8549b77 docs(virtual_keys.md): add enable/disable virtual keys to docs + refactor sidebar 2024-09-21 22:20:39 -07:00
Ishaan Jaff
cf7dcd9168
[Feat-Proxy] Allow using custom sso handler (#5809)
* update internal user doc string

* add readme on location of /sso routes

* add custom_sso_handler

* docs custom sso

* use secure=True for cookies
2024-09-20 19:14:33 -07:00
Krish Dholakia
98c335acd0
LiteLLM Minor Fixes & Improvements (09/17/2024) (#5742)
* fix(proxy_server.py): use default azure credentials to support azure non-client secret kms

* fix(langsmith.py): raise error if credentials missing

* feat(langsmith.py): support error logging for langsmith + standard logging payload

Fixes https://github.com/BerriAI/litellm/issues/5738

* Fix hardcoding of schema in view check (#5749)

* fix - deal with case when check view exists returns None (#5740)

* Revert "fix - deal with case when check view exists returns None (#5740)" (#5741)

This reverts commit 535228159b.

* test(test_router_debug_logs.py): move to mock response

* Fix hardcoding of schema

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>

* fix(proxy_server.py): allow admin to disable ui via `DISABLE_ADMIN_UI` flag

* fix(router.py): fix default model name value

Fixes 55db19a1e4 (r1763712148)

* fix(utils.py): fix unbound variable error

* feat(rerank/main.py): add azure ai rerank endpoints

Closes https://github.com/BerriAI/litellm/issues/5667

* feat(secret_detection.py): Allow configuring secret detection params

Allows admin to control what plugins to run for secret detection. Prevents overzealous secret detection.

* docs(secret_detection.md): add secret detection guardrail docs

* fix: fix linting errors

* fix - deal with case when check view exists returns None (#5740)

* Revert "fix - deal with case when check view exists returns None (#5740)" (#5741)

This reverts commit 535228159b.

* Litellm fix router testing (#5748)

* test: fix testing - azure changed content policy error logic

* test: fix tests to use mock responses

* test(test_image_generation.py): handle api instability

* test(test_image_generation.py): handle azure api instability

* fix(utils.py): fix unbounded variable error

* fix(utils.py): fix unbounded variable error

* test: refactor test to use mock response

* test: mark flaky azure tests

* Bump next from 14.1.1 to 14.2.10 in /ui/litellm-dashboard (#5753)

Bumps [next](https://github.com/vercel/next.js) from 14.1.1 to 14.2.10.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v14.1.1...v14.2.10)

---
updated-dependencies:
- dependency-name: next
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [Fix] o1-mini causes pydantic warnings on `reasoning_tokens`  (#5754)

* add requester_metadata in standard logging payload

* log requester_metadata in metadata

* use StandardLoggingPayload for logging

* docs StandardLoggingPayload

* fix import

* include standard logging object in failure

* add test for requester metadata

* handle completion_tokens_details

* add test for completion_tokens_details

* [Feat-Proxy-DataDog] Log Redis, Postgres Failure events on DataDog  (#5750)

* dd - start tracking redis status on dd

* add async_service_succes_hook / failure hook in custom logger

* add async_service_failure_hook

* log service failures on dd

* fix import error

* add test for redis errors / warning

* [Fix] Router/ Proxy - Tag Based routing, raise correct error when no deployments found and tag filtering is on  (#5745)

* fix tag routing - raise correct error when no model with tag based routing

* fix error string from tag based routing

* test router tag based routing

* raise 401 error when no tags avialable for deploymen

* linting fix

* [Feat] Log Request metadata on gcs bucket logging (#5743)

* add requester_metadata in standard logging payload

* log requester_metadata in metadata

* use StandardLoggingPayload for logging

* docs StandardLoggingPayload

* fix import

* include standard logging object in failure

* add test for requester metadata

* fix(litellm_logging.py): fix logging message

* fix(rerank_api/main.py): fix linting errors

* fix(custom_guardrails.py): maintain backwards compatibility for older guardrails

* fix(rerank_api/main.py): fix cost tracking for rerank endpoints

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: steffen-sbt <148480574+steffen-sbt@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-17 23:00:04 -07:00
Krish Dholakia
234185ec13
LiteLLM Minor Fixes & Improvements (09/16/2024) (#5723) (#5731)
* LiteLLM Minor Fixes & Improvements (09/16/2024)  (#5723)

* coverage (#5713)

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Move (#5714)

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* fix(litellm_logging.py): fix logging client re-init (#5710)

Fixes https://github.com/BerriAI/litellm/issues/5695

* fix(presidio.py): Fix logging_hook response and add support for additional presidio variables in guardrails config

Fixes https://github.com/BerriAI/litellm/issues/5682

* feat(o1_handler.py): fake streaming for openai o1 models

Fixes https://github.com/BerriAI/litellm/issues/5694

* docs: deprecated traceloop integration in favor of native otel (#5249)

* fix: fix linting errors

* fix: fix linting errors

* fix(main.py): fix o1 import

---------

Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>

* feat(spend_management_endpoints.py): expose `/global/spend/refresh` endpoint for updating material view (#5730)

* feat(spend_management_endpoints.py): expose `/global/spend/refresh` endpoint for updating material view

Supports having `MonthlyGlobalSpend` view be a material view, and exposes an endpoint to refresh it

* fix(custom_logger.py): reset calltype

* fix: fix linting errors

* fix: fix linting error

* fix: fix import

* test(test_databricks.py): fix databricks tests

---------

Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>
2024-09-17 08:05:52 -07:00
Ishaan Jaff
7c2ddba6c6
sambanova support (#5547) (#5703)
* add sambanova support

* sambanova support

* updated api endpoint for sambanova

---------

Co-authored-by: Venu Anuganti <venu@venublog.com>
Co-authored-by: Venu Anuganti <venu@vairmac2020>
2024-09-14 17:23:04 -07:00
Ishaan Jaff
54db564529 add arch diagram 2024-09-07 15:49:51 -07:00
Ishaan Jaff
05505903b2 docs better sidebar 2024-09-07 11:31:07 -07:00
Ishaan Jaff
3984b9080c docs cleanup 2024-09-07 11:23:44 -07:00
Ishaan Jaff
2cf0714b0d docs organize sidebar 2024-09-07 11:23:06 -07:00
Ishaan Jaff
808ba36b55 ui cleanup 2024-09-07 11:20:07 -07:00
Ishaan Jaff
6c30f18f8c docs new presidio language controls 2024-09-04 13:04:19 -07:00
Krish Dholakia
be3c7b401e
LiteLLM Minor fixes + improvements (08/03/2024) (#5488)
* fix(internal_user_endpoints.py): set budget_reset_at for /user/update

* fix(vertex_and_google_ai_studio_gemini.py): handle accumulated json

Fixes https://github.com/BerriAI/litellm/issues/5479

* fix(vertex_ai_and_gemini.py): fix assistant message function call when content is not None

Fixes https://github.com/BerriAI/litellm/issues/5490

* fix(proxy_server.py): generic state uuid for okta sso

* fix(lago.py): improve debug logs

Debugging for https://github.com/BerriAI/litellm/issues/5477

* docs(bedrock.md): add bedrock cross-region inferencing to docs

* fix(azure.py): return azure response headers on aembedding call

* feat(azure.py): return azure response headers for `/audio/transcription`

* fix(types/utils.py): standardize deepseek / anthropic prompt caching usage information

Closes https://github.com/BerriAI/litellm/issues/5285

* docs(usage.md): add docs on litellm usage object

* test(test_completion.py): mark flaky test
2024-09-03 21:21:34 -07:00
Krrish Dholakia
9aa006d353 docs(bedrock.md): add multimodal embedding support to docs 2024-09-03 08:14:10 -07:00
Ishaan Jaff
fd4157cf71 docs add cerebras 2024-08-31 14:57:12 -07:00
Krrish Dholakia
601945d114 docs(docker_quick_start.md): add new quick start doc for litellm proxy 2024-08-29 15:35:39 -07:00
Ishaan Jaff
e396895288 update doc on palm provider 2024-08-27 21:11:24 -07:00
Ishaan Jaff
a5a17c120b docs add rerank api to docs 2024-08-27 18:06:59 -07:00
Ishaan Jaff
68bb735b3b docs use litellm proxy with litellm python sdk 2024-08-26 10:50:24 -07:00
Ishaan Jaff
d10430c881 doc custom guardrail 2024-08-23 09:41:54 -07:00
Ishaan Jaff
b054dd0e45 docs fix 2024-08-22 19:04:14 -07:00
Ishaan Jaff
25609a94ad docs moderation 2024-08-22 18:57:54 -07:00
Ishaan Jaff
b865993b34 docs move pass thru endpoints 2024-08-22 18:49:26 -07:00
Krrish Dholakia
11c7e92b58 docs(sidebars.js): refactor docs 2024-08-22 18:22:50 -07:00
Ishaan Jaff
70f9e41ed9
Merge branch 'main' into litellm_add_bedrock_guardrails 2024-08-22 17:28:49 -07:00
Ishaan Jaff
499b6b3368 doc bedrock guardrails 2024-08-22 16:25:22 -07:00
Michał Pstrąg
a37f004c1d Merge branch 'main' into docs-dbally 2024-08-22 23:25:57 +02:00
Michał Pstrąg
62df7c755b add dbally project 2024-08-22 23:21:40 +02:00
Ishaan Jaff
e7ecb2fe3a fix qdrant litellm on proxy 2024-08-21 12:52:29 -07:00
Ishaan Jaff
30da63bd4f docs move lakera to free 2024-08-20 16:38:37 -07:00
Krrish Dholakia
91aac0533b docs(langfuse.md): add doc on pass through 2024-08-19 21:28:48 -07:00
Ishaan Jaff
c7b3978655
Merge pull request #5288 from BerriAI/litellm_aporia_refactor
[Feat] V2 aporia guardrails litellm
2024-08-19 20:41:45 -07:00
Ishaan Jaff
b7d4031f89 doc aporia_w_litellm 2024-08-19 14:36:55 -07:00
Krrish Dholakia
a494b5b2f3 docs: add docs on bedrock + cohere pass through endpoints 2024-08-19 14:21:56 -07:00
Ishaan Jaff
d9c91838ce docs cleanup 2024-08-17 15:59:23 -07:00
Krish Dholakia
ff6ff133ee
Merge pull request #5260 from BerriAI/google_ai_studio_pass_through
Pass-through endpoints for Gemini - Google AI Studio
2024-08-17 13:51:51 -07:00
Krrish Dholakia
0df41653f3 docs(google_ai_studio.md): add docs on google ai studio pass through endpoints 2024-08-17 13:47:05 -07:00
Ishaan Jaff
9b0bd54571 docs cleanup - reduce emojis 2024-08-17 13:28:34 -07:00
Ishaan Jaff
0d41e2972b docs on oauth 2.0 2024-08-16 13:55:28 -07:00
Ishaan Jaff
7f39f9f97d docs cleanup 2024-08-16 11:38:53 -07:00
Krrish Dholakia
4cef6df4cf docs(sidebar.js): cleanup docs 2024-08-14 09:04:52 -07:00
Krrish Dholakia
0ea056971c docs(prefix.md): add prefix support to docs 2024-08-10 13:55:47 -07:00
Ishaan Jaff
0acc6efa8f docs clean sidebar 2024-08-09 18:09:11 -07:00