Krish Dholakia
3beecfb0d4
LiteLLM Minor Fixes & Improvements (11/13/2024) ( #6729 )
...
* fix(utils.py): add logprobs support for together ai
Fixes
https://github.com/BerriAI/litellm/issues/6724
* feat(pass_through_endpoints/): add anthropic/ pass-through endpoint
adds new `anthropic/` pass-through endpoint + refactors docs
* feat(spend_management_endpoints.py): allow /global/spend/report to query team + customer id
enables seeing spend for a customer in a team
* Add integration with MLflow Tracing (#6147 )
* Add MLflow logger
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Streaming handling
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* lint
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* address comments and fix issues
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* address comments and fix issues
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Move logger construction code
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Add docs
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* async handlers
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* new picture
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* fix(mlflow.py): fix ruff linting errors
* ci(config.yml): add mlflow to ci testing
* fix: fix test
* test: fix test
* Litellm key update fix (#6710 )
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* fix(key_management_endpoints.py): fix /key/update with metadata update
* fix(key_management_endpoints.py): fix key_prepare_update helper
* fix(key_management_endpoints.py): reset value to none if set in key update
* fix: update test
'
* Litellm dev 11 11 2024 (#6693 )
* fix(__init__.py): add 'watsonx_text' as mapped llm api route
Fixes https://github.com/BerriAI/litellm/issues/6663
* fix(opentelemetry.py): fix passing parallel tool calls to otel
Fixes https://github.com/BerriAI/litellm/issues/6677
* refactor(test_opentelemetry_unit_tests.py): create a base set of unit tests for all logging integrations - test for parallel tool call handling
reduces bugs in repo
* fix(__init__.py): update provider-model mapping to include all known provider-model mappings
Fixes https://github.com/BerriAI/litellm/issues/6669
* feat(anthropic): support passing document in llm api call
* docs(anthropic.md): add pdf anthropic call to docs + expose new 'supports_pdf_input' function
* fix(factory.py): fix linting error
* add clear doc string for GCS bucket logging
* Add docs to export logs to Laminar (#6674 )
* Add docs to export logs to Laminar
* minor fix: newline at end of file
* place laminar after http and grpc
* (Feat) Add langsmith key based logging (#6682 )
* add langsmith_api_key to StandardCallbackDynamicParams
* create a file for langsmith types
* langsmith add key / team based logging
* add key based logging for langsmith
* fix langsmith key based logging
* fix linting langsmith
* remove NOQA violation
* add unit test coverage for all helpers in test langsmith
* test_langsmith_key_based_logging
* docs langsmith key based logging
* run langsmith tests in logging callback tests
* fix logging testing
* test_langsmith_key_based_logging
* test_add_callback_via_key_litellm_pre_call_utils_langsmith
* add debug statement langsmith key based logging
* test_langsmith_key_based_logging
* (fix) OpenAI's optional messages[].name does not work with Mistral API (#6701 )
* use helper for _transform_messages mistral
* add test_message_with_name to base LLMChat test
* fix linting
* add xAI on Admin UI (#6680 )
* (docs) add benchmarks on 1K RPS (#6704 )
* docs litellm proxy benchmarks
* docs GCS bucket
* doc fix - reduce clutter on logging doc title
* (feat) add cost tracking stable diffusion 3 on Bedrock (#6676 )
* add cost tracking for sd3
* test_image_generation_bedrock
* fix get model info for image cost
* add cost_calculator for stability 1 models
* add unit testing for bedrock image cost calc
* test_cost_calculator_with_no_optional_params
* add test_cost_calculator_basic
* correctly allow size Optional
* fix cost_calculator
* sd3 unit tests cost calc
* fix raise correct error 404 when /key/info is called on non-existent key (#6653 )
* fix raise correct error on /key/info
* add not_found_error error
* fix key not found in DB error
* use 1 helper for checking token hash
* fix error code on key info
* fix test key gen prisma
* test_generate_and_call_key_info
* test fix test_call_with_valid_model_using_all_models
* fix key info tests
* bump: version 1.52.4 → 1.52.5
* add defaults used for GCS logging
* LiteLLM Minor Fixes & Improvements (11/12/2024) (#6705 )
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* bump: version 1.52.5 → 1.52.6
* (feat) helm hook to sync db schema (#6715 )
* v0 migration job
* fix job
* fix migrations job.yml
* handle standalone DB on helm hook
* fix argo cd annotations
* fix db migration helm hook
* fix migration job
* doc fix Using Http/2 with Hypercorn
* (fix proxy redis) Add redis sentinel support (#6154 )
* add sentinel_password support
* add doc for setting redis sentinel password
* fix redis sentinel - use sentinel password
* Fix: Update gpt-4o costs to that of gpt-4o-2024-08-06 (#6714 )
Fixes #6713
* (fix) using Anthropic `response_format={"type": "json_object"}` (#6721 )
* add support for response_format=json anthropic
* add test_json_response_format to baseLLM ChatTest
* fix test_litellm_anthropic_prompt_caching_tools
* fix test_anthropic_function_call_with_no_schema
* test test_create_json_tool_call_for_response_format
* (feat) Add cost tracking for Azure Dall-e-3 Image Generation + use base class to ensure basic image generation tests pass (#6716 )
* add BaseImageGenTest
* use 1 class for unit testing
* add debugging to BaseImageGenTest
* TestAzureOpenAIDalle3
* fix response_cost_calculator
* test_basic_image_generation
* fix img gen basic test
* fix _select_model_name_for_cost_calc
* fix test_aimage_generation_bedrock_with_optional_params
* fix undo changes cost tracking
* fix response_cost_calculator
* fix test_cost_azure_gpt_35
* fix remove dup test (#6718 )
* (build) update db helm hook
* (build) helm db pre sync hook
* (build) helm db sync hook
* test: run test_team_logging firdst
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
* test: update test
* test: skip anthropic overloaded error
* test: cleanup test
* test: update tests
* test: fix test
* test: handle gemini overloaded model error
* test: handle internal server error
* test: handle anthropic overloaded error
* test: handle claude instability
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
Co-authored-by: Yuki Watanabe <31463517+B-Step62@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
2024-11-15 11:18:31 +05:30
Krrish Dholakia
4f5ff65882
docs(argilla.md): add doc on argilla logging
2024-10-17 22:51:55 -07:00
Jacques Verré
4064bfc6dd
[Feat] Observability integration - Opik by Comet ( #6062 )
...
* Added Opik logging and evaluation
* Updated doc examples
* Default tags should be [] in case appending
* WIP
* Work in progress
* Opik integration
* Opik integration
* Revert changes on litellm_logging.py
* Updated Opik integration for synchronous API calls
* Updated Opik documentation
---------
Co-authored-by: Douglas Blank <doug@comet.com>
Co-authored-by: Doug Blank <doug.blank@gmail.com>
2024-10-10 18:27:50 +05:30
Ishaan Jaff
1bafbf8382
(feat proxy) add v2 maintained LiteLLM grafana dashboard ( #6098 )
...
* add new grafana dashboard litellm
* add v2 grafana dashboard
2024-10-07 18:11:20 +05:30
Ishaan Jaff
2449d258cf
(docs) add 1k rps load test doc ( #6059 )
...
* docs 1k rps load test
* docs load testing
* docs load testing litellm
* docs load testing
* clean up load test doc
* docs prom metrics for load testing
* docs using prometheus on load testing
* doc load testing with prometheus
2024-10-04 16:56:34 +05:30
Krish Dholakia
5c33d1c9af
Litellm Minor Fixes & Improvements (10/03/2024) ( #6049 )
...
* fix(proxy_server.py): remove spendlog fixes from proxy startup logic
Moves https://github.com/BerriAI/litellm/pull/4794 to `/db_scripts` and cleans up some caching-related debug info (easier to trace debug logs)
* fix(langfuse_endpoints.py): Fixes https://github.com/BerriAI/litellm/issues/6041
* fix(azure.py): fix health checks for azure audio transcription models
Fixes https://github.com/BerriAI/litellm/issues/5999
* Feat: Add Literal AI Integration (#5653 )
* feat: add Literal AI integration
* update readme
* Update README.md
* fix: address comments
* fix: remove literalai sdk
* fix: use HTTPHandler
* chore: add test
* fix: add asyncio lock
* fix(literal_ai.py): fix linting errors
* fix(literal_ai.py): fix linting errors
* refactor: cleanup
---------
Co-authored-by: Willy Douhard <willy.douhard@gmail.com>
2024-10-03 18:02:28 -04:00
Jannik Maierhöfer
52e971155a
[docs] updated langfuse integration guide ( #5921 )
2024-09-27 07:49:47 -07:00
Ishaan Jaff
4bdeefd7e4
docs service accounts ( #5900 )
2024-09-25 15:46:13 -07:00
Krish Dholakia
98c335acd0
LiteLLM Minor Fixes & Improvements (09/17/2024) ( #5742 )
...
* fix(proxy_server.py): use default azure credentials to support azure non-client secret kms
* fix(langsmith.py): raise error if credentials missing
* feat(langsmith.py): support error logging for langsmith + standard logging payload
Fixes https://github.com/BerriAI/litellm/issues/5738
* Fix hardcoding of schema in view check (#5749 )
* fix - deal with case when check view exists returns None (#5740 )
* Revert "fix - deal with case when check view exists returns None (#5740 )" (#5741 )
This reverts commit 535228159b55db19a1e4 (r1763712148)https://github.com/BerriAI/litellm/issues/5667
* feat(secret_detection.py): Allow configuring secret detection params
Allows admin to control what plugins to run for secret detection. Prevents overzealous secret detection.
* docs(secret_detection.md): add secret detection guardrail docs
* fix: fix linting errors
* fix - deal with case when check view exists returns None (#5740 )
* Revert "fix - deal with case when check view exists returns None (#5740 )" (#5741 )
This reverts commit 535228159b#5748 )
* test: fix testing - azure changed content policy error logic
* test: fix tests to use mock responses
* test(test_image_generation.py): handle api instability
* test(test_image_generation.py): handle azure api instability
* fix(utils.py): fix unbounded variable error
* fix(utils.py): fix unbounded variable error
* test: refactor test to use mock response
* test: mark flaky azure tests
* Bump next from 14.1.1 to 14.2.10 in /ui/litellm-dashboard (#5753 )
Bumps [next](https://github.com/vercel/next.js ) from 14.1.1 to 14.2.10.
- [Release notes](https://github.com/vercel/next.js/releases )
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js )
- [Commits](https://github.com/vercel/next.js/compare/v14.1.1...v14.2.10 )
---
updated-dependencies:
- dependency-name: next
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* [Fix] o1-mini causes pydantic warnings on `reasoning_tokens` (#5754 )
* add requester_metadata in standard logging payload
* log requester_metadata in metadata
* use StandardLoggingPayload for logging
* docs StandardLoggingPayload
* fix import
* include standard logging object in failure
* add test for requester metadata
* handle completion_tokens_details
* add test for completion_tokens_details
* [Feat-Proxy-DataDog] Log Redis, Postgres Failure events on DataDog (#5750 )
* dd - start tracking redis status on dd
* add async_service_succes_hook / failure hook in custom logger
* add async_service_failure_hook
* log service failures on dd
* fix import error
* add test for redis errors / warning
* [Fix] Router/ Proxy - Tag Based routing, raise correct error when no deployments found and tag filtering is on (#5745 )
* fix tag routing - raise correct error when no model with tag based routing
* fix error string from tag based routing
* test router tag based routing
* raise 401 error when no tags avialable for deploymen
* linting fix
* [Feat] Log Request metadata on gcs bucket logging (#5743 )
* add requester_metadata in standard logging payload
* log requester_metadata in metadata
* use StandardLoggingPayload for logging
* docs StandardLoggingPayload
* fix import
* include standard logging object in failure
* add test for requester metadata
* fix(litellm_logging.py): fix logging message
* fix(rerank_api/main.py): fix linting errors
* fix(custom_guardrails.py): maintain backwards compatibility for older guardrails
* fix(rerank_api/main.py): fix cost tracking for rerank endpoints
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: steffen-sbt <148480574+steffen-sbt@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-17 23:00:04 -07:00
Ishaan Jaff
54db564529
add arch diagram
2024-09-07 15:49:51 -07:00
Ishaan Jaff
b7d4031f89
doc aporia_w_litellm
2024-08-19 14:36:55 -07:00
Krrish Dholakia
7e1f296981
docs(self_serve.md): cleanup docs on how to onboard new users + teams
2024-08-07 19:58:36 -07:00
Ishaan Jaff
617b2b946c
docs using gcs
2024-08-01 15:23:17 -07:00
Krrish Dholakia
d1ffb4de5f
docs(raw_request_response.md): show how to get openai headers from response
2024-07-23 11:40:26 -07:00
Ishaan Jaff
f462e6a46c
langsmith logs
2024-07-17 16:37:24 -07:00
Ishaan Jaff
0d729e94c0
docs - how to invite users to view usage, caching analytics
2024-07-16 17:56:06 -07:00
Ishaan Jaff
1ac1078080
fix cost tracking img
2024-06-29 11:35:57 -07:00
Ishaan Jaff
c6c2617d70
docs - pass through langfuse requests on proxy
2024-06-28 17:53:13 -07:00
Krrish Dholakia
fa68115530
docs(cost_tracking.md): add litellm cost in proxy response headers to docs
2024-06-26 18:14:21 -07:00
Ishaan Jaff
3f803a96a3
docs - setting team budgets on ui
2024-06-19 15:04:01 -07:00
Krrish Dholakia
4eca63ede6
docs(alerting.md): add alerting metadata to docs
2024-06-14 19:04:16 -07:00
Ishaan Jaff
24129ea0f1
docs - custom logout url
2024-06-13 20:54:08 -07:00
Krrish Dholakia
d210eccb79
docs(alerting.md): add expected response teams alerting image to docs
2024-06-13 14:28:43 -07:00
Ishaan Jaff
53ca1fe09f
docs - run custom path
2024-06-12 14:16:31 -07:00
Ishaan Jaff
5eb2822d31
Merge pull request #4133 from BerriAI/litellm_work_with_traceparents
...
[Feat] OTEL - allow propagating traceparent in headers
2024-06-11 14:27:08 -07:00
Ishaan Jaff
d7f1445615
doc - OTEL trace propogation
2024-06-11 14:25:33 -07:00
Krrish Dholakia
7eae0ff7e3
fix(utils.py): allow user to opt in to raw request logging to langfuse
2024-06-11 13:35:22 -07:00
Krrish Dholakia
a8ea7c6d31
docs(ui.md): add okta sso support to docs
2024-06-11 13:17:41 -07:00
Krrish Dholakia
71c6065d98
docs(self_serve.md): cleanup images
2024-06-05 17:20:57 -07:00
Krrish Dholakia
ead4db3831
docs(self_server.md): add docs on how to allow users to create their own keys on proxy ui
2024-06-05 17:12:49 -07:00
Ishaan Jaff
03a98b2d4d
Merge pull request #3991 from BerriAI/litellm_spend_tracking_docs
...
[Doc] - Spend tracking with litellm
2024-06-03 21:54:10 -07:00
Ishaan Jaff
26002006d4
doc - spend tracking with litellm
2024-06-03 13:39:38 -07:00
Ishaan Jaff
de21bdb0ae
docs - show langfuse debug on docs
2024-06-03 12:40:22 -07:00
Krrish Dholakia
5cce31ae61
docs(enterprise.md): update enterprise docs with public model hub and custom email branding information
2024-06-01 09:01:05 -07:00
Krrish Dholakia
939ddf8986
docs(customers.md): tutorial for setting customer budgets on proxy
2024-05-29 17:46:47 -07:00
Ishaan Jaff
963ca495a2
docs- email notifs
2024-05-25 18:07:36 -07:00
alisalim17
01bb26bbba
Revert "Revert "Logfire Integration""
...
This reverts commit b04a8d878a
2024-05-21 11:07:40 +04:00
Krrish Dholakia
b723e608f6
docs(enterprise.md): add swagger - custom routes + branding to docs
2024-05-17 15:31:02 -07:00
Krrish Dholakia
b696d47442
docs(billing.md): update lago screenshot
2024-05-16 15:30:33 -07:00
Krrish Dholakia
3acb31fa49
docs(lago.md): add lago usage-based billing quick-start to docs
2024-05-16 13:24:04 -07:00
Krish Dholakia
b04a8d878a
Revert "Logfire Integration"
2024-05-14 17:38:47 -07:00
alisalim17
7e0b479a37
docs: add documentation for logfire integration
2024-05-04 17:47:54 +04:00
Krrish Dholakia
61d680143f
docs(openmeter.md): add openmeter to docs
2024-05-01 18:31:45 -07:00
Krrish Dholakia
3ce73fce23
docs(hosted.md): add hosted proxy info to docs
2024-04-20 09:30:28 -07:00
Krrish Dholakia
003cd3b102
docs(vertex.md): add tutorial for using vertex ai with gcp service account
2024-04-04 21:28:28 -07:00
Ishaan Jaff
ffa29ddfef
(docs) cleanup
2024-03-29 13:10:26 -07:00
Krrish Dholakia
526aa9230f
docs(call_hooks.md): show result in docs
2024-03-27 21:04:51 -07:00
Ishaan Jaff
038c9d5781
(docs) litellm + datadog
2024-03-18 17:06:00 -07:00
ishaan-jaff
91a47dc17a
(docs) how to run a locust load test
2024-03-15 07:37:50 -07:00
ishaan-jaff
2d71f54afb
(docs) load test litellm
2024-03-08 15:18:06 -08:00
