Krish Dholakia
b11bc0374e
Litellm dev 11 20 2024 ( #6838 )
...
* feat(customer_endpoints.py): support passing budget duration via `/customer/new` endpoint
Closes https://github.com/BerriAI/litellm/issues/5651
* docs: add missing params to swagger + api documentation test
* docs: add documentation for all key endpoints
documents all params on swagger
* docs(internal_user_endpoints.py): document all /user/new params
Ensures all params are documented
* docs(team_endpoints.py): add missing documentation for team endpoints
Ensures 100% param documentation on swagger
* docs(organization_endpoints.py): document all org params
Adds documentation for all params in org endpoint
* docs(customer_endpoints.py): add coverage for all params on /customer endpoints
ensures all /customer/* params are documented
* ci(config.yml): add endpoint doc testing to ci/cd
* fix: fix internal_user_endpoints.py
* fix(internal_user_endpoints.py): support 'duration' param
* fix(partner_models/main.py): fix anthropic re-raise exception on vertex
* fix: fix pydantic obj
* build(model_prices_and_context_window.json): add new vertex claude model names
vertex claude changed model names - causes cost tracking errors
2024-11-21 05:20:37 +05:30
Krish Dholakia
3beecfb0d4
LiteLLM Minor Fixes & Improvements (11/13/2024) ( #6729 )
...
* fix(utils.py): add logprobs support for together ai
Fixes
https://github.com/BerriAI/litellm/issues/6724
* feat(pass_through_endpoints/): add anthropic/ pass-through endpoint
adds new `anthropic/` pass-through endpoint + refactors docs
* feat(spend_management_endpoints.py): allow /global/spend/report to query team + customer id
enables seeing spend for a customer in a team
* Add integration with MLflow Tracing (#6147 )
* Add MLflow logger
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Streaming handling
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* lint
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* address comments and fix issues
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* address comments and fix issues
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Move logger construction code
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* Add docs
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* async handlers
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* new picture
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
* fix(mlflow.py): fix ruff linting errors
* ci(config.yml): add mlflow to ci testing
* fix: fix test
* test: fix test
* Litellm key update fix (#6710 )
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* fix(key_management_endpoints.py): fix /key/update with metadata update
* fix(key_management_endpoints.py): fix key_prepare_update helper
* fix(key_management_endpoints.py): reset value to none if set in key update
* fix: update test
'
* Litellm dev 11 11 2024 (#6693 )
* fix(__init__.py): add 'watsonx_text' as mapped llm api route
Fixes https://github.com/BerriAI/litellm/issues/6663
* fix(opentelemetry.py): fix passing parallel tool calls to otel
Fixes https://github.com/BerriAI/litellm/issues/6677
* refactor(test_opentelemetry_unit_tests.py): create a base set of unit tests for all logging integrations - test for parallel tool call handling
reduces bugs in repo
* fix(__init__.py): update provider-model mapping to include all known provider-model mappings
Fixes https://github.com/BerriAI/litellm/issues/6669
* feat(anthropic): support passing document in llm api call
* docs(anthropic.md): add pdf anthropic call to docs + expose new 'supports_pdf_input' function
* fix(factory.py): fix linting error
* add clear doc string for GCS bucket logging
* Add docs to export logs to Laminar (#6674 )
* Add docs to export logs to Laminar
* minor fix: newline at end of file
* place laminar after http and grpc
* (Feat) Add langsmith key based logging (#6682 )
* add langsmith_api_key to StandardCallbackDynamicParams
* create a file for langsmith types
* langsmith add key / team based logging
* add key based logging for langsmith
* fix langsmith key based logging
* fix linting langsmith
* remove NOQA violation
* add unit test coverage for all helpers in test langsmith
* test_langsmith_key_based_logging
* docs langsmith key based logging
* run langsmith tests in logging callback tests
* fix logging testing
* test_langsmith_key_based_logging
* test_add_callback_via_key_litellm_pre_call_utils_langsmith
* add debug statement langsmith key based logging
* test_langsmith_key_based_logging
* (fix) OpenAI's optional messages[].name does not work with Mistral API (#6701 )
* use helper for _transform_messages mistral
* add test_message_with_name to base LLMChat test
* fix linting
* add xAI on Admin UI (#6680 )
* (docs) add benchmarks on 1K RPS (#6704 )
* docs litellm proxy benchmarks
* docs GCS bucket
* doc fix - reduce clutter on logging doc title
* (feat) add cost tracking stable diffusion 3 on Bedrock (#6676 )
* add cost tracking for sd3
* test_image_generation_bedrock
* fix get model info for image cost
* add cost_calculator for stability 1 models
* add unit testing for bedrock image cost calc
* test_cost_calculator_with_no_optional_params
* add test_cost_calculator_basic
* correctly allow size Optional
* fix cost_calculator
* sd3 unit tests cost calc
* fix raise correct error 404 when /key/info is called on non-existent key (#6653 )
* fix raise correct error on /key/info
* add not_found_error error
* fix key not found in DB error
* use 1 helper for checking token hash
* fix error code on key info
* fix test key gen prisma
* test_generate_and_call_key_info
* test fix test_call_with_valid_model_using_all_models
* fix key info tests
* bump: version 1.52.4 → 1.52.5
* add defaults used for GCS logging
* LiteLLM Minor Fixes & Improvements (11/12/2024) (#6705 )
* fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
* fix(caching_handler.py): don't pass args to caching
* fix(caching): remove all *args from caching.py
* fix(caching): consistent function signatures + abc method
* test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one
* fix(router.py): drop redis password requirement
* fix(proxy_server.py): fix faulty slack alerting check
* fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
* test: update test
* bump: version 1.52.5 → 1.52.6
* (feat) helm hook to sync db schema (#6715 )
* v0 migration job
* fix job
* fix migrations job.yml
* handle standalone DB on helm hook
* fix argo cd annotations
* fix db migration helm hook
* fix migration job
* doc fix Using Http/2 with Hypercorn
* (fix proxy redis) Add redis sentinel support (#6154 )
* add sentinel_password support
* add doc for setting redis sentinel password
* fix redis sentinel - use sentinel password
* Fix: Update gpt-4o costs to that of gpt-4o-2024-08-06 (#6714 )
Fixes #6713
* (fix) using Anthropic `response_format={"type": "json_object"}` (#6721 )
* add support for response_format=json anthropic
* add test_json_response_format to baseLLM ChatTest
* fix test_litellm_anthropic_prompt_caching_tools
* fix test_anthropic_function_call_with_no_schema
* test test_create_json_tool_call_for_response_format
* (feat) Add cost tracking for Azure Dall-e-3 Image Generation + use base class to ensure basic image generation tests pass (#6716 )
* add BaseImageGenTest
* use 1 class for unit testing
* add debugging to BaseImageGenTest
* TestAzureOpenAIDalle3
* fix response_cost_calculator
* test_basic_image_generation
* fix img gen basic test
* fix _select_model_name_for_cost_calc
* fix test_aimage_generation_bedrock_with_optional_params
* fix undo changes cost tracking
* fix response_cost_calculator
* fix test_cost_azure_gpt_35
* fix remove dup test (#6718 )
* (build) update db helm hook
* (build) helm db pre sync hook
* (build) helm db sync hook
* test: run test_team_logging firdst
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
* test: update test
* test: skip anthropic overloaded error
* test: cleanup test
* test: update tests
* test: fix test
* test: handle gemini overloaded model error
* test: handle internal server error
* test: handle anthropic overloaded error
* test: handle claude instability
---------
Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
Co-authored-by: Yuki Watanabe <31463517+B-Step62@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com>
Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
2024-11-15 11:18:31 +05:30
Dinmukhamed Mailibay
1e2ba3e045
Add docs to export logs to Laminar ( #6674 )
...
* Add docs to export logs to Laminar
* minor fix: newline at end of file
* place laminar after http and grpc
2024-11-11 12:15:47 -08:00
Ronen Schaffer
8a2b6fd8d2
Update opentelemetry_integration.md - Fix typos ( #6618 )
2024-11-06 17:20:31 -08:00
Ishaan Jaff
b75019c1a5
(feat) Arize - Allow using Arize HTTP endpoint ( #6364 )
...
* arize use helper for get_arize_opentelemetry_config
* use helper to get Arize OTEL config
* arize add helpers for arize
* docs allow using arize http endpoint
* fix importing OTEL for Arize
* use static methods for ArizeLogger
* fix ArizeLogger tests
2024-10-23 09:38:35 +05:30
Krrish Dholakia
6e7e96211c
docs(argilla.md): add sampling rate to argilla calls
2024-10-17 22:54:12 -07:00
Krrish Dholakia
4f5ff65882
docs(argilla.md): add doc on argilla logging
2024-10-17 22:51:55 -07:00
Ali Waleed
7ec414a3cf
Feat: Add Langtrace integration ( #5341 )
...
* Feat: Add Langtrace integration
* add langtrace service name
* fix timestamps for traces
* add tests
* Discard Callback + use existing otel logger
* cleanup
* remove print statments
* remove callback
* add docs
* docs
* add logging docs
* format logging
* remove emoji and add litellm proxy example
* format logging
* format `logging.md`
* add langtrace docs to logging.md
* sync conflict
2024-10-11 19:19:53 +05:30
Willy Douhard
8b00d2a25f
Add literalai in the sidebar observability category ( #6163 )
...
* fix: add literalai in the sidebar
* fix: typo
2024-10-11 19:18:47 +05:30
Jacques Verré
4064bfc6dd
[Feat] Observability integration - Opik by Comet ( #6062 )
...
* Added Opik logging and evaluation
* Updated doc examples
* Default tags should be [] in case appending
* WIP
* Work in progress
* Opik integration
* Opik integration
* Revert changes on litellm_logging.py
* Updated Opik integration for synchronous API calls
* Updated Opik documentation
---------
Co-authored-by: Douglas Blank <doug@comet.com>
Co-authored-by: Doug Blank <doug.blank@gmail.com>
2024-10-10 18:27:50 +05:30
Krish Dholakia
5c33d1c9af
Litellm Minor Fixes & Improvements (10/03/2024) ( #6049 )
...
* fix(proxy_server.py): remove spendlog fixes from proxy startup logic
Moves https://github.com/BerriAI/litellm/pull/4794 to `/db_scripts` and cleans up some caching-related debug info (easier to trace debug logs)
* fix(langfuse_endpoints.py): Fixes https://github.com/BerriAI/litellm/issues/6041
* fix(azure.py): fix health checks for azure audio transcription models
Fixes https://github.com/BerriAI/litellm/issues/5999
* Feat: Add Literal AI Integration (#5653 )
* feat: add Literal AI integration
* update readme
* Update README.md
* fix: address comments
* fix: remove literalai sdk
* fix: use HTTPHandler
* chore: add test
* fix: add asyncio lock
* fix(literal_ai.py): fix linting errors
* fix(literal_ai.py): fix linting errors
* refactor: cleanup
---------
Co-authored-by: Willy Douhard <willy.douhard@gmail.com>
2024-10-03 18:02:28 -04:00
Ishaan Jaff
b817974c8e
docs clean up langfuse.md
2024-09-28 18:59:02 -07:00
Krrish Dholakia
70df474e64
docs: resolve imports
2024-09-27 13:36:29 -07:00
Jannik Maierhöfer
52e971155a
[docs] updated langfuse integration guide ( #5921 )
2024-09-27 07:49:47 -07:00
Ishaan Jaff
a9caba33ef
[Feat] Allow setting custom arize endpoint ( #5709 )
...
* set arize endpoint
* docs arize endpoint
* fix arize endpoint
2024-09-21 13:12:00 -07:00
Ishaan Jaff
a4549b5b6c
docs update what gets logged on gcs buckets
2024-09-18 10:18:57 -07:00
Krish Dholakia
234185ec13
LiteLLM Minor Fixes & Improvements (09/16/2024) ( #5723 ) ( #5731 )
...
* LiteLLM Minor Fixes & Improvements (09/16/2024) (#5723 )
* coverage (#5713 )
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* Move (#5714 )
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix(litellm_logging.py): fix logging client re-init (#5710 )
Fixes https://github.com/BerriAI/litellm/issues/5695
* fix(presidio.py): Fix logging_hook response and add support for additional presidio variables in guardrails config
Fixes https://github.com/BerriAI/litellm/issues/5682
* feat(o1_handler.py): fake streaming for openai o1 models
Fixes https://github.com/BerriAI/litellm/issues/5694
* docs: deprecated traceloop integration in favor of native otel (#5249 )
* fix: fix linting errors
* fix: fix linting errors
* fix(main.py): fix o1 import
---------
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>
* feat(spend_management_endpoints.py): expose `/global/spend/refresh` endpoint for updating material view (#5730 )
* feat(spend_management_endpoints.py): expose `/global/spend/refresh` endpoint for updating material view
Supports having `MonthlyGlobalSpend` view be a material view, and exposes an endpoint to refresh it
* fix(custom_logger.py): reset calltype
* fix: fix linting errors
* fix: fix linting error
* fix: fix import
* test(test_databricks.py): fix databricks tests
---------
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>
2024-09-17 08:05:52 -07:00
Ishaan Jaff
0fca960221
docs langfuse link
2024-08-27 08:14:03 -07:00
MarkRx
19e5299a6d
Support LangSmith parent_run_id, trace_id, session_id
2024-08-21 16:09:30 -04:00
Ishaan Jaff
9b0bd54571
docs cleanup - reduce emojis
2024-08-17 13:28:34 -07:00
Krrish Dholakia
d991e1320c
docs(langfuse_integration.md): add disable logging for specific calls to docs
2024-08-16 17:36:13 -07:00
Ishaan Jaff
9314341b9e
docs show fields logged on gcs bucket
2024-08-05 08:58:54 -07:00
Ishaan Jaff
3e39e8eb8f
docs setting service accounts
2024-08-01 15:31:58 -07:00
Ishaan Jaff
617b2b946c
docs using gcs
2024-08-01 15:23:17 -07:00
Ishaan Jaff
e378ab8bc9
docs - logging langsmith tags
2024-07-24 07:12:36 -07:00
Krrish Dholakia
d1ffb4de5f
docs(raw_request_response.md): show how to get openai headers from response
2024-07-23 11:40:26 -07:00
Krish Dholakia
e770ac48a2
Merge branch 'main' into litellm_braintrust_integration
2024-07-22 22:40:39 -07:00
Krrish Dholakia
dd6d58d29b
docs(braintrust.md): add braintrust.md to docs
2024-07-22 17:47:08 -07:00
Ishaan Jaff
d3bced56bb
docs - langsmith
2024-07-22 15:17:24 -07:00
Ishaan Jaff
488aca98a1
doc arize ai
2024-07-22 14:27:45 -07:00
maamalama
9d157c50a4
Helicone headers to metadata
2024-07-17 17:06:42 -07:00
Ishaan Jaff
ee53b9093b
Merge pull request #4758 from BerriAI/litellm_langsmith_async_support
...
[Feat] Use Async Httpx client for langsmith logging
2024-07-17 16:54:40 -07:00
Ishaan Jaff
f462e6a46c
langsmith logs
2024-07-17 16:37:24 -07:00
Krrish Dholakia
e587d32058
docs(scrub_data.md): add scrub logging data tutorial to docs
2024-07-17 15:39:31 -07:00
Krish Dholakia
1362a91d66
Merge pull request #4612 from colegottdank/main
...
Update Helicone Docs
2024-07-11 22:00:30 -07:00
Krrish Dholakia
2f5f85cb52
docs(callbacks.md): cleanup docs
2024-07-09 08:43:03 -07:00
colegottdank
0de776bca2
bye palm
2024-07-08 15:30:33 -07:00
colegottdank
e7c16cc488
Update Helicone docs
2024-07-08 15:27:58 -07:00
colegottdank
a3d9e34b26
Fix 2
2024-07-08 13:56:53 -07:00
Ishaan Jaff
70a605b3cc
docs - update telemetry
2024-06-24 13:21:16 -07:00
Ishaan Jaff
7b414a73a7
docs - show setting trace name
2024-06-18 20:12:00 -07:00
Ishaan Jaff
604f9689d0
fix setting langfuse attributes thru header
2024-06-18 12:09:58 -07:00
Ishaan Jaff
991604986d
docs - logfire
2024-06-17 19:47:31 -07:00
Ishaan Jaff
61ff712a08
fix deps
2024-06-17 19:33:10 -07:00
Ishaan Jaff
b4acd483c4
update logging docs
2024-06-17 17:31:58 -07:00
Ishaan Jaff
d8dd0408b0
docs - fix build time issues
2024-06-15 16:12:37 -07:00
Krish Dholakia
056913fd70
Merge pull request #3559 from Intellegam/main
...
Langfuse integration support for `parent_observation_id` parameter
2024-06-14 06:55:45 -07:00
Krrish Dholakia
fb96f07ccb
docs(raw_request_prod.md): add seeing raw request / response for proxy on docs
2024-06-12 11:00:29 -07:00
Krrish Dholakia
7eae0ff7e3
fix(utils.py): allow user to opt in to raw request logging to langfuse
2024-06-11 13:35:22 -07:00
Takashi Iwamoto
dbfdbf37c9
Fix log message in Custom Callbacks doc
2024-06-07 22:20:16 +09:00
