Commit graph

18325 commits

Author SHA1 Message Date
Krrish Dholakia
ad1e77b516 test: run test_team_logging firdst 2024-11-14 00:37:21 +05:30
Ishaan Jaff
dd680536d9 (build) helm db sync hook 2024-11-14 00:24:54 +05:30
Ishaan Jaff
6771647030 (build) helm db pre sync hook 2024-11-14 00:24:54 +05:30
Ishaan Jaff
be64f924ab (build) update db helm hook 2024-11-14 00:24:54 +05:30
Ishaan Jaff
a096c01841 fix remove dup test (#6718) 2024-11-14 00:24:54 +05:30
Ishaan Jaff
e93713318d (feat) Add cost tracking for Azure Dall-e-3 Image Generation + use base class to ensure basic image generation tests pass (#6716)
* add BaseImageGenTest

* use 1 class for unit testing

* add debugging to BaseImageGenTest

* TestAzureOpenAIDalle3

* fix response_cost_calculator

* test_basic_image_generation

* fix img gen basic test

* fix _select_model_name_for_cost_calc

* fix test_aimage_generation_bedrock_with_optional_params

* fix undo changes cost tracking

* fix response_cost_calculator

* fix test_cost_azure_gpt_35
2024-11-14 00:24:54 +05:30
Ishaan Jaff
4b0c7b5087 (fix) using Anthropic response_format={"type": "json_object"} (#6721)
* add support for response_format=json anthropic

* add test_json_response_format to baseLLM ChatTest

* fix test_litellm_anthropic_prompt_caching_tools

* fix test_anthropic_function_call_with_no_schema

* test test_create_json_tool_call_for_response_format
2024-11-14 00:24:54 +05:30
Kilian Lieret
124b600aeb Fix: Update gpt-4o costs to that of gpt-4o-2024-08-06 (#6714)
Fixes #6713
2024-11-14 00:24:54 +05:30
Ishaan Jaff
2bd0167ec7 (fix proxy redis) Add redis sentinel support (#6154)
* add sentinel_password support

* add doc for setting redis sentinel password

* fix redis sentinel - use sentinel password
2024-11-14 00:24:54 +05:30
Ishaan Jaff
2359de465b doc fix Using Http/2 with Hypercorn 2024-11-14 00:24:54 +05:30
Ishaan Jaff
20f119b1cf fix migration job 2024-11-14 00:24:54 +05:30
Ishaan Jaff
e68cac59af fix db migration helm hook 2024-11-14 00:24:54 +05:30
Ishaan Jaff
172bff0ca5 fix argo cd annotations 2024-11-14 00:24:54 +05:30
Ishaan Jaff
3677ced946 handle standalone DB on helm hook 2024-11-14 00:24:54 +05:30
Ishaan Jaff
79a50e6477 fix migrations job.yml 2024-11-14 00:24:54 +05:30
Ishaan Jaff
390541c774 (feat) helm hook to sync db schema (#6715)
* v0 migration job

* fix job
2024-11-14 00:24:54 +05:30
Krrish Dholakia
1f489dd875 bump: version 1.52.5 → 1.52.6 2024-11-14 00:24:54 +05:30
Krish Dholakia
09365cb758 LiteLLM Minor Fixes & Improvements (11/12/2024) (#6705)
* fix(caching): convert arg to equivalent kwargs in llm caching handler

prevent unexpected errors

* fix(caching_handler.py): don't pass args to caching

* fix(caching): remove all *args from caching.py

* fix(caching): consistent function signatures + abc method

* test(caching_unit_tests.py): add unit tests for llm caching

ensures coverage for common caching scenarios across different implementations

* refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one

* fix(router.py): drop redis password requirement

* fix(proxy_server.py): fix faulty slack alerting check

* fix(langfuse.py): avoid copying functions/thread lock objects in metadata

fixes metadata copy error when parent otel span in metadata

* test: update test
2024-11-14 00:24:54 +05:30
Ishaan Jaff
1adc6d5f0c add defaults used for GCS logging 2024-11-14 00:24:54 +05:30
Ishaan Jaff
fae59f257c bump: version 1.52.4 → 1.52.5 2024-11-14 00:24:54 +05:30
Ishaan Jaff
5fb3a69405 fix raise correct error 404 when /key/info is called on non-existent key (#6653)
* fix raise correct error on /key/info

* add not_found_error error

* fix key not found in DB error

* use 1 helper for checking token hash

* fix error code on key info

* fix test key gen prisma

* test_generate_and_call_key_info

* test fix test_call_with_valid_model_using_all_models

* fix key info tests
2024-11-14 00:24:54 +05:30
Ishaan Jaff
99b205d5b1 (feat) add cost tracking stable diffusion 3 on Bedrock (#6676)
* add cost tracking for sd3

* test_image_generation_bedrock

* fix get model info for image cost

* add cost_calculator for stability 1 models

* add unit testing for bedrock image cost calc

* test_cost_calculator_with_no_optional_params

* add test_cost_calculator_basic

* correctly allow size Optional

* fix cost_calculator

* sd3 unit tests cost calc
2024-11-14 00:24:54 +05:30
Ishaan Jaff
37ee207cde (docs) add benchmarks on 1K RPS (#6704)
* docs litellm proxy benchmarks

* docs GCS bucket

* doc fix - reduce clutter on logging doc title
2024-11-14 00:24:54 +05:30
Ishaan Jaff
cff7aa5208 add xAI on Admin UI (#6680) 2024-11-14 00:24:54 +05:30
Ishaan Jaff
f3da4af03e (fix) OpenAI's optional messages[].name does not work with Mistral API (#6701)
* use helper for _transform_messages mistral

* add test_message_with_name to base LLMChat test

* fix linting
2024-11-14 00:24:54 +05:30
Ishaan Jaff
91bfa58a25 (Feat) Add langsmith key based logging (#6682)
* add langsmith_api_key to StandardCallbackDynamicParams

* create a file for langsmith types

* langsmith add key / team based logging

* add key based logging for langsmith

* fix langsmith key based logging

* fix linting langsmith

* remove NOQA violation

* add unit test coverage for all helpers in test langsmith

* test_langsmith_key_based_logging

* docs langsmith key based logging

* run langsmith tests in logging callback tests

* fix logging testing

* test_langsmith_key_based_logging

* test_add_callback_via_key_litellm_pre_call_utils_langsmith

* add debug statement langsmith key based logging

* test_langsmith_key_based_logging
2024-11-14 00:24:54 +05:30
Dinmukhamed Mailibay
3ad8092044 Add docs to export logs to Laminar (#6674)
* Add docs to export logs to Laminar

* minor fix: newline at end of file

* place laminar after http and grpc
2024-11-14 00:24:54 +05:30
Ishaan Jaff
60e438840c add clear doc string for GCS bucket logging 2024-11-14 00:24:54 +05:30
Krish Dholakia
e470c087ee Litellm dev 11 11 2024 (#6693)
* fix(__init__.py): add 'watsonx_text' as mapped llm api route

Fixes https://github.com/BerriAI/litellm/issues/6663

* fix(opentelemetry.py): fix passing parallel tool calls to otel

Fixes https://github.com/BerriAI/litellm/issues/6677

* refactor(test_opentelemetry_unit_tests.py): create a base set of unit tests for all logging integrations - test for parallel tool call handling

reduces bugs in repo

* fix(__init__.py): update provider-model mapping to include all known provider-model mappings

Fixes https://github.com/BerriAI/litellm/issues/6669

* feat(anthropic): support passing document in llm api call

* docs(anthropic.md): add pdf anthropic call to docs + expose new 'supports_pdf_input' function

* fix(factory.py): fix linting error
2024-11-14 00:24:54 +05:30
Krrish Dholakia
e3f3f64afa fix: update test
'
2024-11-13 14:46:07 +05:30
Krrish Dholakia
4dc0d2d83d fix(key_management_endpoints.py): reset value to none if set in key update 2024-11-13 12:20:33 +05:30
Krrish Dholakia
e3e4b29106 fix(key_management_endpoints.py): fix key_prepare_update helper 2024-11-13 11:41:30 +05:30
Krrish Dholakia
e86a47b0ed fix(key_management_endpoints.py): fix /key/update with metadata update 2024-11-12 22:58:18 +05:30
Krrish Dholakia
75446852f8 test: update test 2024-11-12 21:25:17 +05:30
Krrish Dholakia
96464a317c fix(langfuse.py): avoid copying functions/thread lock objects in metadata
fixes metadata copy error when parent otel span in metadata
2024-11-12 15:26:43 +05:30
Krrish Dholakia
da6cb4ce1a fix(proxy_server.py): fix faulty slack alerting check 2024-11-12 14:16:56 +05:30
Krrish Dholakia
56d0f53c75 fix(router.py): drop redis password requirement 2024-11-12 14:08:50 +05:30
Krrish Dholakia
fcec2a325e refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one 2024-11-12 13:32:11 +05:30
Krrish Dholakia
16bbed72d4 test(caching_unit_tests.py): add unit tests for llm caching
ensures coverage for common caching scenarios across different implementations
2024-11-12 13:21:22 +05:30
Krrish Dholakia
0bc9864c09 fix(caching): consistent function signatures + abc method 2024-11-12 12:44:52 +05:30
Krrish Dholakia
a210214858 fix(caching): remove all *args from caching.py 2024-11-12 12:23:36 +05:30
Krrish Dholakia
a6ef9ab851 fix(caching_handler.py): don't pass args to caching 2024-11-12 12:17:53 +05:30
Krrish Dholakia
cba545a79d fix(caching): convert arg to equivalent kwargs in llm caching handler
prevent unexpected errors
2024-11-12 12:14:38 +05:30
Krrish Dholakia
b8ae08b8eb bump: version 1.52.3 → 1.52.4 2024-11-11 22:57:33 +05:30
Ishaan Jaff
eb92ed4156
(Feat) 273% improvement GCS Bucket Logger - use Batched Logging (#6679)
* use CustomBatchLogger for GCS

* add GCS bucket logging type

* use batch logging for GCs bucket

* add gcs_bucket

* allow setting flush_interval on CustomBatchLogger

* set GCS_FLUSH_INTERVAL to 1s

* fix test_key_logging

* fix test_key_logging

* add docs on new env vars
2024-11-11 11:35:34 +05:30
Ishaan Jaff
70aa85af1f fix model cost map stability.sd3-large-v1:0 2024-11-08 19:51:35 -08:00
Ishaan Jaff
979dfe8ab2
(feat) Add Bedrock Stability.ai Stable Diffusion 3 Image Generation models (#6673)
* add bedrock image gen async support

* added async support for bedrock image gen

* move image gen testing

* add AmazonStability3Config

* add AmazonStability3Config config

* update AmazonStabilityConfig

* update get_optional_params_image_gen

* use 1 helper for _get_request_body

* add transform_response_dict_to_openai_response for stability3

* test sd3-large-v1:0

* unit testing for bedrock image gen

* fix load_vertex_ai_credentials

* fix test_aimage_generation_vertex_ai

* add stability.sd3-large-v1:0 to model cost map

* add stability.stability.sd3-large-v1:0 to docs
2024-11-08 19:26:03 -08:00
Ishaan Jaff
0871c33a24
Merge pull request #6672 from BerriAI/litellm_add_async_bedrock_image_gen
(feat) add bedrock image gen async support
2024-11-08 19:25:02 -08:00
Ishaan Jaff
61026e189d
(feat) Add support for logging to GCS Buckets with folder paths (#6675)
* use helper to log

* gcs _handle_folders_in_bucket_name

* add test_basic_gcs_logger_with_folder_in_bucket_name

* run gcs testing in logging callback tests

* include correct deps

* fix gcs bucket logging test

* fix test_basic_gcs_logger_with_folder_in_bucket_name

* fix test_get_gcs_logging_config_without_service_account

* fix test gcs bucket

* remove unused file
2024-11-08 19:24:18 -08:00
Ishaan Jaff
086e1ac5bc move image gen testing 2024-11-08 14:10:08 -08:00