phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-27 03:34:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
17576 commits 2497 branches 897 tags 550 MiB
Commit graph

143 commits

Author SHA1 Message Date
Ishaan Jaff
02325f33d7 Merge branch 'main' into litellm_allow_turning_off_message_logging_for_callbacks 2024-09-09 21:59:36 -07:00
Krish Dholakia
52849e6422 LiteLLM Minor Fixes and Improvements (09/07/2024) (#5580) 
* fix(litellm_logging.py): set completion_start_time_float to end_time_float if none

Fixes https://github.com/BerriAI/litellm/issues/5500

* feat(_init_.py): add new 'openai_text_completion_compatible_providers' list

Fixes https://github.com/BerriAI/litellm/issues/5558

Handles correctly routing fireworks ai calls when done via text completions

* fix: fix linting errors

* fix: fix linting errors

* fix(openai.py): fix exception raised

* fix(openai.py): fix error handling

* fix(_redis.py): allow all supported arguments for redis cluster (#5554)

* Revert "fix(_redis.py): allow all supported arguments for redis cluster (#5554)" (#5583)

This reverts commit f2191ef4cb.

* fix(router.py): return model alias w/ underlying deployment on router.get_model_list()

Fixes https://github.com/BerriAI/litellm/issues/5524#issuecomment-2336410666

* test: handle flaky tests

---------

Co-authored-by: Jonas Dittrich <58814480+Kakadus@users.noreply.github.com>
 2024-09-09 18:54:17 -07:00
Ishaan Jaff
09f1631f60 use redact_message_input_output_from_custom_logger 2024-09-09 16:02:24 -07:00
Ishaan Jaff
e87ab72aa4 refactor redact_message_input_output_from_custom_logger 2024-09-09 16:00:47 -07:00
Ishaan Jaff
07623cd234 Merge pull request #5573 from BerriAI/litellm_add_rerank_spend_tracking 
[Feat] Add cost tracking for cohere rerank
 2024-09-06 18:11:30 -07:00
Krish Dholakia
2cab33b061 LiteLLM Minor Fixes and Improvements (08/06/2024) (#5567) 
* fix(utils.py): return citations for perplexity streaming

Fixes https://github.com/BerriAI/litellm/issues/5535

* fix(anthropic/chat.py): support fallbacks for anthropic streaming (#5542)

* fix(anthropic/chat.py): support fallbacks for anthropic streaming

Fixes https://github.com/BerriAI/litellm/issues/5512

* fix(anthropic/chat.py): use module level http client if none given (prevents early client closure)

* fix: fix linting errors

* fix(http_handler.py): fix raise_for_status error handling

* test: retry flaky test

* fix otel type

* fix(bedrock/embed): fix error raising

* test(test_openai_batches_and_files.py): skip azure batches test (for now) quota exceeded

* fix(test_router.py): skip azure batch route test (for now) - hit batch quota limits

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

* All `model_group_alias` should show up in `/models`, `/model/info` , `/model_group/info` (#5539)

* fix(router.py): support returning model_alias model names in `/v1/models`

* fix(proxy_server.py): support returning model alias'es on `/model/info`

* feat(router.py): support returning model group alias for `/model_group/info`

* fix(proxy_server.py): fix linting errors

* fix(proxy_server.py): fix linting errors

* build(model_prices_and_context_window.json): add amazon titan text premier pricing information

Closes https://github.com/BerriAI/litellm/issues/5560

* feat(litellm_logging.py): log standard logging response object for pass through endpoints. Allows bedrock /invoke agent calls to be correctly logged to langfuse + s3

* fix(success_handler.py): fix linting error

* fix(success_handler.py): fix linting errors

* fix(team_endpoints.py): Allows admin to update team member budgets

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
 2024-09-06 17:16:24 -07:00
Ishaan Jaff
8bd57b6167 add cost tracking for rerank+ test 2024-09-06 16:06:19 -07:00
Ishaan Jaff
6f68e860e0 fix import error 2024-09-05 10:09:44 -07:00
Ishaan Jaff
7370a994f5 use correct type hints for audio transcriptions 2024-09-05 09:12:27 -07:00
Ishaan Jaff
b468ccbb77 Merge pull request #5514 from BerriAI/litellm_add_presidio 
[Fix-Refactor] support presidio on new guardrails config
 2024-09-04 16:09:54 -07:00
Stefan Bokarev
7558e49d78 stream response (#5516) 2024-09-04 16:02:29 -07:00
Ishaan Jaff
a5f892bb6d fix init presidio guardrail 2024-09-04 15:22:37 -07:00
Ishaan Jaff
5e67a32d07 handle logging_only logic for guardrails 2024-09-04 13:57:04 -07:00
Ishaan Jaff
fdf9cbb846 fix get llm provider logic 2024-09-04 09:28:23 -07:00
Ishaan Jaff
44a1677c7c test get llm provider 2024-09-04 09:22:19 -07:00
Krish Dholakia
ca4e746545 LiteLLM minor fixes + improvements (31/08/2024) (#5464) 
* fix(vertex_endpoints.py): fix vertex ai pass through endpoints

* test(test_streaming.py): skip model due to end of life

* feat(custom_logger.py): add special callback for model hitting tpm/rpm limits

Closes https://github.com/BerriAI/litellm/issues/4096
 2024-09-01 13:31:42 -07:00
Krish Dholakia
e474c3665a Bedrock Embeddings refactor + model support (#5462) 
* refactor(bedrock): initial commit to refactor bedrock to a folder

Improve code readability + maintainability

* refactor: more refactor work

* fix: fix imports

* feat(bedrock/embeddings.py): support translating embedding into amazon embedding formats

* fix: fix linting errors

* test: skip test on end of life model

* fix(cohere/embed.py): fix linting error

* fix(cohere/embed.py): fix typing

* fix(cohere/embed.py): fix post-call logging for cohere embedding call

* test(test_embeddings.py): fix error message assertion in test
 2024-09-01 13:29:58 -07:00
Krish Dholakia
321b0961b5 fix: Minor LiteLLM Fixes + Improvements (29/08/2024) (#5436) 
* fix(model_checks.py): support returning wildcard models on `/v1/models`

Fixes https://github.com/BerriAI/litellm/issues/4903

* fix(bedrock_httpx.py): support calling bedrock via api_base

Closes https://github.com/BerriAI/litellm/pull/4587

* fix(litellm_logging.py): only leave last 4 char of gemini key unmasked

Fixes https://github.com/BerriAI/litellm/issues/5433

* feat(router.py): support setting 'weight' param for models on router

Closes https://github.com/BerriAI/litellm/issues/5410

* test(test_bedrock_completion.py): add unit test for custom api base

* fix(model_checks.py): handle no "/" in model
 2024-08-29 22:40:25 -07:00
Ishaan Jaff
da43356242 Merge pull request #5431 from BerriAI/litellm_Add_fireworks_ai_health_check 
[Fix-Proxy] /health check for provider wildcard models (fireworks/*)
 2024-08-29 14:25:05 -07:00
Ishaan Jaff
d2e286e45d add util to pick_cheapest_model_from_llm_provider 2024-08-29 09:27:20 -07:00
Krish Dholakia
3e8f5009f4 fix(utils.py): correctly log streaming cache hits (#5417) (#5426) 
Fixes https://github.com/BerriAI/litellm/issues/5401
 2024-08-28 22:50:33 -07:00
Ishaan Jaff
bae54ca642 use cost per token for jamba 2024-08-27 14:18:04 -07:00
Krrish Dholakia
145e7794b4 build(config.yml): bump anyio version 2024-08-27 07:37:06 -07:00
Krrish Dholakia
d02cfcde97 fix(asyncify.py): fix linting errors 2024-08-27 07:37:06 -07:00
Krrish Dholakia
d12ec470f7 fix(asyncify.py): fix linting errors 2024-08-27 07:37:06 -07:00
Krrish Dholakia
07dd3c640b perf(sagemaker.py): asyncify hf prompt template check 
leads to 189% improvement in RPS @ 100 users
 2024-08-27 07:37:06 -07:00
Krrish Dholakia
21bad0aa73 fix(streaming_utils.py): fix generic_chunk_has_all_required_fields 2024-08-26 21:13:02 -07:00
Krrish Dholakia
b989762bb0 fix(sagemaker.py): support streaming for messages api 
Fixes https://github.com/BerriAI/litellm/issues/5372
 2024-08-26 15:08:08 -07:00
Ishaan Jaff
8162208a5c track api_call_start_time 2024-08-22 13:52:03 -07:00
Krrish Dholakia
d6bc37374e feat(litellm_logging.py): add 'saved_cache_cost' to standard logging payload (s3) 2024-08-21 16:58:07 -07:00
Krrish Dholakia
ac5c6c8751 fix(litellm_pre_call_utils.py): handle dynamic keys via api correctly 2024-08-21 13:37:21 -07:00
Krrish Dholakia
77a6f597e0 fix(litellm_logging.py): add stricter check for special param being non none 2024-08-20 21:35:02 -07:00
Krrish Dholakia
0091f64ff1 fix(utils.py): ensure consistent cost calc b/w returned header and logged object 2024-08-20 19:01:20 -07:00
Krish Dholakia
e49e454929 Merge pull request #5287 from BerriAI/litellm_fix_response_cost_cal 
fix(cost_calculator.py): only override base model if custom pricing is set
 2024-08-20 11:42:48 -07:00
Ishaan Jaff
c82714757a Merge pull request #5288 from BerriAI/litellm_aporia_refactor 
[Feat] V2 aporia guardrails litellm
 2024-08-19 20:41:45 -07:00
Krrish Dholakia
cf1a1605a6 feat(cost_calculator.py): only override base model if custom pricing is set 2024-08-19 16:05:49 -07:00
Ishaan Jaff
6af497e383 feat run aporia as post call success hook 2024-08-19 11:25:31 -07:00
Krrish Dholakia
0d82089136 test(test_caching.py): re-introduce testing for s3 cache w/ streaming 
Closes https://github.com/BerriAI/litellm/issues/3268
 2024-08-19 10:56:48 -07:00
Krrish Dholakia
1856ac585d feat(pass_through_endpoints.py): add pass-through support for all cohere endpoints 2024-08-17 16:57:55 -07:00
Krrish Dholakia
29bedae79f feat(google_ai_studio_endpoints.py): support pass-through endpoint for all google ai studio requests 
New Feature
 2024-08-17 10:46:59 -07:00
Krish Dholakia
88fccb2427 Merge branch 'main' into litellm_log_model_price_information 2024-08-16 19:34:16 -07:00
Krish Dholakia
0916197c9d Merge pull request #5244 from BerriAI/litellm_better_error_logging_sentry 
refactor: replace .error() with .exception() logging for better debugging on sentry
 2024-08-16 19:16:20 -07:00
Ishaan Jaff
937471223a fix databricks streaming test 2024-08-16 16:56:08 -07:00
Krrish Dholakia
9609505d0c fix(litellm_logging.py): fix price information logging to s3 2024-08-16 16:42:38 -07:00
Krrish Dholakia
ef51f8600d feat(litellm_logging.py): support logging model price information to s3 logs 2024-08-16 16:21:34 -07:00
Krrish Dholakia
2874b94fb1 refactor: replace .error() with .exception() logging for better debugging on sentry 2024-08-16 09:22:47 -07:00
Krrish Dholakia
b381dec0a8 fix(litellm_logging.py): wrap function to safely fail 2024-08-15 18:05:06 -07:00
Krrish Dholakia
c0448b9641 feat(litellm_logging.py): cleanup payload + add response cost to logged payload 2024-08-15 17:53:25 -07:00
Krrish Dholakia
cf87c64348 fix(litellm_logging.py): fix standard payload 2024-08-15 17:33:40 -07:00
Krrish Dholakia
b08492bc29 fix(s3.py): fix s3 logging payload to have valid json values 
Previously pydantic objects were being stringified, making them unparsable
 2024-08-15 17:09:02 -07:00