phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-26 19:24:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
17240 commits 2490 branches 894 tags 548 MiB
Commit graph

271 commits

Author SHA1 Message Date
Krish Dholakia
ca4e746545 LiteLLM minor fixes + improvements (31/08/2024) (#5464) 
* fix(vertex_endpoints.py): fix vertex ai pass through endpoints

* test(test_streaming.py): skip model due to end of life

* feat(custom_logger.py): add special callback for model hitting tpm/rpm limits

Closes https://github.com/BerriAI/litellm/issues/4096
 2024-09-01 13:31:42 -07:00
Krish Dholakia
e474c3665a Bedrock Embeddings refactor + model support (#5462) 
* refactor(bedrock): initial commit to refactor bedrock to a folder

Improve code readability + maintainability

* refactor: more refactor work

* fix: fix imports

* feat(bedrock/embeddings.py): support translating embedding into amazon embedding formats

* fix: fix linting errors

* test: skip test on end of life model

* fix(cohere/embed.py): fix linting error

* fix(cohere/embed.py): fix typing

* fix(cohere/embed.py): fix post-call logging for cohere embedding call

* test(test_embeddings.py): fix error message assertion in test
 2024-09-01 13:29:58 -07:00
Krish Dholakia
e12bd3e548 Minor LiteLLM Fixes and Improvements (#5456) 
* fix(utils.py): support 'drop_params' for embedding requests

Fixes https://github.com/BerriAI/litellm/issues/5444

* feat(vertex_ai_non_gemini.py): support function param in messages

* test: skip test - model end of life

* fix(vertex_ai_non_gemini.py): fix gemini history parsing
 2024-08-31 17:58:10 -07:00
Krish Dholakia
5f993f46a0 anthropic prompt caching cost tracking (#5453) 
* fix(utils.py): support 'drop_params' for embedding requests

Fixes https://github.com/BerriAI/litellm/issues/5444

* feat(anthropic/cost_calculation.py): Support calculating cost for prompt caching on anthropic

* feat(types/utils.py): allows us to migrate to openai's equivalent, once that comes out

* fix: fix linting errors

* test: mark flaky test
 2024-08-31 14:09:35 -07:00
Krish Dholakia
321b0961b5 fix: Minor LiteLLM Fixes + Improvements (29/08/2024) (#5436) 
* fix(model_checks.py): support returning wildcard models on `/v1/models`

Fixes https://github.com/BerriAI/litellm/issues/4903

* fix(bedrock_httpx.py): support calling bedrock via api_base

Closes https://github.com/BerriAI/litellm/pull/4587

* fix(litellm_logging.py): only leave last 4 char of gemini key unmasked

Fixes https://github.com/BerriAI/litellm/issues/5433

* feat(router.py): support setting 'weight' param for models on router

Closes https://github.com/BerriAI/litellm/issues/5410

* test(test_bedrock_completion.py): add unit test for custom api base

* fix(model_checks.py): handle no "/" in model
 2024-08-29 22:40:25 -07:00
Krish Dholakia
20fcd8ebdf fix(google_ai_studio): working context caching (#5421) 
* fix(google_ai_studio): working context caching

* feat(vertex_ai_context_caching.py): support async cache check calls

* fix(vertex_and_google_ai_studio_gemini.py): fix setting headers

* fix(vertex_ai_parter_models): fix import

* fix(vertex_and_google_ai_studio_gemini.py): fix input

* test(test_amazing_vertex_completion.py): fix test
 2024-08-29 07:00:30 -07:00
Krish Dholakia
0e0db9a5af Merge branch 'main' into litellm_main_staging 2024-08-28 18:05:27 -07:00
Krrish Dholakia
6e4f0a95da fix(router.py): fix cooldown check 2024-08-28 16:38:42 -07:00
Krrish Dholakia
57330d2d0d feat(batch_embed_content_transformation.py): support google ai studio /batchEmbedContent endpoint 
Allows for multiple strings to be given for embedding
 2024-08-27 19:23:50 -07:00
Krrish Dholakia
6a483a1908 fix(embeddings_handler.py): initial working commit for google ai studio text embeddings /embedContent endpoint 2024-08-27 18:14:56 -07:00
Krish Dholakia
34eb1206c6 Merge pull request #5358 from BerriAI/litellm_fix_retry_after 
fix retry after - cooldown individual models based on their specific 'retry-after' header
 2024-08-27 11:50:14 -07:00
Krrish Dholakia
074e30fa10 feat(vertex_ai_context_caching.py): support making context caching calls to vertex ai in a normal chat completion call (anthropic caching format) 
Closes https://github.com/BerriAI/litellm/issues/5213
 2024-08-26 22:19:01 -07:00
Krish Dholakia
6ff17f1acd Merge pull request #5368 from BerriAI/litellm_vertex_function_support 
feat(vertex_httpx.py): support 'functions' param for gemini google ai studio + vertex ai
 2024-08-26 22:11:42 -07:00
Krrish Dholakia
2c979758d1 fix(types/utils.py): map finish reason to openai compatible 2024-08-26 22:09:02 -07:00
Krrish Dholakia
b989762bb0 fix(sagemaker.py): support streaming for messages api 
Fixes https://github.com/BerriAI/litellm/issues/5372
 2024-08-26 15:08:08 -07:00
Krrish Dholakia
aedc6652d4 feat(vertex_httpx.py): support functions param for gemini google ai studio + vertex ai 
Closes https://github.com/BerriAI/litellm/issues/5344
 2024-08-26 10:59:01 -07:00
Krrish Dholakia
27a5cd12e0 fix(utils.py): correctly re-raise the headers from an exception, if present 
Fixes issue where retry after on router was not using azure / openai numbers
 2024-08-24 12:30:30 -07:00
Ishaan Jaff
2864d16fa1 Merge branch 'main' into litellm_allow_using_azure_ad_token_auth 2024-08-22 18:21:24 -07:00
Ishaan Jaff
2f01a22ef7 add bedrock guardrails support 2024-08-22 16:09:55 -07:00
Ishaan Jaff
5f1a03e897 add types for BedrockMessage 2024-08-22 15:40:58 -07:00
Ishaan Jaff
b16752f0bc add new litellm params for client_id, tenant_id etc 2024-08-22 11:37:30 -07:00
Krrish Dholakia
900d8ecbf0 feat(factory.py): enable 'user_continue_message' for interweaving user/assistant messages when provider requires it 
allows bedrock to be used with autogen
 2024-08-22 11:03:33 -07:00
Ishaan Jaff
26354fbb9d add azure_ad_token_provider as all litellm params 2024-08-22 10:59:18 -07:00
Krrish Dholakia
2dd616bad0 fix(ollama_chat.py): fix passing assistant message with tool call param 
Fixes https://github.com/BerriAI/litellm/issues/5319
 2024-08-22 10:00:03 -07:00
Ishaan Jaff
e4fe5924a5 Merge pull request #5326 from BerriAI/litellm_Add_vertex_multimodal_embedding 
[Feat] add vertex multimodal embedding support
 2024-08-21 17:06:43 -07:00
Krrish Dholakia
d6bc37374e feat(litellm_logging.py): add 'saved_cache_cost' to standard logging payload (s3) 2024-08-21 16:58:07 -07:00
Ishaan Jaff
710ae63957 add VertexMultimodalEmbeddingRequest type 2024-08-21 14:25:47 -07:00
Ishaan Jaff
2f0bd38f3a support lakera ai category thresholds 2024-08-20 17:19:24 -07:00
Ishaan Jaff
319690ab5e feat - guardrails v2 2024-08-19 18:24:20 -07:00
Krrish Dholakia
ef51f8600d feat(litellm_logging.py): support logging model price information to s3 logs 2024-08-16 16:21:34 -07:00
Ishaan Jaff
b82d120c47 add provider_specific_fields to GenericStreamingChunk 2024-08-16 11:38:22 -07:00
Krish Dholakia
ca07898fbb Merge pull request #5235 from BerriAI/litellm_fix_s3_logs 
fix(s3.py): fix s3 logging payload to have valid json values
 2024-08-15 23:00:18 -07:00
Ishaan Jaff
953a67ba4c refactor sagemaker to be async 2024-08-15 18:18:02 -07:00
Krrish Dholakia
c0448b9641 feat(litellm_logging.py): cleanup payload + add response cost to logged payload 2024-08-15 17:53:25 -07:00
Krrish Dholakia
cf87c64348 fix(litellm_logging.py): fix standard payload 2024-08-15 17:33:40 -07:00
Krrish Dholakia
b08492bc29 fix(s3.py): fix s3 logging payload to have valid json values 
Previously pydantic objects were being stringified, making them unparsable
 2024-08-15 17:09:02 -07:00
Ishaan Jaff
7fc2657a26 add test for large context in system message for anthropic 2024-08-14 17:03:10 -07:00
Ishaan Jaff
41ce2ef904 add anthropic cache controls 2024-08-14 14:56:49 -07:00
Krrish Dholakia
4e5d5354c2 build(model_prices_and_context_window.json): add 'supports_assistant_prefill' to model info map 
Closes https://github.com/BerriAI/litellm/issues/4881
 2024-08-10 14:15:12 -07:00
Krrish Dholakia
e2249961cb fix(types/utils.py): handle null completion tokens 
Fixes https://github.com/BerriAI/litellm/issues/5096
 2024-08-10 09:23:03 -07:00
Krrish Dholakia
091fe9af67 fix(router.py): fix types 2024-08-09 12:24:48 -07:00
Krrish Dholakia
482acc7ee1 fix(router.py): fallback on 400-status code requests 2024-08-09 12:16:49 -07:00
Krrish Dholakia
ec4051592b fix(anthropic.py): handle scenario where anthropic returns invalid json string for tool call while streaming 
Fixes https://github.com/BerriAI/litellm/issues/5063
 2024-08-07 09:24:11 -07:00
Krish Dholakia
0044fd0041 Merge branch 'main' into litellm_support_lakera_config_thresholds 2024-08-06 22:47:13 -07:00
Krrish Dholakia
2ec7cb4153 fix(utils.py): fix types 2024-08-06 12:23:22 -07:00
Krrish Dholakia
8500f6d087 feat(caching.py): enable caching on provider-specific optional params 
Closes https://github.com/BerriAI/litellm/issues/5049
 2024-08-05 11:18:59 -07:00
Krrish Dholakia
14d0ae6aa4 fix(types/router.py): remove model_info pydantic field 
Fixes https://github.com/BerriAI/litellm/issues/5042
 2024-08-05 09:58:44 -07:00
Krrish Dholakia
5810708c71 feat(anthropic_adapter.py): support streaming requests for /v1/messages endpoint 
Fixes https://github.com/BerriAI/litellm/issues/5011
 2024-08-03 20:16:19 -07:00
Krrish Dholakia
c1e5792ac4 fix(types/utils.py): fix linting errors 2024-08-03 11:48:33 -07:00
Krrish Dholakia
147ecc635e fix(bedrock.py): fix response format for bedrock image generation response 
Fixes https://github.com/BerriAI/litellm/issues/5010
 2024-08-03 09:46:49 -07:00