David Manouchehri
6375affeea
(gemini): Fix Cloudflare AI Gateway typo. ( #5429 )
2024-09-02 07:52:18 -07:00
Simon S. Viloria
1c9a82771a
fix response_format={'type': 'json_object'} not working for Azure models ( #5468 )
2024-09-01 13:31:13 -07:00
Krish Dholakia
e474c3665a
Bedrock Embeddings refactor + model support ( #5462 )
...
* refactor(bedrock): initial commit to refactor bedrock to a folder
Improve code readability + maintainability
* refactor: more refactor work
* fix: fix imports
* feat(bedrock/embeddings.py): support translating embedding into amazon embedding formats
* fix: fix linting errors
* test: skip test on end of life model
* fix(cohere/embed.py): fix linting error
* fix(cohere/embed.py): fix typing
* fix(cohere/embed.py): fix post-call logging for cohere embedding call
* test(test_embeddings.py): fix error message assertion in test
2024-09-01 13:29:58 -07:00
Krish Dholakia
e12bd3e548
Minor LiteLLM Fixes and Improvements ( #5456 )
...
* fix(utils.py): support 'drop_params' for embedding requests
Fixes https://github.com/BerriAI/litellm/issues/5444
* feat(vertex_ai_non_gemini.py): support function param in messages
* test: skip test - model end of life
* fix(vertex_ai_non_gemini.py): fix gemini history parsing
2024-08-31 17:58:10 -07:00
Krish Dholakia
aa9f1896c6
anthropic prompt caching cost tracking ( #5453 )
...
* fix(utils.py): support 'drop_params' for embedding requests
Fixes https://github.com/BerriAI/litellm/issues/5444
* feat(anthropic/cost_calculation.py): Support calculating cost for prompt caching on anthropic
* feat(types/utils.py): allows us to migrate to openai's equivalent, once that comes out
* fix: fix linting errors
* test: mark flaky test
2024-08-31 14:50:12 -07:00
Ishaan Jaff
ccbbb4f3f8
add cerebras config
2024-08-31 08:34:46 -07:00
Ishaan Jaff
2202a02aef
Merge pull request #5455 from BerriAI/litellm_vtx_add_input_type_mapping
...
[Feat] Vertex embeddings - map `input_type` to `text_type`
2024-08-30 17:03:04 -07:00
Ishaan Jaff
9f9b6e81a7
add VertexAITextEmbeddingConfig
2024-08-30 12:53:43 -07:00
Ishaan Jaff
ea12519b98
fix map input_type to task_type for vertex ai
2024-08-30 12:09:07 -07:00
Ishaan Jaff
c40e564779
fix dir structure for tts
2024-08-30 11:44:23 -07:00
Ishaan Jaff
85b376adcc
Merge pull request #5449 from BerriAI/litellm_Fix_vertex_multimodal
...
[Fix-Proxy] Allow running /health checks on vertex multimodal embedding requests
2024-08-30 10:21:42 -07:00
Ishaan Jaff
10771e3bde
add test for test_vertexai_multimodal_embedding_text_input
2024-08-30 09:19:48 -07:00
Krish Dholakia
321b0961b5
fix: Minor LiteLLM Fixes + Improvements (29/08/2024) ( #5436 )
...
* fix(model_checks.py): support returning wildcard models on `/v1/models`
Fixes https://github.com/BerriAI/litellm/issues/4903
* fix(bedrock_httpx.py): support calling bedrock via api_base
Closes https://github.com/BerriAI/litellm/pull/4587
* fix(litellm_logging.py): only leave last 4 char of gemini key unmasked
Fixes https://github.com/BerriAI/litellm/issues/5433
* feat(router.py): support setting 'weight' param for models on router
Closes https://github.com/BerriAI/litellm/issues/5410
* test(test_bedrock_completion.py): add unit test for custom api base
* fix(model_checks.py): handle no "/" in model
2024-08-29 22:40:25 -07:00
David Manouchehri
f91232e130
(bedrock): Add new cross-region inference support for Bedrock.
2024-08-29 17:49:16 +00:00
Krish Dholakia
20fcd8ebdf
fix(google_ai_studio): working context caching ( #5421 )
...
* fix(google_ai_studio): working context caching
* feat(vertex_ai_context_caching.py): support async cache check calls
* fix(vertex_and_google_ai_studio_gemini.py): fix setting headers
* fix(vertex_ai_parter_models): fix import
* fix(vertex_and_google_ai_studio_gemini.py): fix input
* test(test_amazing_vertex_completion.py): fix test
2024-08-29 07:00:30 -07:00
Krrish Dholakia
1a2459bfe8
fix(vertex_ai_partner_models.py): fix vertex import
2024-08-28 18:08:33 -07:00
Krish Dholakia
0e0db9a5af
Merge branch 'main' into litellm_main_staging
2024-08-28 18:05:27 -07:00
Krish Dholakia
996c028127
Merge pull request #5393 from BerriAI/litellm_gemini_embedding_support
...
feat(vertex_ai_and_google_ai_studio): Support Google AI Studio Embedding Endpoint
2024-08-28 13:46:28 -07:00
Ishaan Jaff
29536630ca
update validate_vertex_input
2024-08-28 12:52:26 -07:00
Ishaan Jaff
8b9a015b4f
add ssml input on vertex tts
2024-08-28 12:17:53 -07:00
Krrish Dholakia
7a9f1798ff
fix(main.py): simplify to just use /batchEmbedContent
2024-08-27 21:46:05 -07:00
Krrish Dholakia
57330d2d0d
feat(batch_embed_content_transformation.py): support google ai studio /batchEmbedContent endpoint
...
Allows for multiple strings to be given for embedding
2024-08-27 19:23:50 -07:00
Krrish Dholakia
4bb59b7b2c
feat(embeddings_handler.py): support async gemini embeddings
2024-08-27 18:31:57 -07:00
Ishaan Jaff
a80b2aebbb
add test for rerank on custom api base
2024-08-27 18:25:51 -07:00
Krrish Dholakia
6a483a1908
fix(embeddings_handler.py): initial working commit for google ai studio text embeddings /embedContent endpoint
2024-08-27 18:14:56 -07:00
Krrish Dholakia
882ae8ac2c
fix: initial commit
2024-08-27 17:35:56 -07:00
Ishaan Jaff
001882c14e
Merge pull request #5392 from BerriAI/litellm_add_native_cohere_rerank
...
[Feat] Add cohere rerank and together ai rerank
2024-08-27 17:29:37 -07:00
Ishaan Jaff
4683504385
fix install on 3.8
2024-08-27 17:09:16 -07:00
Krrish Dholakia
c385383f78
fix(openai.py): fix error re-raising
2024-08-27 17:06:25 -07:00
Ishaan Jaff
1be953b565
add async support for rerank
2024-08-27 17:02:48 -07:00
Krrish Dholakia
de5617ce66
feat(vertex_ai_and_google_ai_studio): Support Google AI Studio Embeddings endpoint
...
Closes https://github.com/BerriAI/litellm/issues/5385
2024-08-27 16:53:11 -07:00
Ishaan Jaff
2aa119864a
add rerank params
2024-08-27 16:45:39 -07:00
Ishaan Jaff
70db82a236
add tg ai rerank support
2024-08-27 16:25:54 -07:00
Krrish Dholakia
722ccba323
fix(bedrock_httpx.py): support 'Auth' header as extra_header
...
Fixes https://github.com/BerriAI/litellm/issues/5389#issuecomment-2313677977
2024-08-27 16:08:54 -07:00
Krrish Dholakia
faf04985d6
fix(azure_text.py): fix streaming parsing
2024-08-27 15:52:55 -07:00
Ishaan Jaff
1b1de273fd
Merge pull request #5391 from BerriAI/litellm_add_ai21_support
...
[Feat] Add Vertex AI21 support
2024-08-27 15:06:26 -07:00
Ishaan Jaff
2436dfbbc0
add mock test for ai21
2024-08-27 14:42:13 -07:00
Krrish Dholakia
adcf1fc53d
fix(openai.py): fix post call error logging for aembedding calls
2024-08-27 14:26:06 -07:00
Krrish Dholakia
fedc83784f
fix(anthropic.py): support setting cache control headers, automatically
...
Don't require user to manually pass in 'extra_headers' for anthropic cache control usage
2024-08-27 13:57:03 -07:00
Krrish Dholakia
0088aae02f
fix(azure.py): fix raw response dump
2024-08-27 13:44:38 -07:00
Ishaan Jaff
5f61539e90
refactor partner models to include ai21
2024-08-27 13:35:22 -07:00
Krrish Dholakia
b1289b0d19
fix: fix linting errors
2024-08-27 12:14:23 -07:00
Krish Dholakia
34eb1206c6
Merge pull request #5358 from BerriAI/litellm_fix_retry_after
...
fix retry after - cooldown individual models based on their specific 'retry-after' header
2024-08-27 11:50:14 -07:00
Krrish Dholakia
30bf8a4c1d
test: fix test
2024-08-27 10:46:57 -07:00
Krrish Dholakia
f28b5934fc
fix(sagemaker.py): fix streaming logic
2024-08-27 08:10:47 -07:00
Krrish Dholakia
07dd3c640b
perf(sagemaker.py): asyncify hf prompt template check
...
leads to 189% improvement in RPS @ 100 users
2024-08-27 07:37:06 -07:00
miraclebakelaser
0c14b982f4
fix(factory.py): handle missing 'content' in cohere assistant messages
...
Update cohere_messages_pt_v2 function to check for 'content' existence
2024-08-27 19:38:37 +09:00
Krish Dholakia
144309c65a
Merge branch 'main' into litellm_gemini_context_caching
2024-08-26 22:22:17 -07:00
Krrish Dholakia
fe6ba2d025
fix: fix imports
2024-08-26 22:19:01 -07:00
Krrish Dholakia
d3f9731697
fix: fix unbound var
2024-08-26 22:19:01 -07:00
