Commit graph

13440 commits

Author SHA1 Message Date
Krrish Dholakia
5f87dc229a feat(openai.py): bubble all error information back to client 2025-03-10 15:27:43 -07:00
Krrish Dholakia
c1ec82fbd5 refactor: instrument body param to bubble up on exception 2025-03-10 15:21:04 -07:00
Ishaan Jaff
c1a3cb82a9 docs on contributing 2025-03-10 14:49:27 -07:00
Ishaan Jaff
0fcce63852
Merge pull request #9032 from themrzmaster/feat/jamba_1.6
pricing for jamba new models
2025-03-10 13:59:37 -07:00
Ishaan Jaff
7319fef29d fix linting error 2025-03-10 13:57:50 -07:00
Ishaan Jaff
05ad7a67a7 Revert "ui new build"
This reverts commit 34694d3057.
2025-03-10 13:56:10 -07:00
Ishaan Jaff
34694d3057 ui new build 2025-03-10 12:32:18 -07:00
Ishaan Jaff
666690c31c fix atext_completion 2025-03-10 10:18:03 -07:00
omrishiv
0674491386
add support for Amazon Nova Canvas model (#7838)
* add initial support for Amazon Nova Canvas model

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* adjust name to AmazonNovaCanvas and map function variables to config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* tighten model name check

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* fix quality mapping

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add premium quality in config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* support all Amazon Nova Canvas tasks

* remove unused import

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add tests for image generation tasks and fix payload

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add missing util file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* update model prices backup file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* remove image tasks other than text->image

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

---------

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-03-10 08:02:00 -07:00
vivek-athina
cd4a53d6f2
Merge pull request #4 from BerriAI/main
Update main
2025-03-10 11:13:21 +05:30
Krrish Dholakia
574f5056c8 fix(utils.py): fix linting error 2025-03-09 20:47:12 -07:00
Krish Dholakia
f899b828cf
Support openrouter reasoning_content on streaming (#9094)
* feat(convert_dict_to_response.py): support openrouter format of reasoning content

* fix(transformation.py): fix openrouter streaming with reasoning content

Fixes https://github.com/BerriAI/litellm/issues/8193#issuecomment-270892962

* fix: fix type error
2025-03-09 20:03:59 -07:00
5aaee9
42b7921ca1
fix: perplexity return both delta and message cause OpenWebUI repect text (#9081) 2025-03-09 19:46:31 -07:00
Krish Dholakia
65ef65d360
feat: prioritize api_key over tenant_id for more Azure AD token provi… (#8701)
* feat: prioritize api_key over tenant_id for more Azure AD token provider (#8318)

* fix: prioritize api_key over tenant_id for Azure AD token provider

* test: Add test for Azure AD token provider in router

* fix: fix linting error

---------

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2025-03-09 18:59:37 -07:00
Krrish Dholakia
2c5b2da955 fix: make type object subscriptable 2025-03-09 18:35:10 -07:00
Krish Dholakia
e00d4fb18c
Litellm dev 03 08 2025 p3 (#9089)
* feat(ollama_chat.py): pass down http client to ollama_chat

enables easier testing

* fix(factory.py): fix passing images to ollama's `/api/generate` endpoint

Fixes https://github.com/BerriAI/litellm/issues/6683

* fix(factory.py): fix ollama pt to handle templating correctly
2025-03-09 18:20:56 -07:00
Ishaan Jaff
b6eee01381 Revert "experimental - track anthropic messages as mode"
This reverts commit 22b3862e0d.
2025-03-08 17:38:24 -08:00
Ishaan Jaff
22b3862e0d experimental - track anthropic messages as mode 2025-03-08 17:33:35 -08:00
Ishaan Jaff
b41311bb21
(UI) - Fix show correct count of internal user keys on Users Page (#9082)
* get_user_key_counts

* fix get_user_key_counts

* fix get_user_key_counts

* test_get_users_filters_dashboard_keys

* remove unused func
2025-03-08 16:13:18 -08:00
Ishaan Jaff
73df319f4e
(Clean up) - Allow switching off storing Error Logs in DB (#9084)
* fix - cleanup, dont store ErrorLogs in 2 tables

* async_post_call_failure_hook

* docs disable error logs

* disable_error_logs
2025-03-08 16:12:03 -08:00
Krish Dholakia
4330ef8e81
Fix batches api cost tracking + Log batch models in spend logs / standard logging payload (#9077)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 42s
* feat(batches/): fix batch cost calculation - ensure it's accurate

use the correct cost value - prev. defaulting to non-batch cost

* feat(batch_utils.py): log batch models to spend logs + standard logging payload

makes it easy to understand how cost was calculated

* fix: fix stored payload for test

* test: fix test
2025-03-08 11:47:25 -08:00
Teja Vishwanadha
8c049dfffc
support bytes.IO for audio transcription (#9071) 2025-03-08 08:47:15 -08:00
Ishaan Jaff
e2d612efd9
Bug fix - String data: stripped from entire content in streamed Gemini responses (#9070)
* _strip_sse_data_from_chunk

* use _strip_sse_data_from_chunk

* use _strip_sse_data_from_chunk

* use _strip_sse_data_from_chunk

* _strip_sse_data_from_chunk

* test_strip_sse_data_from_chunk

* _strip_sse_data_from_chunk

* testing

* _strip_sse_data_from_chunk
2025-03-07 21:06:39 -08:00
Krish Dholakia
0e3caf92b9
UI - new API Playground for testing LiteLLM translation (#9073)
* feat: initial commit - enable dev to see translated request

* feat(utils.py): expose new endpoint - `/utils/transform_request` to see the raw request sent by litellm

* feat(transform_request.tsx): allow user to see their transformed request

* refactor(litellm_logging.py): return raw request in 3 parts - api_base, headers, request body

easier to render each individually on UI vs. extracting from combined string

* feat: transform_request.tsx

working e2e raw request viewing

* fix(litellm_logging.py): fix transform viewing for bedrock models

* fix(litellm_logging.py): don't return sensitive headers in raw request headers

prevent accidental leak

* feat(transform_request.tsx): style improvements
2025-03-07 19:39:31 -08:00
Ishaan Jaff
b5eeafdd72
(Docs) OpenWeb x LiteLLM Docker compose + Instructions on spend tracking + logging (#9059)
* docs improve open web ui litellm doc

* docs openweb show teams + keys

* docs open web ui litellm
2025-03-07 17:01:39 -08:00
Krrish Dholakia
36f3276d8c docs: update docs
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 16s
2025-03-07 11:00:12 -08:00
Ishaan Jaff
7f70bdd99b
(Feat) - add pricing for eu.amazon.nova models (#9056)
* add pricing for eu.amazon.nova models

* fix typo in key management endpoints.py
2025-03-07 07:06:17 -08:00
Krish Dholakia
5591354309
Support master key rotations (#9041)
* feat(key_management_endpoints.py): adding support for rotating master key

* feat(key_management_endpoints.py): support decryption-re-encryption of models in db, when master key rotated

* fix(user_api_key_auth.py): raise valid token is None error earlier

enables easier debugging with api key hash in error message

* feat(key_management_endpoints.py): rotate any env vars

* fix(key_management_endpoints.py): uncomment check

* fix: fix linting error
2025-03-06 23:13:30 -08:00
Krrish Dholakia
805679becc feat(handle_jwt.py): support multiple jwt url's 2025-03-06 23:05:54 -08:00
Krish Dholakia
52de1949ef
build(model_prices_and_context_window.json): update azure o1 mini pricing (#9046) 2025-03-06 22:28:06 -08:00
Krish Dholakia
274147bc5e
fix(team_endpoints.py): ensure 404 raised when team not found (#9038)
* fix(team_endpoints.py): ensure 404 raised when team not found

* fix(key_management_endpoints.py): fix adding tags to key when metadata is empty

* fix(key_management_endpoints.py): refactor set metadata field to use common function across keys + teams

reduces scope for errors + easier testing

* fix: fix linting error
2025-03-06 22:04:36 -08:00
Ishaan Jaff
0fed8bcefd ui new build 2025-03-06 21:22:58 -08:00
Ishaan Jaff
73448412e1
ui allow ui or eu api base adding model (#9042) 2025-03-06 21:22:03 -08:00
Ishaan Jaff
2a377b161d _create_redis_cache 2025-03-06 21:15:48 -08:00
Ishaan Jaff
958e71b906
(Docs) connect litellm to open web ui (#9040)
* init doc

* working thinking tutorial

* docs open web ui with litellm

* minor edits

* docs one tab for tutorials
2025-03-06 21:13:00 -08:00
Ishaan Jaff
04e839d846
(AWS Secret Manager) - Using K/V pairs in 1 AWS Secret (#9039)
* fixes for primary_secret_kv_pairs

* _parse_primary_secret

* Using K/V pairs in 1 AWS Secret

* test_primary_secret_functionality
2025-03-06 19:30:18 -08:00
Ishaan Jaff
b02af305de
[Feat] - Display thinking tokens on OpenWebUI (Bedrock, Anthropic, Deepseek) (#9029)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 14s
* if merge_reasoning_content_in_choices

* _optional_combine_thinking_block_in_choices

* stash changes

* working merge_reasoning_content_in_choices with bedrock

* fix litellm_params accessor

* fix streaming handler

* merge_reasoning_content_in_choices

* _optional_combine_thinking_block_in_choices

* test_bedrock_stream_thinking_content_openwebui

* merge_reasoning_content_in_choices

* fix for _optional_combine_thinking_block_in_choices

* linting error fix
2025-03-06 18:32:58 -08:00
Ogun Oz
85d1427710
Fix: Create RedisClusterCache when startup nodes provided in cache args of router (#9010)
Co-authored-by: Ogün Öz <ogun.oz@cobrainer.com>
2025-03-06 17:14:32 -08:00
lucca
af9f85e0d9
fix 2025-03-06 15:57:29 -03:00
lucca
9dee3e2e3f
pricing 2025-03-06 13:28:07 -03:00
Ishaan Jaff
f47987e673
(Refactor) /v1/messages to follow simpler logic for Anthropic API spec (#9013)
* anthropic_messages_handler v0

* fix /messages

* working messages with router methods

* test_anthropic_messages_handler_litellm_router_non_streaming

* test_anthropic_messages_litellm_router_non_streaming_with_logging

* AnthropicMessagesConfig

* _handle_anthropic_messages_response_logging

* working with /v1/messages endpoint

* working /v1/messages endpoint

* refactor to use router factory function

* use aanthropic_messages

* use BaseConfig for Anthropic /v1/messages

* track api key, team on /v1/messages endpoint

* fix get_logging_payload

* BaseAnthropicMessagesTest

* align test config

* test_anthropic_messages_with_thinking

* test_anthropic_streaming_with_thinking

* fix - display anthropic url for debugging

* test_bad_request_error_handling

* test_anthropic_messages_router_streaming_with_bad_request

* fix ProxyException

* test_bad_request_error_handling_streaming

* use provider_specific_header

* test_anthropic_messages_with_extra_headers

* test_anthropic_messages_to_wildcard_model

* fix gcs pub sub test

* standard_logging_payload

* fix unit testing for anthopic /v1/messages support

* fix pass through anthropic messages api

* delete dead code

* fix anthropic pass through response

* revert change to spend tracking utils

* fix get_litellm_metadata_from_kwargs

* fix spend logs payload json

* proxy_pass_through_endpoint_tests

* TestAnthropicPassthroughBasic

* fix pass through tests

* test_async_vertex_proxy_route_api_key_auth

* _handle_anthropic_messages_response_logging

* vertex_credentials

* test_set_default_vertex_config

* test_anthropic_messages_litellm_router_non_streaming_with_logging

* test_ageneric_api_call_with_fallbacks_basic

* test__aadapter_completion
2025-03-06 00:43:08 -08:00
Krish Dholakia
744e10b0f0
Litellm dev 03 05 2025 p3 (#9023)
* fix(invoke_handler.py): fix converse streaming - return signature + ensure consistency with anthropic api response

* build(model_prices_and_context_window.json): fix anthropic api claude-3-7 max output tokens

with beta header this is 128k

Resolves https://github.com/BerriAI/litellm/issues/8964

* feat(handler.py): handle new anthropic 'thinking_delta' block on streaming

Fixes https://github.com/BerriAI/litellm/issues/8825
2025-03-05 22:31:39 -08:00
Krish Dholakia
f6535ae6ad
Support format param for specifying image type (#9019)
* fix(transformation.py): support a 'format' parameter for image's

allow user to specify mime type

* fix: pass mimetype via 'format' param

* feat(gemini/chat/transformation.py): support 'format' param for gemini

* fix(factory.py): support 'format' param on sync bedrock converse calls

* feat(bedrock/converse_transformation.py): support 'format' param for bedrock async calls

* refactor(factory.py): move to supporting 'format' param in base helper

ensures consistency in param support

* feat(gpt_transformation.py): filter out 'format' param

don't send invalid param to openai

* fix(gpt_transformation.py): fix translation

* fix: fix translation error
2025-03-05 19:52:53 -08:00
Krish Dholakia
ec4f665e29
Return signature on anthropic streaming + migrate to signature field instead of signature_delta [MINOR bump] (#9021)
* Fix missing signature_delta in thinking blocks when streaming from Claude 3.7 (#8797)

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* test: update test to enforce signature found

* feat(refactor-signature-param-to-be-'signature'-instead-of-'signature_delta'): keeps it in sync with anthropic

* fix: fix linting error

---------

Co-authored-by: Martin Krasser <krasserm@googlemail.com>
2025-03-05 19:33:54 -08:00
Sunny Wan
a2fed4059e added Snowflake config to ProviderConfigManager 2025-03-05 20:32:18 -05:00
Krrish Dholakia
3be3b802c8 fix: fix linting error 2025-03-05 10:10:53 -08:00
Ishaan Jaff
8d6815ce98 Revert "(UI) - Security Improvement, move to JWT Auth for Admin UI Sessions (#8995)"
This reverts commit 01a44a4e47.
2025-03-05 08:49:20 -08:00
Krrish Dholakia
313b315791 fix: fix linting error 2025-03-05 08:26:26 -08:00
Krish Dholakia
c69ec66dc5
fix(base_aws_llm.py): remove region name before sending in args (#8998)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 12s
* fix(base_aws_llm.py): remove region name before sending in args

* fix(base_aws_llm.py): fix optional param pop position

* fix: fix linting error
2025-03-04 23:05:28 -08:00
Krrish Dholakia
e715c376b9 fix: fix linting error 2025-03-04 22:34:07 -08:00