Krish Dholakia
7cc12bd5c6
LiteLLM Minor Fixes & Improvements (10/18/2024) ( #6320 )
...
* fix(converse_transformation.py): handle cross region model name when getting openai param support
Fixes https://github.com/BerriAI/litellm/issues/6291
* LiteLLM Minor Fixes & Improvements (10/17/2024) (#6293 )
* fix(ui_sso.py): fix faulty admin only check
Fixes https://github.com/BerriAI/litellm/issues/6286
* refactor(sso_helper_utils.py): refactor /sso/callback to use helper utils, covered by unit testing
Prevent future regressions
* feat(prompt_factory): support 'ensure_alternating_roles' param
Closes https://github.com/BerriAI/litellm/issues/6257
* fix(proxy/utils.py): add dailytagspend to expected views
* feat(auth_utils.py): support setting regex for clientside auth credentials
Fixes https://github.com/BerriAI/litellm/issues/6203
* build(cookbook): add tutorial for mlflow + langchain + litellm proxy tracing
* feat(argilla.py): add argilla logging integration
Closes https://github.com/BerriAI/litellm/issues/6201
* fix: fix linting errors
* fix: fix ruff error
* test: fix test
* fix: update vertex ai assumption - parts not always guaranteed (#6296 )
* docs(configs.md): add argila env var to docs
* docs(user_keys.md): add regex doc for clientside auth params
* docs(argilla.md): add doc on argilla logging
* docs(argilla.md): add sampling rate to argilla calls
* bump: version 1.49.6 → 1.49.7
* add gpt-4o-audio models to model cost map (#6306 )
* (code quality) add ruff check PLR0915 for `too-many-statements` (#6309 )
* ruff add PLR0915
* add noqa for PLR0915
* fix noqa
* add # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* add # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* doc fix Turn on / off caching per Key. (#6297 )
* (feat) Support `audio`, `modalities` params (#6304 )
* add audio, modalities param
* add test for gpt audio models
* add get_supported_openai_params for GPT audio models
* add supported params for audio
* test_audio_output_from_model
* bump openai to openai==1.52.0
* bump openai on pyproject
* fix audio test
* fix test mock_chat_response
* handle audio for Message
* fix handling audio for OAI compatible API endpoints
* fix linting
* fix mock dbrx test
* (feat) Support audio param in responses streaming (#6312 )
* add audio, modalities param
* add test for gpt audio models
* add get_supported_openai_params for GPT audio models
* add supported params for audio
* test_audio_output_from_model
* bump openai to openai==1.52.0
* bump openai on pyproject
* fix audio test
* fix test mock_chat_response
* handle audio for Message
* fix handling audio for OAI compatible API endpoints
* fix linting
* fix mock dbrx test
* add audio to Delta
* handle model_response.choices.delta.audio
* fix linting
* build(model_prices_and_context_window.json): add gpt-4o-audio audio token cost tracking
* refactor(model_prices_and_context_window.json): refactor 'supports_audio' to be 'supports_audio_input' and 'supports_audio_output'
Allows for flag to be used for openai + gemini models (both support audio input)
* feat(cost_calculation.py): support cost calc for audio model
Closes https://github.com/BerriAI/litellm/issues/6302
* feat(utils.py): expose new `supports_audio_input` and `supports_audio_output` functions
Closes https://github.com/BerriAI/litellm/issues/6303
* feat(handle_jwt.py): support single dict list
* fix(cost_calculator.py): fix linting errors
* fix: fix linting error
* fix(cost_calculator): move to using standard openai usage cached tokens value
* test: fix test
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2024-10-19 22:23:27 -07:00
Ishaan Jaff
4d1b4beb3d
(refactor) caching use LLMCachingHandler for async_get_cache and set_cache ( #6208 )
...
* use folder for caching
* fix importing caching
* fix clickhouse pyright
* fix linting
* fix correctly pass kwargs and args
* fix test case for embedding
* fix linting
* fix embedding caching logic
* fix refactor handle utils.py
* fix test_embedding_caching_azure_individual_items_reordered
2024-10-14 16:34:01 +05:30
Krish Dholakia
98c34a7e27
LiteLLM Minor Fixes and Improvements (11/09/2024) ( #5634 )
...
* fix(caching.py): set ttl for async_increment cache
fixes issue where ttl for redis client was not being set on increment_cache
Fixes https://github.com/BerriAI/litellm/issues/5609
* fix(caching.py): fix increment cache w/ ttl for sync increment cache on redis
Fixes https://github.com/BerriAI/litellm/issues/5609
* fix(router.py): support adding retry policy + allowed fails policy via config.yaml
* fix(router.py): don't cooldown single deployments
No point, as there's no other deployment to loadbalance with.
* fix(user_api_key_auth.py): support setting allowed email domains on jwt tokens
Closes https://github.com/BerriAI/litellm/issues/5605
* docs(token_auth.md): add user upsert + allowed email domain to jwt auth docs
* fix(litellm_pre_call_utils.py): fix dynamic key logging when team id is set
Fixes issue where key logging would not be set if team metadata was not none
* fix(secret_managers/main.py): load environment variables correctly
Fixes issue where os.environ/ was not being loaded correctly
* test(test_router.py): fix test
* feat(spend_tracking_utils.py): support logging additional usage params - e.g. prompt caching values for deepseek
* test: fix tests
* test: fix test
* test: fix test
* test: fix test
* test: fix test
2024-09-11 22:36:06 -07:00
Krrish Dholakia
2719860c46
build(model_prices_and_context_window.json): add mistral-large on vertex ai pricing
2024-07-27 10:37:18 -07:00
Krrish Dholakia
ccbc7cfb9f
fix(team_endpoints.py): check if key belongs to team before returning /team/info
2024-07-16 21:17:05 -07:00
Krrish Dholakia
a33b9221da
fix(proxy_server.py): fix admin allowed routes
2024-05-22 20:55:21 -07:00
Krrish Dholakia
600b6f7e1d
feat(proxy_server.py): support 'user_id_upsert' flag for jwt_auth
2024-05-15 22:19:59 -07:00
Krrish Dholakia
99653d2d3e
feat(handle_jwt.py): add support for 'team_id_default
...
allows admin to set a default team id for spend-tracking + permissions
2024-05-15 21:33:35 -07:00
Krrish Dholakia
f48cd87cf3
feat(proxy_server.py): make team_id optional for jwt token auth (only enforced, if set)
...
Allows users to use jwt auth for internal chat apps
2024-05-15 21:05:14 -07:00
Jean-Luc Duckworth
d60aa8282e
Fixed typo. test_jwt.py tests pass
2024-05-07 16:08:36 -04:00
Jean-Luc Duckworth
d5767e9403
Expanding jwt access to other RS and PS algos. Updated to resolve merge conflicts.
2024-05-07 15:45:07 -04:00
Sara Ghaemi
66b2b5fab9
made audience optional and updated docs
2024-05-07 11:37:04 -04:00
Sara Ghaemi
1580f993fe
Updated JWT handler to support PEM public key
2024-05-07 11:22:17 -04:00
Krrish Dholakia
f4c7f4f901
fix(proxy_server.py): support tracking org spend
...
currently works when org set for jwt auth
2024-04-11 23:01:21 -07:00
Krrish Dholakia
36ff593c02
feat(proxy_server.py): support cost tracking on user id via JWT-Auth
...
allows admin to track cost for LiteLLM_UserTable via JWT
2024-04-11 18:47:46 -07:00
Krrish Dholakia
e413191493
fix(handle_jwt.py): support firebase public key format
...
adds support for when the public keys are a json instead of nested inside a json
2024-04-11 18:07:13 -07:00
Krrish Dholakia
68a1fa36cf
fix(handle_jwt.py): fix linting issue
2024-03-29 21:35:36 -07:00
phuongnd96
4bf4be3a0f
fix: still verify kid on single pub key fetched
2024-03-30 10:09:38 +07:00
Krrish Dholakia
1e856443e1
feat(proxy/utils.py): enable updating db in a separate server
2024-03-27 16:02:36 -07:00
Krrish Dholakia
752516df1b
fix(handle_jwt.py): support public key caching ttl param
2024-03-26 14:32:55 -07:00
Krrish Dholakia
d90f44fe8e
fix(proxy_server.py): check if team scope in jwt
2024-03-26 14:01:02 -07:00
Krrish Dholakia
7d38c62717
fix(handle_jwt.py): enable team-based jwt-auth access
...
Move auth to check on ‘client_id’ not ‘sub
2024-03-26 12:25:38 -07:00
Krish Dholakia
f15ba10170
Merge pull request #2687 from BerriAI/litellm_jwt_auth_fixes_2
...
Litellm jwt auth fixes
2024-03-25 13:27:19 -07:00
Krrish Dholakia
2e4e97a48f
test(test_jwt.py): add unit tests for jwt auth integration
2024-03-25 13:24:39 -07:00
Krrish Dholakia
c2ffb83c71
fix(handle_jwt.py): cache public keys
...
caches jwt public keys - reducing need for making http calls on every request
2024-03-25 12:36:32 -07:00
Krrish Dholakia
f1b1640a31
fix(handle_jwt.py): support scopes being a list
...
allow scopes in jwt to be a list, not just a space-separated string
2024-03-25 12:30:40 -07:00
Krrish Dholakia
5aa1c10929
fix(handle_jwt.py): don't require kid to be set
2024-03-25 12:28:16 -07:00
Krrish Dholakia
93959ab5aa
fix(handle_jwt.py): allow setting proxy admin role string for jwt auth
2024-03-25 12:20:14 -07:00
Krrish Dholakia
edd00af6f2
fix(handle_jwt.py): enable user to set jwt admin scope string in config
2024-03-25 11:34:49 -07:00
Ishaan Jaff
5d121a9f3c
(fix) stop using f strings with logger
2024-03-25 10:47:18 -07:00
Krrish Dholakia
9bf086386e
fix(handle_jwt.py): add more logging for jwt header
2024-03-22 16:33:32 -07:00
Krrish Dholakia
90e17b5422
fix(handle_jwt.py): track spend for user using jwt auth
2024-03-20 10:55:52 -07:00
Krrish Dholakia
ca970a90c4
fix(handle_jwt.py): remove issuer check
2024-03-20 08:35:23 -07:00
Krrish Dholakia
1c68a7fd1d
refactor(handle_jwt.py): print jwt version
2024-03-19 17:17:11 -07:00
Krrish Dholakia
8c91156842
build: build fixes
2024-03-19 16:59:59 -07:00
Krrish Dholakia
a8d3d51d21
docs(token_based_auth.md): add jwt auth to docs
2024-03-19 16:34:27 -07:00
Krrish Dholakia
302bab6f1f
feat(handle_jwt.py): support authenticating admins into the proxy via jwt's
2024-03-19 15:00:27 -07:00
