Ishaan Jaff
4d1b4beb3d
(refactor) caching use LLMCachingHandler for async_get_cache and set_cache ( #6208 )
...
* use folder for caching
* fix importing caching
* fix clickhouse pyright
* fix linting
* fix correctly pass kwargs and args
* fix test case for embedding
* fix linting
* fix embedding caching logic
* fix refactor handle utils.py
* fix test_embedding_caching_azure_individual_items_reordered
2024-10-14 16:34:01 +05:30
Ishaan Jaff
1fd437e263
(feat proxy) [beta] add support for organization role based access controls ( #6112 )
...
* track LiteLLM_OrganizationMembership
* add add_internal_user_to_organization
* add org membership to schema
* read organization membership when reading user info in auth checks
* add check for valid organization_id
* add test for test_create_new_user_in_organization
* test test_create_new_user_in_organization
* add new ADMIN role
* add test for org admins creating teams
* add test for test_org_admin_create_user_permissions
* test_org_admin_create_user_team_wrong_org_permissions
* test_org_admin_create_user_team_wrong_org_permissions
* fix organization_role_based_access_check
* fix getting user members
* fix TeamBase
* fix types used for use role
* fix type checks
* sync prisma schema
* docs - organization admins
* fix use organization_endpoints for /organization management
* add types for org member endpoints
* fix role name for org admin
* add type for member add response
* add organization/member_add
* add error handling for adding members to an org
* add nice doc string for oranization/member_add
* fix test_create_new_user_in_organization
* linting fix
* use simple route changes
* fix types
* add organization member roles
* add org admin auth checks
* add auth checks for orgs
* test for creating teams as org admin
* simplify org id usage
* fix typo
* test test_org_admin_create_user_team_wrong_org_permissions
* fix type check issue
* code quality fix
* fix schema.prisma
2024-10-09 15:18:18 +05:30
Krish Dholakia
d57be47b0f
Litellm ruff linting enforcement ( #5992 )
...
* ci(config.yml): add a 'check_code_quality' step
Addresses https://github.com/BerriAI/litellm/issues/5991
* ci(config.yml): check why circle ci doesn't pick up this test
* ci(config.yml): fix to run 'check_code_quality' tests
* fix(__init__.py): fix unprotected import
* fix(__init__.py): don't remove unused imports
* build(ruff.toml): update ruff.toml to ignore unused imports
* fix: fix: ruff + pyright - fix linting + type-checking errors
* fix: fix linting errors
* fix(lago.py): fix module init error
* fix: fix linting errors
* ci(config.yml): cd into correct dir for checks
* fix(proxy_server.py): fix linting error
* fix(utils.py): fix bare except
causes ruff linting errors
* fix: ruff - fix remaining linting errors
* fix(clickhouse.py): use standard logging object
* fix(__init__.py): fix unprotected import
* fix: ruff - fix linting errors
* fix: fix linting errors
* ci(config.yml): cleanup code qa step (formatting handled in local_testing)
* fix(_health_endpoints.py): fix ruff linting errors
* ci(config.yml): just use ruff in check_code_quality pipeline for now
* build(custom_guardrail.py): include missing file
* style(embedding_handler.py): fix ruff check
2024-10-01 19:44:20 -04:00
Ishaan Jaff
58171f35ef
[Fix proxy perf] Use correct cache key when reading from redis cache ( #5928 )
...
* fix parallel request limiter use correct user id
* async def get_user_object(
fix
* use safe get_internal_user_object
* fix store internal users in redis correctly
2024-09-26 18:13:35 -07:00
Ishaan Jaff
f6cdb4ca0d
[Perf improvement Proxy] Use Dual Cache for getting key and team objects ( #5903 )
...
* use dual cache - perf
* fix auth checks
* fix budget checks for keys
* fix get / set team tests
2024-09-25 19:56:17 -07:00
Ishaan Jaff
7cbcf538c6
[Feat] Improve OTEL Tracking - Require all Redis Cache reads to be logged on OTEL ( #5881 )
...
* fix use previous internal usage caching logic
* fix test_dual_cache_uses_redis
* redis track event_metadata in service logging
* show otel error on _get_parent_otel_span_from_kwargs
* track parent otel span on internal usage cache
* update_request_status
* fix internal usage cache
* fix linting
* fix test internal usage cache
* fix linting error
* show event metadata in redis set
* fix test_get_team_redis
* fix test_get_team_redis
* test_proxy_logging_setup
2024-09-25 10:57:08 -07:00
Krish Dholakia
8039b95aaf
LiteLLM Minor Fixes & Improvements (09/21/2024) ( #5819 )
...
* fix(router.py): fix error message
* Litellm disable keys (#5814 )
* build(schema.prisma): allow blocking/unblocking keys
Fixes https://github.com/BerriAI/litellm/issues/5328
* fix(key_management_endpoints.py): fix pop
* feat(auth_checks.py): allow admin to enable/disable virtual keys
Closes https://github.com/BerriAI/litellm/issues/5328
* docs(vertex.md): add auth section for vertex ai
Addresses - https://github.com/BerriAI/litellm/issues/5768#issuecomment-2365284223
* build(model_prices_and_context_window.json): show which models support prompt_caching
Closes https://github.com/BerriAI/litellm/issues/5776
* fix(router.py): allow setting default priority for requests
* fix(router.py): add 'retry-after' header for concurrent request limit errors
Fixes https://github.com/BerriAI/litellm/issues/5783
* fix(router.py): correctly raise and use retry-after header from azure+openai
Fixes https://github.com/BerriAI/litellm/issues/5783
* fix(user_api_key_auth.py): fix valid token being none
* fix(auth_checks.py): fix model dump for cache management object
* fix(user_api_key_auth.py): pass prisma_client to obj
* test(test_otel.py): update test for new key check
* test: fix test
2024-09-21 18:51:53 -07:00
Krish Dholakia
9c8fdee068
Additional Fixes (09/17/2024) ( #5759 )
...
* fix(auth_checks.py): check if key has all model access via wildcard routing
Fixes issue where key with `openai/*` couldn't call gpt models
* fix(slack_alerting.py): expose flag for disabling failed spend tracking alerts
2024-09-17 23:02:12 -07:00
Krish Dholakia
4657a40ef1
LiteLLM Minor Fixes and Improvements (09/12/2024) ( #5658 )
...
* fix(factory.py): handle tool call content as list
Fixes https://github.com/BerriAI/litellm/issues/5652
* fix(factory.py): enforce stronger typing
* fix(router.py): return model alias in /v1/model/info and /v1/model_group/info
* fix(user_api_key_auth.py): move noisy warning message to debug
cleanup logs
* fix(types.py): cleanup pydantic v2 deprecated param
Fixes https://github.com/BerriAI/litellm/issues/5649
* docs(gemini.md): show how to pass inline data to gemini api
Fixes https://github.com/BerriAI/litellm/issues/5674
2024-09-12 23:04:06 -07:00
Ishaan Jaff
0b63625673
add check for admin only routes
2024-09-03 15:03:32 -07:00
Ishaan Jaff
253ef5f995
allow setting allowed routes on proxy
2024-09-03 13:59:31 -07:00
Ishaan Jaff
9c2d974f5e
fix checking request body
2024-08-28 14:10:09 -07:00
Ishaan Jaff
ba1912afd1
add check for is_request_body_safe
2024-08-28 13:40:14 -07:00
Krrish Dholakia
d7b525f391
feat(auth_checks.py): allow team to call all models, when explicitly set via /*
2024-08-22 16:38:56 -07:00
Ishaan Jaff
08db691dec
use model access groups for teams
2024-08-17 16:45:53 -07:00
Krrish Dholakia
5703da9b42
fix(user_api_key_auth.py): Fixes https://github.com/BerriAI/litellm/issues/5111
2024-08-08 10:30:15 -07:00
Krish Dholakia
baf01b47d8
Merge branch 'main' into litellm_personal_user_budgets
2024-08-07 19:59:50 -07:00
Krrish Dholakia
ff373663a3
fix: fix tests
2024-08-07 15:02:04 -07:00
Krrish Dholakia
d832327ccf
fix(user_api_key_auth.py): respect team budgets over user budget, if key belongs to team
...
Closes https://github.com/BerriAI/litellm/issues/5097
2024-08-07 14:32:27 -07:00
Krish Dholakia
c82fc0cac2
Merge branch 'main' into litellm_support_lakera_config_thresholds
2024-08-06 22:47:13 -07:00
Krrish Dholakia
b77edc59ed
fix(user_api_key_cache): fix check to not raise error if team object is missing
2024-07-30 18:25:04 -07:00
Krrish Dholakia
142f4fefd0
fix(auth_checks.py): fix redis usage for team cached objects
2024-07-30 17:30:00 -07:00
Krrish Dholakia
92b539b42a
fix(auth_checks.py): handle writing team object to redis caching correctly
2024-07-29 07:51:44 -07:00
Krrish Dholakia
2c71f6dd04
feat(auth_check.py): support using redis cache for team objects
...
Allows team update / check logic to work across instances instantly
2024-07-25 19:35:29 -07:00
Krrish Dholakia
6ab2527fdc
feat(auth_check.py): support using redis cache for team objects
...
Allows team update / check logic to work across instances instantly
2024-07-24 18:14:49 -07:00
Krish Dholakia
c4db6aa15e
Merge pull request #4810 from BerriAI/litellm_team_modify_guardrails
...
feat(auth_checks.py): Allow admin to disable team from turning on/off guardrails
2024-07-22 22:32:24 -07:00
Ishaan Jaff
b64755d2a1
check is_llm_api_route
2024-07-22 14:43:30 -07:00
Krrish Dholakia
8b3c8102a7
feat(auth_checks.py): Allow admin to disable team from turning on/off guardrails.
2024-07-20 18:39:05 -07:00
Krrish Dholakia
35e640076b
fix(user_api_key_auth.py): update valid token cache with updated team object cache
2024-07-19 17:06:49 -07:00
Ishaan Jaff
b30fa64e2a
fix - use helper to check if a route is openai route
2024-07-09 12:00:07 -07:00
Krrish Dholakia
5729eb5168
fix(user_api_key_auth.py): ensure user has access to fallback models
...
for client side fallbacks, checks if user has access to fallback models
2024-06-20 16:02:19 -07:00
Krrish Dholakia
6558abf845
fix(proxy_server.py): track team spend for cached team object
...
fixes issue where team budgets for jwt tokens weren't asserted
2024-06-18 17:10:12 -07:00
Krrish Dholakia
c27ae34a39
fix(proxy_server.py): use consistent 400-status code error code for exceeded budget errors
...
standardizes error code for budget exceeded errors to status code 400
2024-06-11 16:10:58 -07:00
Ishaan Jaff
2cf3133669
Merge branch 'main' into litellm_svc_logger
2024-06-07 14:01:54 -07:00
Ishaan Jaff
87533bacf7
fix importing Span
2024-06-07 09:55:59 -07:00
Ishaan Jaff
37e7a7b2d5
fix - log_to_opentelemetry
2024-06-06 22:28:01 -07:00
Ishaan Jaff
32655b9ef2
fix auth checks
2024-06-06 22:13:13 -07:00
Ishaan Jaff
fcb1427a8c
fix log_to_opentelemetry
2024-06-06 21:29:40 -07:00
Ishaan Jaff
b551462518
feat -v0 parent_otel_span in basic db reads
2024-06-06 19:49:18 -07:00
Ishaan Jaff
aaf35c7966
feat - enforce enforced_params as a premium feature
2024-06-06 09:03:31 -07:00
Ishaan Jaff
883ee08225
feat - enforce params in requests
2024-06-06 08:54:59 -07:00
Ishaan Jaff
a4b6a959d8
fix literal usage
2024-05-30 14:28:53 -07:00
Krrish Dholakia
cfcf5969c8
fix(proxy_server.py): fix end user object check when master key used
...
check if end user max budget exceeded for master key
2024-05-29 17:20:59 -07:00
Krrish Dholakia
a33b9221da
fix(proxy_server.py): fix admin allowed routes
2024-05-22 20:55:21 -07:00
Ishaan Jaff
eac7e70dca
feat - include litellm_budget table when getting end_user
2024-05-22 13:30:08 -07:00
Krrish Dholakia
600b6f7e1d
feat(proxy_server.py): support 'user_id_upsert' flag for jwt_auth
2024-05-15 22:19:59 -07:00
Krrish Dholakia
f48cd87cf3
feat(proxy_server.py): make team_id optional for jwt token auth (only enforced, if set)
...
Allows users to use jwt auth for internal chat apps
2024-05-15 21:05:14 -07:00
Krrish Dholakia
e3f25a4a1f
fix(auth_checks.py): fix 'get_end_user_object'
...
await cache get
2024-05-09 13:05:56 -07:00
Krrish Dholakia
3d18897d69
feat(router.py): enable filtering model group by 'allowed_model_region'
2024-05-08 22:10:17 -07:00
Ishaan Jaff
e49fe47d2e
fix - only run global_proxy_spend on chat completion calls
2024-04-27 14:11:00 -07:00
