phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-25 10:44:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
17927 commits 2480 branches 894 tags 534 MiB
Commit graph

77 commits

Author SHA1 Message Date
Krish Dholakia
d57be47b0f
Litellm ruff linting enforcement (#5992) 
* ci(config.yml): add a 'check_code_quality' step

Addresses https://github.com/BerriAI/litellm/issues/5991

* ci(config.yml): check why circle ci doesn't pick up this test

* ci(config.yml): fix to run 'check_code_quality' tests

* fix(__init__.py): fix unprotected import

* fix(__init__.py): don't remove unused imports

* build(ruff.toml): update ruff.toml to ignore unused imports

* fix: fix: ruff + pyright - fix linting + type-checking errors

* fix: fix linting errors

* fix(lago.py): fix module init error

* fix: fix linting errors

* ci(config.yml): cd into correct dir for checks

* fix(proxy_server.py): fix linting error

* fix(utils.py): fix bare except

causes ruff linting errors

* fix: ruff - fix remaining linting errors

* fix(clickhouse.py): use standard logging object

* fix(__init__.py): fix unprotected import

* fix: ruff - fix linting errors

* fix: fix linting errors

* ci(config.yml): cleanup code qa step (formatting handled in local_testing)

* fix(_health_endpoints.py): fix ruff linting errors

* ci(config.yml): just use ruff in check_code_quality pipeline for now

* build(custom_guardrail.py): include missing file

* style(embedding_handler.py): fix ruff check
 2024-10-01 19:44:20 -04:00
Ishaan Jaff
922c8ac758
[Feat-Proxy] add service accounts backend (#5852) 
* service_account_settings on config

* add service account checks

* call service_account_checks

* add testing for service accounts
 2024-09-23 16:37:02 -07:00
Krish Dholakia
8039b95aaf
LiteLLM Minor Fixes & Improvements (09/21/2024) (#5819) 
* fix(router.py): fix error message

* Litellm disable keys (#5814)

* build(schema.prisma): allow blocking/unblocking keys

Fixes https://github.com/BerriAI/litellm/issues/5328

* fix(key_management_endpoints.py): fix pop

* feat(auth_checks.py): allow admin to enable/disable virtual keys

Closes https://github.com/BerriAI/litellm/issues/5328

* docs(vertex.md): add auth section for vertex ai

Addresses - https://github.com/BerriAI/litellm/issues/5768#issuecomment-2365284223

* build(model_prices_and_context_window.json): show which models support prompt_caching

Closes https://github.com/BerriAI/litellm/issues/5776

* fix(router.py): allow setting default priority for requests

* fix(router.py): add 'retry-after' header for concurrent request limit errors

Fixes https://github.com/BerriAI/litellm/issues/5783

* fix(router.py): correctly raise and use retry-after header from azure+openai

Fixes https://github.com/BerriAI/litellm/issues/5783

* fix(user_api_key_auth.py): fix valid token being none

* fix(auth_checks.py): fix model dump for cache management object

* fix(user_api_key_auth.py): pass prisma_client to obj

* test(test_otel.py): update test for new key check

* test: fix test
 2024-09-21 18:51:53 -07:00
Ishaan Jaff
e6018a464f
[ Proxy - User Management]: If user assigned to a team don't show Default Team (#5791) 
* rename endpoint to ui_settings

* ui allow DEFAULT_TEAM_DISABLED

* fix logic

* docs Set `default_team_disabled: true` on your litellm config.yaml
 2024-09-19 17:13:58 -07:00
Ishaan Jaff
91e58d9049
[Feat] Add proxy level prometheus metrics (#5789) 
* add Proxy Level Tracking Metrics doc

* update service logger

* prometheus - track litellm_proxy_failed_requests_metric

* use REQUESTED_MODEL

* fix prom request_data
 2024-09-19 17:13:07 -07:00
Ishaan Jaff
eba76377ca
[Chore-Proxy] enforce jwt auth as enterprise feature (#5770) 
* enforce prometheus as enterprise feature

* show correct error on prometheus metric when not enrterprise user

* docs promethues metrics enforced

* docs enforce JWT auth

* enforce JWT auth as enterprise feature

* fix merge conflicts
 2024-09-18 16:28:37 -07:00
Krish Dholakia
234185ec13
LiteLLM Minor Fixes & Improvements (09/16/2024) (#5723) (#5731) 
* LiteLLM Minor Fixes & Improvements (09/16/2024)  (#5723)

* coverage (#5713)

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Move (#5714)

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* fix(litellm_logging.py): fix logging client re-init (#5710)

Fixes https://github.com/BerriAI/litellm/issues/5695

* fix(presidio.py): Fix logging_hook response and add support for additional presidio variables in guardrails config

Fixes https://github.com/BerriAI/litellm/issues/5682

* feat(o1_handler.py): fake streaming for openai o1 models

Fixes https://github.com/BerriAI/litellm/issues/5694

* docs: deprecated traceloop integration in favor of native otel (#5249)

* fix: fix linting errors

* fix: fix linting errors

* fix(main.py): fix o1 import

---------

Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>

* feat(spend_management_endpoints.py): expose `/global/spend/refresh` endpoint for updating material view (#5730)

* feat(spend_management_endpoints.py): expose `/global/spend/refresh` endpoint for updating material view

Supports having `MonthlyGlobalSpend` view be a material view, and exposes an endpoint to refresh it

* fix(custom_logger.py): reset calltype

* fix: fix linting errors

* fix: fix linting error

* fix: fix import

* test(test_databricks.py): fix databricks tests

---------

Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>
 2024-09-17 08:05:52 -07:00
Krish Dholakia
60709a0753
LiteLLM Minor Fixes and Improvements (09/13/2024) (#5689) 
* refactor: cleanup unused variables + fix pyright errors

* feat(health_check.py): Closes https://github.com/BerriAI/litellm/issues/5686

* fix(o1_reasoning.py): add stricter check for o-1 reasoning model

* refactor(mistral/): make it easier to see mistral transformation logic

* fix(openai.py): fix openai o-1 model param mapping

Fixes https://github.com/BerriAI/litellm/issues/5685

* feat(main.py): infer finetuned gemini model from base model

Fixes https://github.com/BerriAI/litellm/issues/5678

* docs(vertex.md): update docs to call finetuned gemini models

* feat(proxy_server.py): allow admin to hide proxy model aliases

Closes https://github.com/BerriAI/litellm/issues/5692

* docs(load_balancing.md): add docs on hiding alias models from proxy config

* fix(base.py): don't raise notimplemented error

* fix(user_api_key_auth.py): fix model max budget check

* fix(router.py): fix elif

* fix(user_api_key_auth.py): don't set team_id to empty str

* fix(team_endpoints.py): fix response type

* test(test_completion.py): handle predibase error

* test(test_proxy_server.py): fix test

* fix(o1_transformation.py): fix max_completion_token mapping

* test(test_image_generation.py): mark flaky test
 2024-09-14 10:02:55 -07:00
Krish Dholakia
4657a40ef1
LiteLLM Minor Fixes and Improvements (09/12/2024) (#5658) 
* fix(factory.py): handle tool call content as list

Fixes https://github.com/BerriAI/litellm/issues/5652

* fix(factory.py): enforce stronger typing

* fix(router.py): return model alias in /v1/model/info and /v1/model_group/info

* fix(user_api_key_auth.py): move noisy warning message to debug

cleanup logs

* fix(types.py): cleanup pydantic v2 deprecated param

Fixes https://github.com/BerriAI/litellm/issues/5649

* docs(gemini.md): show how to pass inline data to gemini api

Fixes https://github.com/BerriAI/litellm/issues/5674
 2024-09-12 23:04:06 -07:00
Krish Dholakia
98c34a7e27
LiteLLM Minor Fixes and Improvements (11/09/2024) (#5634) 
* fix(caching.py): set ttl for async_increment cache

fixes issue where ttl for redis client was not being set on increment_cache

Fixes https://github.com/BerriAI/litellm/issues/5609

* fix(caching.py): fix increment cache w/ ttl for sync increment cache on redis

Fixes https://github.com/BerriAI/litellm/issues/5609

* fix(router.py): support adding retry policy + allowed fails policy via config.yaml

* fix(router.py): don't cooldown single deployments

No point, as there's no other deployment to loadbalance with.

* fix(user_api_key_auth.py): support setting allowed email domains on jwt tokens

Closes https://github.com/BerriAI/litellm/issues/5605

* docs(token_auth.md): add user upsert + allowed email domain to jwt auth docs

* fix(litellm_pre_call_utils.py): fix dynamic key logging when team id is set

Fixes issue where key logging would not be set if team metadata was not none

* fix(secret_managers/main.py): load environment variables correctly

Fixes issue where os.environ/ was not being loaded correctly

* test(test_router.py): fix test

* feat(spend_tracking_utils.py): support logging additional usage params - e.g. prompt caching values for deepseek

* test: fix tests

* test: fix test

* test: fix test

* test: fix test

* test: fix test
 2024-09-11 22:36:06 -07:00
Krish Dholakia
ca37bb9de5
fix(pass_through_endpoints): support bedrock agents via pass through (#5527) 2024-09-04 22:22:22 -07:00
Krish Dholakia
1e7e538261
LiteLLM Minor fixes + improvements (08/04/2024) (#5505) 
* Minor IAM AWS OIDC Improvements (#5246)

* AWS IAM: Temporary tokens are valid across all regions after being issued, so it is wasteful to request one for each region.

* AWS IAM: Include an inline policy, to help reduce misuse of overly permissive IAM roles.

* (test_bedrock_completion.py): Ensure we are testing cross AWS region OIDC flow.

* fix(router.py): log rejected requests

Fixes https://github.com/BerriAI/litellm/issues/5498

* refactor: don't use verbose_logger.exception, if exception is raised

User might already have handling for this. But alerting systems in prod will raise this as an unhandled error.

* fix(datadog.py): support setting datadog source as an env var

Fixes https://github.com/BerriAI/litellm/issues/5508

* docs(logging.md): add dd_source to datadog docs

* fix(proxy_server.py): expose `/customer/list` endpoint for showing all customers

* (bedrock): Fix usage with Cloudflare AI Gateway, and proxies in general. (#5509)

* feat(anthropic.py): support 'cache_control' param for content when it is a string

* Revert "(bedrock): Fix usage with Cloudflare AI Gateway, and proxies in gener…" (#5519)

This reverts commit 3fac0349c2.

* refactor: ci/cd run again

---------

Co-authored-by: David Manouchehri <david.manouchehri@ai.moda>
 2024-09-04 22:16:55 -07:00
Ishaan Jaff
0b63625673 add check for admin only routes 2024-09-03 15:03:32 -07:00
Ishaan Jaff
253ef5f995 allow setting allowed routes on proxy 2024-09-03 13:59:31 -07:00
Ishaan Jaff
748cc80783 fix auth checks for provider routes 2024-08-29 16:40:46 -07:00
Ishaan Jaff
a982cb7ae5
Merge pull request #5420 from BerriAI/litellm_add_oauth2_mapping 
[Feat-Proxy] Add hook for oauth2 proxy headers
 2024-08-28 16:34:26 -07:00
Ishaan Jaff
946e3b63e1 add hook for oauth2 proxy 2024-08-28 15:51:20 -07:00
Ishaan Jaff
d3a5162103 add checks for safe request body 2024-08-28 14:07:53 -07:00
Ishaan Jaff
c27640e6e4 add /rerank test 2024-08-27 17:50:37 -07:00
Krish Dholakia
509ae0ca71
Merge pull request #5308 from BerriAI/litellm_team_admin_permissions 
feat(user_api_key_auth.py): allow team admin to add new members to team
 2024-08-21 14:21:22 -07:00
Krrish Dholakia
a61f3e7656 refactor(team_endpoints.py): refactor auth checks for team member endpoints to ui team admin to manage it 2024-08-20 16:57:18 -07:00
Krrish Dholakia
fa6c9bf42e feat(user_api_key_auth.py): allow team admin to add new members to team 2024-08-20 14:01:12 -07:00
Krish Dholakia
f888204a12
Merge pull request #5287 from BerriAI/litellm_fix_response_cost_cal 
fix(cost_calculator.py): only override base model if custom pricing is set
 2024-08-20 11:42:48 -07:00
Ishaan Jaff
cae51fc7fc fix _get_request_ip_address 2024-08-19 21:08:27 -07:00
Krrish Dholakia
e86cb90f98 fix(user_api_key_auth.py): fix client_ip 2024-08-19 21:01:54 -07:00
Krrish Dholakia
417547b6f9 fix(user_api_key_auth.py): log requester ip address to logs on request rejection 
Closes https://github.com/BerriAI/litellm/issues/5220
 2024-08-19 11:03:58 -07:00
Krish Dholakia
a8dd2b6910
Merge pull request #5244 from BerriAI/litellm_better_error_logging_sentry 
refactor: replace .error() with .exception() logging for better debugging on sentry
 2024-08-16 19:16:20 -07:00
Ishaan Jaff
ac833f415d docs oauh 2.0 enterprise feature 2024-08-16 14:00:24 -07:00
Ishaan Jaff
cd28b6607e add debugging for oauth2.0 2024-08-16 13:40:32 -07:00
Krrish Dholakia
61f4b71ef7 refactor: replace .error() with .exception() logging for better debugging on sentry 2024-08-16 09:22:47 -07:00
Krish Dholakia
aa2267bddb
Merge branch 'main' into litellm_fix_pass_through_endpoints 2024-08-14 14:59:38 -07:00
Krrish Dholakia
5fdbfcee44 fix(user_api_key_auth.py): more precisely expand scope to handle 'basic' tokens 2024-08-13 22:00:33 -07:00
Ishaan Jaff
e0978378c1 return detailed error message on check_valid_ip 2024-08-13 21:29:21 -07:00
Krrish Dholakia
742e3cbccf feat(user_api_key_auth.py): support calling langfuse with litellm user_api_key_auth 2024-08-13 20:26:24 -07:00
Ishaan Jaff
d0e6ca659f add test for test_check_valid_ip_sent_with_x_forwarded_for 2024-08-13 15:54:53 -07:00
Ishaan Jaff
b94c982ec9 use _check_valid_ip 2024-08-13 15:41:01 -07:00
Ishaan Jaff
ab208eb546 check use_x_forwarded_for 2024-08-13 15:18:24 -07:00
Krrish Dholakia
2ed2ae96af fix(user_api_key_auth.py): move warning to debug log 2024-08-12 10:35:44 -07:00
Krrish Dholakia
5703da9b42 fix(user_api_key_auth.py): Fixes https://github.com/BerriAI/litellm/issues/5111 2024-08-08 10:30:15 -07:00
Krrish Dholakia
f76261af35 refactor(user_api_key_auth.py): refactor to replace user_id_information list with pydantic user_obj 
Allows using the 'get_user_object' function in user_api_key_auth, keeping it consistent across jwt-auth and key-auth
 2024-08-07 15:33:55 -07:00
Krrish Dholakia
ff373663a3 fix: fix tests 2024-08-07 15:02:04 -07:00
Krrish Dholakia
d832327ccf fix(user_api_key_auth.py): respect team budgets over user budget, if key belongs to team 
Closes https://github.com/BerriAI/litellm/issues/5097
 2024-08-07 14:32:27 -07:00
Krrish Dholakia
818a70838d fix(user_api_key_auth.py): fix _get_user_role 2024-08-05 20:54:27 -07:00
Krrish Dholakia
5c6dcd6029 test: improve debugging for test 2024-08-05 19:41:08 -07:00
Ishaan Jaff
aa7174c270 use get_request_route 2024-08-05 10:13:47 -07:00
Krrish Dholakia
f610fba58f fix(user_api_key_auth.py): handle older user_role's 
Fixes issue where older user_role's (e.g. app_user) weren't being recognized. + Adds testing for it
 2024-08-05 08:57:06 -07:00
Krrish Dholakia
186e5f55e8 fix(user_api_key_auth.py): fix linting errors 2024-08-02 16:34:32 -07:00
Krrish Dholakia
1080c7014e build(ui): allow admin_viewer to view teams tab 
Allows admin viewe role to see available teams on proxy ui
 2024-08-02 16:19:40 -07:00
Krrish Dholakia
b77edc59ed fix(user_api_key_cache): fix check to not raise error if team object is missing 2024-07-30 18:25:04 -07:00
Krrish Dholakia
142f4fefd0 fix(auth_checks.py): fix redis usage for team cached objects 2024-07-30 17:30:00 -07:00