Commit graph

2132 commits

Author SHA1 Message Date
Ishaan Jaff
1262c5cb39 fix - /organization/new logic 2024-05-15 17:27:46 -07:00
Ishaan Jaff
9f43761000 fix - show alerts on /completions endpoint 2024-05-15 15:45:21 -07:00
Krrish Dholakia
6a357b4275 feat(proxy_server.py): Enabling Admin to control general settings on proxy ui 2024-05-15 15:26:57 -07:00
Ishaan Jaff
c00d232900
Merge pull request #3655 from BerriAI/litellm_only_show_debug_info_server_side
Fix Proxy Server - only show API base, Model server log exceptions, not on client side
2024-05-15 14:20:14 -07:00
Ishaan Jaff
1a11b151b7 fix - show litellm debug info in errors 2024-05-15 14:18:12 -07:00
Ishaan Jaff
f6e342303e show debug info on slack alerts 2024-05-15 13:23:44 -07:00
Ishaan Jaff
74f093bb4b
Merge pull request #3653 from BerriAI/litellm_fix_text_completions
[Fix] - Alerting on `/completions` -  don't raise hanging request alert for /completions
2024-05-15 11:28:55 -07:00
Krrish Dholakia
dba713ea43 fix(router.py): add validation for how router fallbacks are setup
prevent user errors
2024-05-15 10:44:16 -07:00
Ishaan Jaff
2e81347607 fix - don't raise hanging request alert for /completions 2024-05-15 10:27:02 -07:00
Krrish Dholakia
5dcf3d672c feat(proxy_server.py): new /end_user/info endpoint
get spend for a specific end-user
2024-05-15 09:50:52 -07:00
Krrish Dholakia
51a02de4cf refactor(proxy_server.py): update doc string for /user/update 2024-05-15 08:25:14 -07:00
Krrish Dholakia
8117af664c fix(huggingface_restapi.py): fix task extraction from model name 2024-05-15 07:28:19 -07:00
Ishaan Jaff
e7af8d61cd fix check_request_disconnected = None case 2024-05-14 20:38:32 -07:00
Ishaan Jaff
4466982507 feat - cancel tasks when fast api request is cancelled 2024-05-14 19:58:51 -07:00
Krrish Dholakia
9eee2f3889 docs(prod.md): add 'disable load_dotenv' tutorial to docs 2024-05-14 19:13:22 -07:00
Krrish Dholakia
1ab4974773 fix: disable 'load_dotenv' for prod environments 2024-05-14 19:09:36 -07:00
Krrish Dholakia
1db1af1154 fix(types): fix typing 2024-05-14 17:09:36 -07:00
Krrish Dholakia
888c53e774 fix(proxy/_types.py): fix linting errors 2024-05-14 17:02:11 -07:00
Ishaan Jaff
aa1615c757
Merge pull request #3626 from BerriAI/litellm_reset_spend_per_team_api_key
feat - reset spend per team, api_key [Only Master Key]
2024-05-14 11:49:07 -07:00
Ishaan Jaff
787c02c8db fix - return success spend reset 2024-05-14 11:38:39 -07:00
Ishaan Jaff
7e56e27226 fix security for global_spend_reset 2024-05-14 11:04:50 -07:00
Krish Dholakia
2c867ea9a5
Merge pull request #3600 from msabramo/msabramo/fix-pydantic-warnings
Update pydantic code to fix warnings
2024-05-13 22:00:39 -07:00
Ishaan Jaff
3686789c36 feat - reset spend per team, api_key 2024-05-13 18:23:23 -07:00
Ishaan Jaff
f19e6724ee ui - new build 2024-05-13 17:48:24 -07:00
Ishaan Jaff
6ea5a276f8 backend - show spend per tag by time 2024-05-13 17:13:48 -07:00
Krrish Dholakia
c7b3193944 fix(proxy/_types.py): allow jwt admin to access /team/list route 2024-05-13 16:07:31 -07:00
Ishaan Jaff
ea9b4dc439
Merge pull request #3619 from BerriAI/litellm_show_spend_reports
[Feat] -  `/global/spend/report`
2024-05-13 16:06:02 -07:00
Ishaan Jaff
eb2d6ba20a ui - new build 2024-05-13 15:56:59 -07:00
Krrish Dholakia
b4a8665d11 fix(utils.py): fix custom pricing when litellm model != response obj model name 2024-05-13 15:25:35 -07:00
Ishaan Jaff
12cf9d71c7 feat - /spend/report endpoint 2024-05-13 15:01:02 -07:00
Ishaan Jaff
9d4b727913 fix - show team based spend reports 2024-05-13 13:56:48 -07:00
Ishaan Jaff
21845bc061
Merge pull request #3609 from BerriAI/litellm_send_daily_spend_report
[Feat] send weekly spend reports by Team/Tag
2024-05-13 12:45:37 -07:00
Ishaan Jaff
40b2f33a80 fix - only schedule spend alerting when db is not none 2024-05-13 12:30:54 -07:00
Marc Abramowitz
1efa2e65ac Merge branch 'main' into msabramo/fix-pydantic-warnings 2024-05-13 11:34:06 -07:00
Marc Abramowitz
73541b1f17 Merge branch 'msabramo/pydantic_replace_root_validator_with_model_validator' into msabramo/fix-pydantic-warnings 2024-05-13 11:25:55 -07:00
Ishaan Jaff
aac81c59b5 test - weekly / monthly spend report alerts on /health/services 2024-05-13 10:50:26 -07:00
Ishaan Jaff
197eb44832 fix scheduling spend reports 2024-05-13 10:45:22 -07:00
Ishaan Jaff
4a679bb640 schedule weekly/monthly spend reports 2024-05-13 10:44:19 -07:00
Marc Abramowitz
4ccdbfd67d model_config = ConfigDict instead of model_config = dict 2024-05-13 10:38:01 -07:00
Marc Abramowitz
abd90e6296 Revert root_validator => model_validator change
I have it in a separate PR:

https://github.com/BerriAI/litellm/pull/3611
2024-05-13 10:33:04 -07:00
Marc Abramowitz
ad8c3ac2c3 Change pydantic root_validator to model_validator
pydantic v1 uses `root_validator` and pydantic v2 uses `model_validator`.

pydantic v2 emits a warning when `root_validator` is used. E.g.:

```
litellm/proxy/_types.py:225
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:225:
  PydanticDeprecatedSince20: Pydantic V1 style `@root_validator`
  validators are deprecated.
  You should migrate to Pydantic V2 style `@model_validator` validators,
  see the migration guide for more details.
  Deprecated in Pydantic V2.0 to be removed in V3.0.
  See Pydantic V2 Migration Guide at
  https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)
```

This change eliminates those warnings with pydantic v2, while retaining
compatibility with pydantic v1.

pydantic 2.7.1 before

```
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
...
litellm/proxy/_types.py:225
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:225: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)
...
========================== 10 passed, 2 skipped, 39 warnings in 8.67s ===========================
```

pydantic 2.7.1 after

```
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
...
========================== 10 passed, 2 skipped, 27 warnings in 9.85s ===========================
```

pydantic 1.10.5 after

```
$ poetry run pip install 'pydantic<2'
...
Successfully installed pydantic-1.10.15
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
...
=========================== 10 passed, 2 skipped, 1 warning in 8.13s ============================
```
2024-05-13 10:23:44 -07:00
Ishaan Jaff
2cd584a3ad feat - send_monthly_spend_report 2024-05-13 10:17:40 -07:00
Ishaan Jaff
07247452c5 feat - show monthly spend reports 2024-05-13 10:10:44 -07:00
Krrish Dholakia
13e1577753 fix(slack_alerting.py): don't fire spam alerts when backend api call fails 2024-05-13 10:04:43 -07:00
Ishaan Jaff
50f3677989 feat - _get_weekly_spend_reports 2024-05-13 09:26:51 -07:00
Ishaan Jaff
b7bbaf1a68 feat - send daily spend reports 2024-05-13 09:25:31 -07:00
Krrish Dholakia
c3293474dd fix(proxy_server.py): return 'allowed-model-region' in headers 2024-05-13 08:48:16 -07:00
Ishaan Jaff
514c5737f8
Merge pull request #3587 from BerriAI/litellm_proxy_use_batch_completions_model_csv
[Feat] Use csv values for proxy batch completions (OpenAI Python compatible)
2024-05-13 07:55:12 -07:00
Marc Abramowitz
876840e995 Update pydantic code to fix warnings
Before:

```shell
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
====================================== test session starts ======================================
platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0
rootdir: /Users/abramowi/Code/OpenSource/litellm
plugins: anyio-4.3.0, asyncio-0.23.6, mock-3.14.0
asyncio: mode=Mode.STRICT
collected 12 items

litellm/tests/test_proxy_server.py s..........s                                           [100%]

======================================= warnings summary ========================================
../../../Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/pydantic/_internal/_config.py:284: 25 warnings
  /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/pydantic/_internal/_config.py:284: PydanticDeprecatedSince20: Support for class-based `config` is deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    warnings.warn(DEPRECATION_MESSAGE, DeprecationWarning)

litellm/proxy/_types.py:225
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:225: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:312
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:312: PydanticDeprecatedSince20: `pydantic.config.Extra` is deprecated, use literal values instead (e.g. `extra='allow'`). Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    extra = Extra.allow  # Allow extra fields

litellm/proxy/_types.py:315
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:315: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:344
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:344: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:391
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:391: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:460
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:460: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:480
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:480: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:493
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:493: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:538
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:538: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:575
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:575: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:870
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:870: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:897
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:897: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:918
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:918: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/utils.py:43
  /Users/abramowi/Code/OpenSource/litellm/litellm/utils.py:43: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources  # type: ignore

litellm/tests/test_proxy_server.py::test_bedrock_embedding
litellm/tests/test_proxy_server.py::test_chat_completion
litellm/tests/test_proxy_server.py::test_chat_completion_azure
litellm/tests/test_proxy_server.py::test_chat_completion_optional_params
litellm/tests/test_proxy_server.py::test_embedding
litellm/tests/test_proxy_server.py::test_engines_model_chat_completions
litellm/tests/test_proxy_server.py::test_health
litellm/tests/test_proxy_server.py::test_img_gen
litellm/tests/test_proxy_server.py::test_openai_deployments_model_chat_completions_azure
  /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead.
    warnings.warn(message, DeprecationWarning)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================== 10 passed, 2 skipped, 48 warnings in 9.83s ===========================
```

After:

```shell
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
====================================== test session starts ======================================
platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0
rootdir: /Users/abramowi/Code/OpenSource/litellm
plugins: anyio-4.3.0, asyncio-0.23.6, mock-3.14.0
asyncio: mode=Mode.STRICT
collected 12 items

litellm/tests/test_proxy_server.py s..........s                                           [100%]

======================================= warnings summary ========================================
litellm/utils.py:43
  /Users/abramowi/Code/OpenSource/litellm/litellm/utils.py:43: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources  # type: ignore

litellm/tests/test_proxy_server.py::test_bedrock_embedding
litellm/tests/test_proxy_server.py::test_chat_completion
litellm/tests/test_proxy_server.py::test_chat_completion_azure
litellm/tests/test_proxy_server.py::test_chat_completion_optional_params
litellm/tests/test_proxy_server.py::test_embedding
litellm/tests/test_proxy_server.py::test_engines_model_chat_completions
litellm/tests/test_proxy_server.py::test_health
litellm/tests/test_proxy_server.py::test_img_gen
litellm/tests/test_proxy_server.py::test_openai_deployments_model_chat_completions_azure
  /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead.
    warnings.warn(message, DeprecationWarning)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================== 10 passed, 2 skipped, 10 warnings in 9.62s ===========================
```
2024-05-12 10:11:48 -07:00
Krrish Dholakia
b4684d5132 fix(proxy_server.py): linting fix 2024-05-11 22:05:01 -07:00