Commit graph

63 commits

Author SHA1 Message Date
Krrish Dholakia
9613f67f08 fix(types/init.py): don't import openai assistants types by default 2024-05-15 08:50:31 -07:00
Krrish Dholakia
57fb1ec852 fix(types): fix typing 2024-05-14 17:09:36 -07:00
Krrish Dholakia
46d2d6bba9 fix(types/router.py): fix python3.8 typing issue 2024-05-14 16:56:07 -07:00
Krrish Dholakia
56d50695fe fix(types/router.py): fix arbitrary_types_allowed flag 2024-05-14 16:53:02 -07:00
Krrish Dholakia
fdee5b962b fix(types/router.py): fix router pydantic v1 errors 2024-05-14 16:49:55 -07:00
Krrish Dholakia
487567a77f fix(types/completion.py): fix completion configdict pydantic v1 error 2024-05-14 16:39:50 -07:00
Krrish Dholakia
76e19a3695 fix(types/completion.py): linting fixes 2024-05-14 13:19:38 -07:00
Krrish Dholakia
3a55ec6b36 ci: test linting change for pydantic v1 2024-05-14 11:34:16 -07:00
Krish Dholakia
bef8a7f1d9 Merge pull request #3600 from msabramo/msabramo/fix-pydantic-warnings
Update pydantic code to fix warnings
2024-05-13 22:00:39 -07:00
Krrish Dholakia
6f20389bd5 feat(router.py): enable default fallbacks
allow user to define a generic list of fallbacks, in case a new deployment is bad

Closes https://github.com/BerriAI/litellm/issues/3623
2024-05-13 17:49:56 -07:00
Marc Abramowitz
b61cc97771 Merge branch 'msabramo/pydantic_replace_root_validator_with_model_validator' into msabramo/fix-pydantic-warnings 2024-05-13 11:25:55 -07:00
Marc Abramowitz
3048ea5129 Use Config class w/ arbitrary_types_allowed for pydantic v1
pydantic v2 warns about using a Config class.
But without this, pydantic v1 will raise an error:
    RuntimeError: no validator found for <class 'openai.Timeout'>,
    see `arbitrary_types_allowed` in Config
Putting arbitrary_types_allowed = True in the ConfigDict doesn't work in pydantic v1.
So we mostly use model_config = ConfigDict(...) and then only use the Config
class with arbitrary_types_allowed = True for pydantic v1.
2024-05-13 11:22:50 -07:00
Marc Abramowitz
de9b515230 Replace some Config classes with model_config
in `litellm/types/router.py`

I didn't replace the ones that have `allow_arbitrary_types` because changing
those seems to break pydantic v1 compatibility.
2024-05-13 11:04:13 -07:00
Marc Abramowitz
7b4f06c7b7 Revert litellm/types/router.py changes
These don't work in pydantic v1.
2024-05-13 10:34:05 -07:00
Marc Abramowitz
d352ad2a27 Update pydantic code to fix warnings
Before:

```shell
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
====================================== test session starts ======================================
platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0
rootdir: /Users/abramowi/Code/OpenSource/litellm
plugins: anyio-4.3.0, asyncio-0.23.6, mock-3.14.0
asyncio: mode=Mode.STRICT
collected 12 items

litellm/tests/test_proxy_server.py s..........s                                           [100%]

======================================= warnings summary ========================================
../../../Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/pydantic/_internal/_config.py:284: 25 warnings
  /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/pydantic/_internal/_config.py:284: PydanticDeprecatedSince20: Support for class-based `config` is deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    warnings.warn(DEPRECATION_MESSAGE, DeprecationWarning)

litellm/proxy/_types.py:225
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:225: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:312
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:312: PydanticDeprecatedSince20: `pydantic.config.Extra` is deprecated, use literal values instead (e.g. `extra='allow'`). Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    extra = Extra.allow  # Allow extra fields

litellm/proxy/_types.py:315
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:315: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:344
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:344: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:391
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:391: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:460
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:460: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:480
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:480: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:493
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:493: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:538
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:538: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:575
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:575: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:870
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:870: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:897
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:897: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:918
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:918: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/utils.py:43
  /Users/abramowi/Code/OpenSource/litellm/litellm/utils.py:43: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources  # type: ignore

litellm/tests/test_proxy_server.py::test_bedrock_embedding
litellm/tests/test_proxy_server.py::test_chat_completion
litellm/tests/test_proxy_server.py::test_chat_completion_azure
litellm/tests/test_proxy_server.py::test_chat_completion_optional_params
litellm/tests/test_proxy_server.py::test_embedding
litellm/tests/test_proxy_server.py::test_engines_model_chat_completions
litellm/tests/test_proxy_server.py::test_health
litellm/tests/test_proxy_server.py::test_img_gen
litellm/tests/test_proxy_server.py::test_openai_deployments_model_chat_completions_azure
  /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead.
    warnings.warn(message, DeprecationWarning)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================== 10 passed, 2 skipped, 48 warnings in 9.83s ===========================
```

After:

```shell
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
====================================== test session starts ======================================
platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0
rootdir: /Users/abramowi/Code/OpenSource/litellm
plugins: anyio-4.3.0, asyncio-0.23.6, mock-3.14.0
asyncio: mode=Mode.STRICT
collected 12 items

litellm/tests/test_proxy_server.py s..........s                                           [100%]

======================================= warnings summary ========================================
litellm/utils.py:43
  /Users/abramowi/Code/OpenSource/litellm/litellm/utils.py:43: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources  # type: ignore

litellm/tests/test_proxy_server.py::test_bedrock_embedding
litellm/tests/test_proxy_server.py::test_chat_completion
litellm/tests/test_proxy_server.py::test_chat_completion_azure
litellm/tests/test_proxy_server.py::test_chat_completion_optional_params
litellm/tests/test_proxy_server.py::test_embedding
litellm/tests/test_proxy_server.py::test_engines_model_chat_completions
litellm/tests/test_proxy_server.py::test_health
litellm/tests/test_proxy_server.py::test_img_gen
litellm/tests/test_proxy_server.py::test_openai_deployments_model_chat_completions_azure
  /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead.
    warnings.warn(message, DeprecationWarning)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================== 10 passed, 2 skipped, 10 warnings in 9.62s ===========================
```
2024-05-12 10:11:48 -07:00
Krish Dholakia
784ae85ba0 Merge branch 'main' into litellm_bedrock_command_r_support 2024-05-11 21:24:42 -07:00
Krrish Dholakia
e2b8883d4c fix(types/bedrock.py): linting fix 2024-05-11 19:49:46 -07:00
Krrish Dholakia
68596ced04 feat(bedrock_httpx.py): working bedrock command-r sync+async streaming 2024-05-11 19:39:51 -07:00
Krrish Dholakia
5185580e3d feat(bedrock_httpx.py): working cohere command r async calls 2024-05-11 15:04:38 -07:00
Krrish Dholakia
691c185ff8 feat(router.py): support region routing for bedrock, vertex ai, watsonx 2024-05-11 11:04:00 -07:00
Krrish Dholakia
0ea8222508 feat(router.py): enable filtering model group by 'allowed_model_region' 2024-05-08 22:10:17 -07:00
CyanideByte
10d9e9f180 Pydantic warning snuck in here 2024-05-07 23:59:18 -07:00
Krish Dholakia
1eb75273cf Merge branch 'main' into litellm_ui_fixes_6 2024-05-07 22:01:04 -07:00
Krrish Dholakia
ae442f895b feat(ui/model_dashboard.tsx): show if model is config or db model 2024-05-07 21:34:18 -07:00
Krrish Dholakia
b9ec7acb08 feat(model_dashboard.tsx): allow user to edit input cost per token for model on ui
also contains fixes for `/model/update`
2024-05-07 20:57:21 -07:00
Krrish Dholakia
1882ee1c4c feat(ui/model_dashboard.tsx): show if model is config or db model 2024-05-07 18:29:14 -07:00
Ishaan Jaff
708dff234d fix typo 2024-05-07 18:27:49 -07:00
Ishaan Jaff
0bdf31c4f3 router- initialize alerting 2024-05-07 18:03:04 -07:00
Ishaan Jaff
a1c7cb1306 add router alerting type 2024-05-07 17:46:18 -07:00
Ishaan Jaff
1d9e4c1051 fix support InternalServerErrorRetries 2024-05-06 17:59:38 -07:00
Ishaan Jaff
8e19215b86 fix - setting model_group_retry_policy in db 2024-05-06 17:43:43 -07:00
Krish Dholakia
fd06009199 Merge pull request #3455 from BerriAI/litellm_assistants_support
feat(openai.py): add support for openai assistants
2024-05-04 22:21:44 -07:00
Krrish Dholakia
138613616e fix(types/openai.py): fix python3.8 typing issue 2024-05-04 22:04:17 -07:00
Krrish Dholakia
0acd05a3d1 fix(typing/openai.py): fix openai typing error (version-related) 2024-05-04 22:02:43 -07:00
Krrish Dholakia
382000eb76 fix(types/openai.py): fix typing import 2024-05-04 21:53:08 -07:00
Krrish Dholakia
b0845d82cd fix(assistants/main.py): support litellm.create_thread() call 2024-05-04 19:35:37 -07:00
Ishaan Jaff
a72a6b0e9e Merge pull request #3456 from BerriAI/litellm_router_set_retry_policy_errors
[FEAT] router set custom num retries for ContentPolicyViolationErrorRetries, RateLimitErrorRetries, BadRequestErrorRetries etc
2024-05-04 18:26:03 -07:00
Ishaan Jaff
18f3de5a2d init router retry policy 2024-05-04 16:59:14 -07:00
Krish Dholakia
bc773aa052 Merge pull request #3448 from BerriAI/litellm_anthropic_fix
fix(factory.py): support 'function' openai message role for anthropic
2024-05-04 16:36:05 -07:00
Ishaan Jaff
1a96bbea64 Merge pull request #1530 from TanaroSch/main
change max_tokens type to int
2024-05-04 12:47:15 -07:00
Krrish Dholakia
d222dd8f82 fix(factory.py): support 'function' openai message role for anthropic
Fixes https://github.com/BerriAI/litellm/issues/3446
2024-05-04 10:03:30 -07:00
Krrish Dholakia
cfb6df4987 fix(main.py): support new 'supports_system_message=False' param
Fixes https://github.com/BerriAI/litellm/issues/3325
2024-05-03 21:31:45 -07:00
Krrish Dholakia
aca36d8b79 feat(proxy_server.py): return api base in response headers
Closes https://github.com/BerriAI/litellm/issues/2631
2024-05-03 15:27:32 -07:00
Krrish Dholakia
c39f8f3ef1 fix(lowest_tpm_rpm_v2.py): add more detail to 'No deployments available' error message 2024-04-29 15:04:37 -07:00
Krrish Dholakia
940ca0a4c1 fix(router.py): fix typing error 2024-04-29 07:25:39 -07:00
CyanideByte
a6c350e2bb protected_namespaces fixed for model_info 2024-04-27 13:08:45 -07:00
Krish Dholakia
26bacef87b Merge branch 'main' into litellm_default_router_retries 2024-04-27 11:21:57 -07:00
Krrish Dholakia
08e36547d6 fix(router.py): allow passing httpx.timeout to timeout param in router
Closes https://github.com/BerriAI/litellm/issues/3162
2024-04-26 14:57:19 -07:00
Krrish Dholakia
f1b2405fe0 fix(router.py): fix default retry logic 2024-04-25 11:57:27 -07:00
Krrish Dholakia
f4bd85a489 fix(proxy_server.py): fix /config/update/
allows updating router config via UI and having the change be propogated across all proxy instances by persisting config changes to the db
2024-04-24 16:42:42 -07:00