Commit graph

52 commits

Author SHA1 Message Date
Ishaan Jaff
eb203c051a feat - set custom AllowedFailsPolicy 2024-06-01 17:26:21 -07:00
Krish Dholakia
8375e9621c
Merge pull request #3954 from BerriAI/litellm_simple_request_prioritization
feat(scheduler.py): add request prioritization scheduler
2024-05-31 23:29:09 -07:00
Krrish Dholakia
79287a7584 feat(scheduler.py): add request prioritization scheduler
allow user to set priority for a request
2024-05-31 18:51:13 -07:00
lj
27ed72405b
Merge branch 'main' into fix-pydantic-warnings-again 2024-05-31 11:35:42 +08:00
Ishaan Jaff
eccda76edc router - include updated at and created at in model info 2024-05-27 15:53:16 -07:00
Ishaan Jaff
b5f883ab74 feat - show openai params on model hub ui 2024-05-27 08:49:51 -07:00
Krrish Dholakia
22b6b99b34 feat(proxy_server.py): expose new /model_group/info endpoint
returns model-group level info on supported params, max tokens, pricing, etc.
2024-05-26 14:07:35 -07:00
Ishaan Jaff
be273b3c3b fix - show correct base_model in slack alerts 2024-05-17 16:07:02 -07:00
lj
f3d0f003fb
Removed config dict type definition 2024-05-17 10:39:00 +08:00
lj
7c31eccdc2
Update model config in router.py 2024-05-16 16:53:05 +08:00
Ishaan Jaff
1a67f244fb fix types/router.py 2024-05-15 19:46:30 -07:00
Krrish Dholakia
ad7e289802 fix(types/router.py): fix python3.8 typing issue 2024-05-14 16:56:07 -07:00
Krrish Dholakia
b984fba73f fix(types/router.py): fix arbitrary_types_allowed flag 2024-05-14 16:53:02 -07:00
Krrish Dholakia
e5a1050e8d fix(types/router.py): fix router pydantic v1 errors 2024-05-14 16:49:55 -07:00
Krish Dholakia
2c867ea9a5
Merge pull request #3600 from msabramo/msabramo/fix-pydantic-warnings
Update pydantic code to fix warnings
2024-05-13 22:00:39 -07:00
Krrish Dholakia
5488bf4921 feat(router.py): enable default fallbacks
allow user to define a generic list of fallbacks, in case a new deployment is bad

Closes https://github.com/BerriAI/litellm/issues/3623
2024-05-13 17:49:56 -07:00
Marc Abramowitz
e261a9b2c2 Use Config class w/ arbitrary_types_allowed for pydantic v1
pydantic v2 warns about using a Config class.
But without this, pydantic v1 will raise an error:
    RuntimeError: no validator found for <class 'openai.Timeout'>,
    see `arbitrary_types_allowed` in Config
Putting arbitrary_types_allowed = True in the ConfigDict doesn't work in pydantic v1.
So we mostly use model_config = ConfigDict(...) and then only use the Config
class with arbitrary_types_allowed = True for pydantic v1.
2024-05-13 11:22:50 -07:00
Marc Abramowitz
f233cde36c Replace some Config classes with model_config
in `litellm/types/router.py`

I didn't replace the ones that have `allow_arbitrary_types` because changing
those seems to break pydantic v1 compatibility.
2024-05-13 11:04:13 -07:00
Marc Abramowitz
eacafa7cd7 Revert litellm/types/router.py changes
These don't work in pydantic v1.
2024-05-13 10:34:05 -07:00
Marc Abramowitz
876840e995 Update pydantic code to fix warnings
Before:

```shell
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
====================================== test session starts ======================================
platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0
rootdir: /Users/abramowi/Code/OpenSource/litellm
plugins: anyio-4.3.0, asyncio-0.23.6, mock-3.14.0
asyncio: mode=Mode.STRICT
collected 12 items

litellm/tests/test_proxy_server.py s..........s                                           [100%]

======================================= warnings summary ========================================
../../../Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/pydantic/_internal/_config.py:284: 25 warnings
  /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/pydantic/_internal/_config.py:284: PydanticDeprecatedSince20: Support for class-based `config` is deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    warnings.warn(DEPRECATION_MESSAGE, DeprecationWarning)

litellm/proxy/_types.py:225
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:225: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:312
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:312: PydanticDeprecatedSince20: `pydantic.config.Extra` is deprecated, use literal values instead (e.g. `extra='allow'`). Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    extra = Extra.allow  # Allow extra fields

litellm/proxy/_types.py:315
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:315: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:344
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:344: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:391
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:391: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:460
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:460: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:480
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:480: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:493
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:493: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:538
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:538: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:575
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:575: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:870
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:870: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:897
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:897: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:918
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:918: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/utils.py:43
  /Users/abramowi/Code/OpenSource/litellm/litellm/utils.py:43: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources  # type: ignore

litellm/tests/test_proxy_server.py::test_bedrock_embedding
litellm/tests/test_proxy_server.py::test_chat_completion
litellm/tests/test_proxy_server.py::test_chat_completion_azure
litellm/tests/test_proxy_server.py::test_chat_completion_optional_params
litellm/tests/test_proxy_server.py::test_embedding
litellm/tests/test_proxy_server.py::test_engines_model_chat_completions
litellm/tests/test_proxy_server.py::test_health
litellm/tests/test_proxy_server.py::test_img_gen
litellm/tests/test_proxy_server.py::test_openai_deployments_model_chat_completions_azure
  /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead.
    warnings.warn(message, DeprecationWarning)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================== 10 passed, 2 skipped, 48 warnings in 9.83s ===========================
```

After:

```shell
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
====================================== test session starts ======================================
platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0
rootdir: /Users/abramowi/Code/OpenSource/litellm
plugins: anyio-4.3.0, asyncio-0.23.6, mock-3.14.0
asyncio: mode=Mode.STRICT
collected 12 items

litellm/tests/test_proxy_server.py s..........s                                           [100%]

======================================= warnings summary ========================================
litellm/utils.py:43
  /Users/abramowi/Code/OpenSource/litellm/litellm/utils.py:43: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources  # type: ignore

litellm/tests/test_proxy_server.py::test_bedrock_embedding
litellm/tests/test_proxy_server.py::test_chat_completion
litellm/tests/test_proxy_server.py::test_chat_completion_azure
litellm/tests/test_proxy_server.py::test_chat_completion_optional_params
litellm/tests/test_proxy_server.py::test_embedding
litellm/tests/test_proxy_server.py::test_engines_model_chat_completions
litellm/tests/test_proxy_server.py::test_health
litellm/tests/test_proxy_server.py::test_img_gen
litellm/tests/test_proxy_server.py::test_openai_deployments_model_chat_completions_azure
  /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead.
    warnings.warn(message, DeprecationWarning)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================== 10 passed, 2 skipped, 10 warnings in 9.62s ===========================
```
2024-05-12 10:11:48 -07:00
Krrish Dholakia
6714854bb7 feat(router.py): support region routing for bedrock, vertex ai, watsonx 2024-05-11 11:04:00 -07:00
Krrish Dholakia
3d18897d69 feat(router.py): enable filtering model group by 'allowed_model_region' 2024-05-08 22:10:17 -07:00
CyanideByte
b7c60031ca Pydantic warning snuck in here 2024-05-07 23:59:18 -07:00
Krish Dholakia
0e709fdc21
Merge branch 'main' into litellm_ui_fixes_6 2024-05-07 22:01:04 -07:00
Krrish Dholakia
fbcda918de feat(ui/model_dashboard.tsx): show if model is config or db model 2024-05-07 21:34:18 -07:00
Krrish Dholakia
5a16bec6a1 feat(model_dashboard.tsx): allow user to edit input cost per token for model on ui
also contains fixes for `/model/update`
2024-05-07 20:57:21 -07:00
Krrish Dholakia
312249ca44 feat(ui/model_dashboard.tsx): show if model is config or db model 2024-05-07 18:29:14 -07:00
Ishaan Jaff
dc74204427 fix typo 2024-05-07 18:27:49 -07:00
Ishaan Jaff
c08352a0ce router- initialize alerting 2024-05-07 18:03:04 -07:00
Ishaan Jaff
5fd3b12d34 add router alerting type 2024-05-07 17:46:18 -07:00
Ishaan Jaff
6a056c0c17 fix support InternalServerErrorRetries 2024-05-06 17:59:38 -07:00
Ishaan Jaff
116e555a03 fix - setting model_group_retry_policy in db 2024-05-06 17:43:43 -07:00
Krish Dholakia
6be20f5fc6
Merge pull request #3455 from BerriAI/litellm_assistants_support
feat(openai.py): add support for openai assistants
2024-05-04 22:21:44 -07:00
Krrish Dholakia
681a95e37b fix(assistants/main.py): support litellm.create_thread() call 2024-05-04 19:35:37 -07:00
Ishaan Jaff
6d1981fbaa init router retry policy 2024-05-04 16:59:14 -07:00
Krrish Dholakia
5b39f8e282 feat(proxy_server.py): return api base in response headers
Closes https://github.com/BerriAI/litellm/issues/2631
2024-05-03 15:27:32 -07:00
Krrish Dholakia
f10a066d36 fix(lowest_tpm_rpm_v2.py): add more detail to 'No deployments available' error message 2024-04-29 15:04:37 -07:00
Krrish Dholakia
0a6b6302f1 fix(router.py): fix typing error 2024-04-29 07:25:39 -07:00
CyanideByte
e1786848cb protected_namespaces fixed for model_info 2024-04-27 13:08:45 -07:00
Krish Dholakia
1a06f009d1
Merge branch 'main' into litellm_default_router_retries 2024-04-27 11:21:57 -07:00
Krrish Dholakia
7730520fb0 fix(router.py): allow passing httpx.timeout to timeout param in router
Closes https://github.com/BerriAI/litellm/issues/3162
2024-04-26 14:57:19 -07:00
Krrish Dholakia
160acc085a fix(router.py): fix default retry logic 2024-04-25 11:57:27 -07:00
Krrish Dholakia
f54510b6ee fix(proxy_server.py): fix /config/update/
allows updating router config via UI and having the change be propogated across all proxy instances by persisting config changes to the db
2024-04-24 16:42:42 -07:00
Ishaan Jaff
3d1a158b63 feat - update deployments 2024-04-24 09:53:42 -07:00
Krrish Dholakia
13cd252f3e fix(proxy_server.py): ensure id used in delete deployment matches id used in litellm Router 2024-04-16 15:17:18 -07:00
Krrish Dholakia
ea1574c160 test(test_openai_endpoints.py): add concurrency testing for user defined rate limits on proxy 2024-04-12 18:56:13 -07:00
Krrish Dholakia
a47a719caa fix(router.py): generate consistent model id's
having the same id for a deployment, lets redis usage caching work across multiple instances
2024-04-10 15:23:57 -07:00
unclecode
5554e2c359 Continue fixing the issue #2832: Add protected_namespaces to another to Config class within the router.py 2024-04-08 12:49:40 +08:00
unclecode
1ace192155 Fix issue #2832: Add protected_namespaces to Config class within utils.py, router.py and completion.py to avoid the warning message. 2024-04-08 12:43:17 +08:00
Krrish Dholakia
460546956d fix(utils.py): fix import 2024-04-06 18:37:38 -07:00