forked from phoenix/litellm-mirror
876840e995
Before: ```shell $ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py ====================================== test session starts ====================================== platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0 rootdir: /Users/abramowi/Code/OpenSource/litellm plugins: anyio-4.3.0, asyncio-0.23.6, mock-3.14.0 asyncio: mode=Mode.STRICT collected 12 items litellm/tests/test_proxy_server.py s..........s [100%] ======================================= warnings summary ======================================== ../../../Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/pydantic/_internal/_config.py:284: 25 warnings /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/pydantic/_internal/_config.py:284: PydanticDeprecatedSince20: Support for class-based `config` is deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ warnings.warn(DEPRECATION_MESSAGE, DeprecationWarning) litellm/proxy/_types.py:225 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:225: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:312 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:312: PydanticDeprecatedSince20: `pydantic.config.Extra` is deprecated, use literal values instead (e.g. `extra='allow'`). Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ extra = Extra.allow # Allow extra fields litellm/proxy/_types.py:315 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:315: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:344 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:344: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:391 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:391: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:460 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:460: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:480 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:480: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:493 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:493: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:538 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:538: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:575 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:575: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:870 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:870: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:897 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:897: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:918 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:918: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/utils.py:43 /Users/abramowi/Code/OpenSource/litellm/litellm/utils.py:43: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html import pkg_resources # type: ignore litellm/tests/test_proxy_server.py::test_bedrock_embedding litellm/tests/test_proxy_server.py::test_chat_completion litellm/tests/test_proxy_server.py::test_chat_completion_azure litellm/tests/test_proxy_server.py::test_chat_completion_optional_params litellm/tests/test_proxy_server.py::test_embedding litellm/tests/test_proxy_server.py::test_engines_model_chat_completions litellm/tests/test_proxy_server.py::test_health litellm/tests/test_proxy_server.py::test_img_gen litellm/tests/test_proxy_server.py::test_openai_deployments_model_chat_completions_azure /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead. warnings.warn(message, DeprecationWarning) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ========================== 10 passed, 2 skipped, 48 warnings in 9.83s =========================== ``` After: ```shell $ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py ====================================== test session starts ====================================== platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0 rootdir: /Users/abramowi/Code/OpenSource/litellm plugins: anyio-4.3.0, asyncio-0.23.6, mock-3.14.0 asyncio: mode=Mode.STRICT collected 12 items litellm/tests/test_proxy_server.py s..........s [100%] ======================================= warnings summary ======================================== litellm/utils.py:43 /Users/abramowi/Code/OpenSource/litellm/litellm/utils.py:43: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html import pkg_resources # type: ignore litellm/tests/test_proxy_server.py::test_bedrock_embedding litellm/tests/test_proxy_server.py::test_chat_completion litellm/tests/test_proxy_server.py::test_chat_completion_azure litellm/tests/test_proxy_server.py::test_chat_completion_optional_params litellm/tests/test_proxy_server.py::test_embedding litellm/tests/test_proxy_server.py::test_engines_model_chat_completions litellm/tests/test_proxy_server.py::test_health litellm/tests/test_proxy_server.py::test_img_gen litellm/tests/test_proxy_server.py::test_openai_deployments_model_chat_completions_azure /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead. warnings.warn(message, DeprecationWarning) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ========================== 10 passed, 2 skipped, 10 warnings in 9.62s =========================== ``` |
||
|---|---|---|
| .. | ||
| _experimental | ||
| auth | ||
| db | ||
| example_config_yaml | ||
| hooks | ||
| proxy_load_test | ||
| queue | ||
| secret_managers | ||
| tests | ||
| .gitignore | ||
| __init__.py | ||
| _new_secret_config.yaml | ||
| _super_secret_config.yaml | ||
| _types.py | ||
| admin_ui.py | ||
| cached_logo.jpg | ||
| custom_callbacks.py | ||
| enterprise | ||
| health_check.py | ||
| lambda.py | ||
| llamaguard_prompt.txt | ||
| logo.jpg | ||
| openapi.json | ||
| otel_config.yaml | ||
| post_call_rules.py | ||
| proxy_cli.py | ||
| proxy_config.yaml | ||
| proxy_server.py | ||
| README.md | ||
| schema.prisma | ||
| start.sh | ||
| utils.py | ||
litellm-proxy
A local, fast, and lightweight OpenAI-compatible server to call 100+ LLM APIs.
usage
$ pip install litellm
$ litellm --model ollama/codellama
#INFO: Ollama running on http://0.0.0.0:8000
replace openai base
import openai # openai v1.0.0+
client = openai.OpenAI(api_key="anything",base_url="http://0.0.0.0:8000") # set proxy to base_url
# request sent to model set on litellm proxy, `litellm --model`
response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [
{
"role": "user",
"content": "this is a test request, write a short poem"
}
])
print(response)
See how to call Huggingface,Bedrock,TogetherAI,Anthropic, etc.