Commit graph

96 commits

Author SHA1 Message Date
Krish Dholakia
ba03736077
Merge branch 'main' into litellm_contributor_prs_03_24_2025_p1 2025-04-09 22:42:36 -07:00
Krish Dholakia
d4d3c4f697
build: bump litellm-proxy-extras version (#9771) 2025-04-05 09:02:52 -07:00
Ishaan Jaff
a771d17794 add prometheus-client to dev deps
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 17s
Helm unit test / unit-test (push) Successful in 22s
2025-04-04 22:28:45 -07:00
Adrian Lyjak
d640bc0a00
fix #8425, passthrough kwargs during acompletion, and unwrap extra_body for openrouter (#9747) 2025-04-03 22:19:40 -07:00
Krish Dholakia
62ad84fb64
UI (new_usage.tsx): Report 'total_tokens' + report success/failure calls (#9675)
* feat(internal_user_endpoints.py): return 'total_tokens' in `/user/daily/analytics`

* test(test_internal_user_endpoints.py): add unit test to assert spend metrics and dailyspend metadata always report the same fields

* build(schema.prisma): record success + failure calls to daily user table

allows understanding why model requests might exceed provider requests (e.g. user hit rate limit error)

* fix(internal_user_endpoints.py): report success / failure requests in API

* fix(proxy/utils.py): default to success

status can be missing or none at times for successful requests

* feat(new_usage.tsx): show success/failure calls on UI

* style(new_usage.tsx): ui cleanup

* fix: fix linting error

* fix: fix linting error

* feat(litellm-proxy-extras/): add new migration files
2025-03-31 22:48:43 -07:00
Krish Dholakia
1604f87663
install prisma migration files - connects litellm proxy to litellm's prisma migration files (#9637)
* build(README.md): initial commit adding a separate folder for additional proxy files. Meant to reduce size of core package

* build(litellm-proxy-extras/): new pip package for storing migration files

allows litellm proxy to use migration files, without adding them to core repo

* build(litellm-proxy-extras/): cleanup pyproject.toml

* build: move prisma migration files inside new proxy extras package

* build(run_migration.py): update script to write to correct folder

* build(proxy_cli.py): load in migration files from litellm-proxy-extras

Closes https://github.com/BerriAI/litellm/issues/9558

* build: add MIT license to litellm-proxy-extras

* test: update test

* fix: fix schema

* bump: version 0.1.0 → 0.1.1

* build(publish-proxy-extras.sh): add script for publishing new proxy-extras version

* build(liccheck.ini): add litellm-proxy-extras to authorized packages

* fix(litellm-proxy-extras/utils.py): move prisma migrate logic inside extra proxy pkg

easier since migrations folder already there

* build(pre-commit-config.yaml): add litellm_proxy_extras to ci tests

* docs(config_settings.md): document new env var

* build(pyproject.toml): bump relevant files when litellm-proxy-extras version changed

* build(pre-commit-config.yaml): run poetry check on litellm-proxy-extras as well
2025-03-29 15:27:09 -07:00
Krish Dholakia
9b7ebb6a7d
build(pyproject.toml): add new dev dependencies - for type checking (#9631)
* build(pyproject.toml): add new dev dependencies - for type checking

* build: reformat files to fit black

* ci: reformat to fit black

* ci(test-litellm.yml): make tests run clear

* build(pyproject.toml): add ruff

* fix: fix ruff checks

* build(mypy/): fix mypy linting errors

* fix(hashicorp_secret_manager.py): fix passing cert for tls auth

* build(mypy/): resolve all mypy errors

* test: update test

* fix: fix black formatting

* build(pre-commit-config.yaml): use poetry run black

* fix(proxy_server.py): fix linting error

* fix: fix ruff safe representation error
2025-03-29 11:02:13 -07:00
Krish Dholakia
0865e52db3
fix(proxy_server.py): get master key from environment, if not set in … (#9617)
* fix(proxy_server.py): get master key from environment, if not set in general settings or general settings not set at all

* test: mark flaky test

* test(test_proxy_server.py): mock prisma client

* ci: add new github workflow for testing just the mock tests

* fix: fix linting error

* ci(conftest.py): add conftest.py to isolate proxy tests

* build(pyproject.toml): add respx to dev dependencies

* build(pyproject.toml): add prisma to dev dependencies

* test: fix mock prompt management tests to use a mock anthropic key

* ci(test-litellm.yml): parallelize mock testing

make it run faster

* build(pyproject.toml): add hypercorn as dev dep

* build(pyproject.toml): separate proxy vs. core dev dependencies

make it easier for non-proxy contributors to run tests locally - e.g. no need to install hypercorn

* ci(test-litellm.yml): pin python version

* test(test_rerank.py): move test - cannot be mocked, requires aws credentials for e2e testing

* ci: add thank you message to ci

* test: add mock env var to test

* test: add autouse to tests

* test: test mock env vars for e2e tests
2025-03-28 12:32:04 -07:00
Jason
7a92a03565
Add new model provider Novita AI (#7582)
* feat: add new model provider Novita AI

* feat: use deepseek r1 model for examples in Novita AI docs

* fix: fix tests

* fix: fix tests for novita

* fix: fix novita transformation
2025-03-25 09:03:46 -07:00
Ishaan Jaff
08a4ba1b7e Merge branch 'main' into litellm_exp_mcp_server 2025-03-24 19:03:56 -07:00
Ishaan Jaff
395e4e73d0 langchain_mcp_adapters 2025-03-24 18:59:32 -07:00
Ishaan Jaff
ab1f87bb59 add "mcp==1.5.0" to optional pyproject for litellm proxy 2025-03-24 18:56:32 -07:00
Tyler Hutcherson
7864cd1f76 update redisvl dependency 2025-03-24 08:42:11 -04:00
Ishaan Jaff
6802408057 bump gunicorn - fix security issue on gunicorn 2025-03-21 21:58:43 -07:00
Krrish Dholakia
c3ee93cd81 build(pyproject.toml): add boto3 to proxy deps on pyproject
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 18s
Helm unit test / unit-test (push) Successful in 21s
fix issue with running unit tests
2025-03-18 09:15:45 -07:00
Krrish Dholakia
ee6c9576d4 fix: bump websockets version on ci/cd 2025-03-13 14:38:22 -07:00
Tomer Bin
a8f6232eed Lock poetry after rebase 2025-03-13 08:56:19 +02:00
Krrish Dholakia
450935b6b9 build(pyproject.toml): bump openai version for responses types 2025-03-12 15:00:35 -07:00
Ishaan Jaff
ff48e574d5
fix loosen httpx restriction on pip (#8255) 2025-02-04 16:10:48 -08:00
Krish Dholakia
c17342ac5b
fix(openai/): allows 'reasoning_effort' param to be passed correctly (#8227)
* fix(openai/): allows 'reasoning_effort' param to be passed correctly

Fixes https://github.com/BerriAI/litellm/issues/8217

* test: update test to handle gemini token counter change

* fix(factory.py): fix bedrock http:// handling

* test: fix test

* test: update testing for new openai sdk
2025-02-03 22:39:10 -08:00
Krish Dholakia
8353caa485
build(pyproject.toml): bump uvicorn depedency requirement (#7773)
* build(pyproject.toml): bump uvicorn depedency requirement

Fixes https://github.com/BerriAI/litellm/issues/7768

* fix(anthropic/chat/transformation.py): fix is_vertex_request check to actually use optional param passed in

Fixes https://github.com/BerriAI/litellm/issues/6898#issuecomment-2590860695

* fix(o1_transformation.py): fix azure o1 'is_o1_model' check to just check for o1 in model string

https://github.com/BerriAI/litellm/issues/7743

* test: load vertex creds
2025-01-14 21:47:11 -08:00
Ishaan Jaff
4c608a1a32 fix uvloop critical fix 2025-01-10 13:47:18 -08:00
Ishaan Jaff
2507c275f6
(proxy perf improvement) - use uvloop for higher RPS (10%-20% higher RPS) (#7662)
* uvicorn use uvloop

* fix uvloop==0.21.0

* add uvloop to pyproject

* test_completion_response_ratelimit_headers
2025-01-09 18:11:20 -08:00
Ishaan Jaff
3158dcf88b
(Security fix) - Upgrade to fastapi==0.115.5 (#7447)
* fix upgrade fast api

* bump fastapi

* update a proxy startup tests

* remove unused test file

* update tests

* bump fast api
2024-12-28 17:08:19 -08:00
Ishaan Jaff
4f49f836aa
(security fix) - bump fast api, fastapi-sso, python-multipart - fix snyk vulnerabilities (#7417)
* bump fast api, fastapi-sso, python-multipart

* bump cryptography

* bump deps

* bump proxy_logging_guardrails_model_info_tests

* reduce num changes

* undo change

* undo change

* undo fast api change
2024-12-25 19:56:35 -08:00
Krish Dholakia
3671829e39
Complete 'requests' library removal (#7350)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 12s
* refactor: initial commit moving watsonx_text to base_llm_http_handler + clarifying new provider directory structure

* refactor(watsonx/completion/handler.py): move to using base llm http handler

removes 'requests' library usage

* fix(watsonx_text/transformation.py): fix result transformation

migrates to transformation.py, for usage with base llm http handler

* fix(streaming_handler.py): migrate watsonx streaming to transformation.py

ensures streaming works with base llm http handler

* fix(streaming_handler.py): fix streaming linting errors and remove watsonx conditional logic

* fix(watsonx/): fix chat route post completion route refactor

* refactor(watsonx/embed): refactor watsonx to use base llm http handler for embedding calls as well

* refactor(base.py): remove requests library usage from litellm

* build(pyproject.toml): remove requests library usage

* fix: fix linting errors

* fix: fix linting errors

* fix(types/utils.py): fix validation errors for modelresponsestream

* fix(replicate/handler.py): fix linting errors

* fix(litellm_logging.py): handle modelresponsestream object

* fix(streaming_handler.py): fix modelresponsestream args

* fix: remove unused imports

* test: fix test

* fix: fix test

* test: fix test

* test: fix tests

* test: fix test

* test: fix patch target

* test: fix test
2024-12-22 07:21:25 -08:00
Ishaan Jaff
2a8d64991f
(fix) 'utf-8' codec can't encode characters error on OpenAI (#7018)
* test_openai_multilingual

* pin httpx

* fix openai pyproject

* test_multilingual_requests

* TestOpenAIChatCompletion

* fix test anthropic completion
2024-12-03 20:33:14 -08:00
Ishaan Jaff
c047d51cc8
(feat) add Predicted Outputs for OpenAI (#6594)
* bump openai to openai==1.54.0

* add 'prediction' param

* testing fix bedrock deprecated cohere.command-text-v14

* test test_openai_prediction_param.py

* test_openai_prediction_param_with_caching

* doc Predicted Outputs

* doc Predicted Output
2024-11-04 21:16:57 -08:00
Ishaan Jaff
13e0b3f626
(feat) Support audio, modalities params (#6304)
* add audio, modalities param

* add test for gpt audio models

* add get_supported_openai_params for GPT audio models

* add supported params for audio

* test_audio_output_from_model

* bump openai to openai==1.52.0

* bump openai on pyproject

* fix audio test

* fix test mock_chat_response

* handle audio for Message

* fix handling audio for OAI compatible API endpoints

* fix linting

* fix mock dbrx test
2024-10-18 19:14:25 +05:30
Ishaan Jaff
4e88fd65e1
(feat) openai prompt caching (non streaming) - add prompt_tokens_details in usage response (#6039)
* add prompt_tokens_details in usage response

* use _prompt_tokens_details as a param in Usage

* fix linting errors

* fix type error

* fix ci/cd deps

* bump deps for openai

* bump deps openai

* fix llm translation testing

* fix llm translation embedding
2024-10-03 23:31:10 +05:30
Ishaan Jaff
ded40e4d41 bump openai to 1.45.0 2024-09-12 14:18:15 -07:00
Ishaan Jaff
ac6f15ae2c fix pynacl error 2024-08-24 09:48:45 -07:00
Krrish Dholakia
1008f24b16 build(requirements.txt): bump openai version 2024-08-07 08:08:37 -07:00
dependabot[bot]
9fe5d9caa6
Bump azure-identity from 1.16.0 to 1.16.1
Bumps [azure-identity](https://github.com/Azure/azure-sdk-for-python) from 1.16.0 to 1.16.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-python/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/esrp_release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-python/compare/azure-identity_1.16.0...azure-identity_1.16.1)

---
updated-dependencies:
- dependency-name: azure-identity
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-07-12 01:28:58 +00:00
dependabot[bot]
0daa9e94a6
build(deps): bump zipp from 3.18.2 to 3.19.1
Bumps [zipp](https://github.com/jaraco/zipp) from 3.18.2 to 3.19.1.
- [Release notes](https://github.com/jaraco/zipp/releases)
- [Changelog](https://github.com/jaraco/zipp/blob/main/NEWS.rst)
- [Commits](https://github.com/jaraco/zipp/compare/v3.18.2...v3.19.1)

---
updated-dependencies:
- dependency-name: zipp
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-07-09 19:26:57 +00:00
Ishaan Jaff
f4eb04640c bump: version 1.41.9 → 1.41.10 2024-07-06 15:30:27 -07:00
James Braza
a30157d659 Ran 'poetry lock --no-update' 2024-07-02 13:02:07 -04:00
Ishaan Jaff
6091c7798e use cache tools as dep 2024-06-24 17:31:59 -07:00
Krrish Dholakia
40ca30813e test(test_python_38.py): add coverage for non-gen settings config.yaml flow
Adds testing for this https://github.com/BerriAI/litellm/issues/4324
2024-06-20 20:29:10 -07:00
dependabot[bot]
ee729a5d1d
build(deps): bump urllib3 from 2.2.1 to 2.2.2
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.2.1 to 2.2.2.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.2.1...2.2.2)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-17 23:06:47 +00:00
Krrish Dholakia
478d8c8200 build(poetry.lock): updating poetry.lock 2024-06-15 11:03:16 -07:00
James Braza
917e8678db Added mypy to the dev Poetry group 2024-06-11 14:43:32 -07:00
sha-ahammed
575d94f5ef fix: poetry lock --no-update 2024-06-06 21:09:13 +05:30
sha-ahammed
76aab91d63 Revert "fix: Authentication Error, No module named 'nacl'."
This reverts commit 499933f943.
2024-06-06 20:56:16 +05:30
sha-ahammed
499933f943 fix: Authentication Error, No module named 'nacl'. 2024-06-05 16:46:27 +05:30
yujonglee
2d2671eea9 deps 2024-06-02 19:54:43 +09:00
yujonglee
d464b97810 init 2024-06-02 13:53:43 +09:00
Ishaan Jaff
25812d3573 bump openai version 2024-05-21 11:33:57 -07:00
Ishaan Jaff
c9fd6dbb22 feat - bump OpenAI version 2024-05-21 11:32:33 -07:00
Marc Abramowitz
82f43f98b3 Update FastAPI to update starlette to fix warnings
The httpx package emits some warnings when we run the tests in
`test_proxy_server.py`:

```shell
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
...
litellm/tests/test_proxy_server.py::test_bedrock_embedding
litellm/tests/test_proxy_server.py::test_chat_completion
litellm/tests/test_proxy_server.py::test_chat_completion_azure
litellm/tests/test_proxy_server.py::test_chat_completion_optional_params
litellm/tests/test_proxy_server.py::test_embedding
litellm/tests/test_proxy_server.py::test_engines_model_chat_completions
litellm/tests/test_proxy_server.py::test_health
litellm/tests/test_proxy_server.py::test_img_gen
litellm/tests/test_proxy_server.py::test_openai_deployments_model_chat_completions_azure
  .../site-packages/httpx/_client.py:680:
  DeprecationWarning: The 'app' shortcut is now deprecated.
  Use the explicit style 'transport=WSGITransport(app=...)' instead.
    warnings.warn(message, DeprecationWarning)
...
```

Fixing this requires updating starlette, but the version of FastAPI in use
prevents that. So I updated FastAPI to update starlette to fix the httpx
warnings.

When run in combination with the changes in GH-3600, the tests pass and only
emit 1 warning!

```shell
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
====================================== test session starts ======================================
platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0
rootdir: /Users/abramowi/Code/OpenSource/litellm
plugins: anyio-4.3.0, mock-3.14.0
collected 12 items

litellm/tests/test_proxy_server.py s..........s                                           [100%]

======================================= warnings summary ========================================
litellm/utils.py:43
  /Users/abramowi/Code/OpenSource/litellm/litellm/utils.py:43: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources  # type: ignore

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== 10 passed, 2 skipped, 1 warning in 8.81s ============================
```
2024-05-12 12:26:23 -07:00