Commit graph

101 commits

Author SHA1 Message Date
Ishaan Jaff
db4ebe10c8 bump litellm-proxy-extras 2025-04-19 09:14:33 -07:00
Ishaan Jaff
2b14978d9d bump litellm proxy extras 2025-04-16 19:28:16 -07:00
Ishaan Jaff
ca593e003a bump litellm-proxy-extras==0.1.9
Some checks failed
Publish Prisma Migrations / publish-migrations (push) Failing after 36s
Read Version from pyproject.toml / read-version (push) Successful in 45s
Helm unit test / unit-test (push) Successful in 54s
2025-04-15 22:49:24 -07:00
Krish Dholakia
9b77559ccf
Add aggregate team based usage logging (#10039)
* feat(schema.prisma): initial commit adding aggregate table for team spend

allows team spend to be visible at 1m+ logs

* feat(db_spend_update_writer.py): support logging aggregate team spend

allows usage dashboard to work at 1m+ logs

* feat(litellm-proxy-extras/): add new migration file

* fix(db_spend_update_writer.py): fix return type

* build: bump requirements

* fix: fix ruff error
2025-04-15 20:58:48 -07:00
Krrish Dholakia
0853b89864 build: use new litellm-proxy-extras version 2025-04-12 10:55:47 -07:00
Ishaan Jaff
fb0c3d9e18
[DB / Infra] Add new column team_member_permissions (#9941)
* add team_member_permissions to team table

* add migration.sql file

* fix poetry lock

* fix prisma migrations

* fix poetry lock

* fix migration
2025-04-12 09:06:04 -07:00
Krish Dholakia
421e0a3004
Litellm add managed files db (#9930)
* fix(openai.py): ensure openai file object shows up on logs

* fix(managed_files.py): return unified file id as b64 str

allows retrieve file id to work as expected

* fix(managed_files.py): apply decoded file id transformation

* fix: add unit test for file id + decode logic

* fix: initial commit for litellm_proxy support with CRUD Endpoints

* fix(managed_files.py): support retrieve file operation

* fix(managed_files.py): support for DELETE endpoint for files

* fix(managed_files.py): retrieve file content support

supports retrieve file content api from openai

* fix: fix linting error

* test: update tests

* fix: fix linting error

* feat(managed_files.py): support reading / writing files in DB

* feat(managed_files.py): support deleting file from DB on delete

* test: update testing

* fix(spend_tracking_utils.py): ensure each file create request is logged correctly

* fix(managed_files.py): fix storing / returning managed file object from cache

* fix(files/main.py): pass litellm params to azure route

* test: fix test

* build: add new prisma migration

* build: bump requirements

* test: add more testing

* refactor: cleanup post merge w/ main

* fix: fix code qa errors
2025-04-12 08:24:46 -07:00
Krish Dholakia
d4d3c4f697
build: bump litellm-proxy-extras version (#9771) 2025-04-05 09:02:52 -07:00
Ishaan Jaff
a771d17794 add prometheus-client to dev deps
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 17s
Helm unit test / unit-test (push) Successful in 22s
2025-04-04 22:28:45 -07:00
Adrian Lyjak
d640bc0a00
fix #8425, passthrough kwargs during acompletion, and unwrap extra_body for openrouter (#9747) 2025-04-03 22:19:40 -07:00
Krish Dholakia
62ad84fb64
UI (new_usage.tsx): Report 'total_tokens' + report success/failure calls (#9675)
* feat(internal_user_endpoints.py): return 'total_tokens' in `/user/daily/analytics`

* test(test_internal_user_endpoints.py): add unit test to assert spend metrics and dailyspend metadata always report the same fields

* build(schema.prisma): record success + failure calls to daily user table

allows understanding why model requests might exceed provider requests (e.g. user hit rate limit error)

* fix(internal_user_endpoints.py): report success / failure requests in API

* fix(proxy/utils.py): default to success

status can be missing or none at times for successful requests

* feat(new_usage.tsx): show success/failure calls on UI

* style(new_usage.tsx): ui cleanup

* fix: fix linting error

* fix: fix linting error

* feat(litellm-proxy-extras/): add new migration files
2025-03-31 22:48:43 -07:00
Krish Dholakia
1604f87663
install prisma migration files - connects litellm proxy to litellm's prisma migration files (#9637)
* build(README.md): initial commit adding a separate folder for additional proxy files. Meant to reduce size of core package

* build(litellm-proxy-extras/): new pip package for storing migration files

allows litellm proxy to use migration files, without adding them to core repo

* build(litellm-proxy-extras/): cleanup pyproject.toml

* build: move prisma migration files inside new proxy extras package

* build(run_migration.py): update script to write to correct folder

* build(proxy_cli.py): load in migration files from litellm-proxy-extras

Closes https://github.com/BerriAI/litellm/issues/9558

* build: add MIT license to litellm-proxy-extras

* test: update test

* fix: fix schema

* bump: version 0.1.0 → 0.1.1

* build(publish-proxy-extras.sh): add script for publishing new proxy-extras version

* build(liccheck.ini): add litellm-proxy-extras to authorized packages

* fix(litellm-proxy-extras/utils.py): move prisma migrate logic inside extra proxy pkg

easier since migrations folder already there

* build(pre-commit-config.yaml): add litellm_proxy_extras to ci tests

* docs(config_settings.md): document new env var

* build(pyproject.toml): bump relevant files when litellm-proxy-extras version changed

* build(pre-commit-config.yaml): run poetry check on litellm-proxy-extras as well
2025-03-29 15:27:09 -07:00
Krish Dholakia
9b7ebb6a7d
build(pyproject.toml): add new dev dependencies - for type checking (#9631)
* build(pyproject.toml): add new dev dependencies - for type checking

* build: reformat files to fit black

* ci: reformat to fit black

* ci(test-litellm.yml): make tests run clear

* build(pyproject.toml): add ruff

* fix: fix ruff checks

* build(mypy/): fix mypy linting errors

* fix(hashicorp_secret_manager.py): fix passing cert for tls auth

* build(mypy/): resolve all mypy errors

* test: update test

* fix: fix black formatting

* build(pre-commit-config.yaml): use poetry run black

* fix(proxy_server.py): fix linting error

* fix: fix ruff safe representation error
2025-03-29 11:02:13 -07:00
Krish Dholakia
0865e52db3
fix(proxy_server.py): get master key from environment, if not set in … (#9617)
* fix(proxy_server.py): get master key from environment, if not set in general settings or general settings not set at all

* test: mark flaky test

* test(test_proxy_server.py): mock prisma client

* ci: add new github workflow for testing just the mock tests

* fix: fix linting error

* ci(conftest.py): add conftest.py to isolate proxy tests

* build(pyproject.toml): add respx to dev dependencies

* build(pyproject.toml): add prisma to dev dependencies

* test: fix mock prompt management tests to use a mock anthropic key

* ci(test-litellm.yml): parallelize mock testing

make it run faster

* build(pyproject.toml): add hypercorn as dev dep

* build(pyproject.toml): separate proxy vs. core dev dependencies

make it easier for non-proxy contributors to run tests locally - e.g. no need to install hypercorn

* ci(test-litellm.yml): pin python version

* test(test_rerank.py): move test - cannot be mocked, requires aws credentials for e2e testing

* ci: add thank you message to ci

* test: add mock env var to test

* test: add autouse to tests

* test: test mock env vars for e2e tests
2025-03-28 12:32:04 -07:00
Ishaan Jaff
08a4ba1b7e Merge branch 'main' into litellm_exp_mcp_server 2025-03-24 19:03:56 -07:00
Ishaan Jaff
395e4e73d0 langchain_mcp_adapters 2025-03-24 18:59:32 -07:00
Ishaan Jaff
ab1f87bb59 add "mcp==1.5.0" to optional pyproject for litellm proxy 2025-03-24 18:56:32 -07:00
Tyler Hutcherson
7864cd1f76 update redisvl dependency 2025-03-24 08:42:11 -04:00
Ishaan Jaff
6802408057 bump gunicorn - fix security issue on gunicorn 2025-03-21 21:58:43 -07:00
Krrish Dholakia
c3ee93cd81 build(pyproject.toml): add boto3 to proxy deps on pyproject
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 18s
Helm unit test / unit-test (push) Successful in 21s
fix issue with running unit tests
2025-03-18 09:15:45 -07:00
Krrish Dholakia
ee6c9576d4 fix: bump websockets version on ci/cd 2025-03-13 14:38:22 -07:00
Tomer Bin
a8f6232eed Lock poetry after rebase 2025-03-13 08:56:19 +02:00
Krrish Dholakia
450935b6b9 build(pyproject.toml): bump openai version for responses types 2025-03-12 15:00:35 -07:00
Ishaan Jaff
ff48e574d5
fix loosen httpx restriction on pip (#8255) 2025-02-04 16:10:48 -08:00
Krish Dholakia
c17342ac5b
fix(openai/): allows 'reasoning_effort' param to be passed correctly (#8227)
* fix(openai/): allows 'reasoning_effort' param to be passed correctly

Fixes https://github.com/BerriAI/litellm/issues/8217

* test: update test to handle gemini token counter change

* fix(factory.py): fix bedrock http:// handling

* test: fix test

* test: update testing for new openai sdk
2025-02-03 22:39:10 -08:00
Krish Dholakia
8353caa485
build(pyproject.toml): bump uvicorn depedency requirement (#7773)
* build(pyproject.toml): bump uvicorn depedency requirement

Fixes https://github.com/BerriAI/litellm/issues/7768

* fix(anthropic/chat/transformation.py): fix is_vertex_request check to actually use optional param passed in

Fixes https://github.com/BerriAI/litellm/issues/6898#issuecomment-2590860695

* fix(o1_transformation.py): fix azure o1 'is_o1_model' check to just check for o1 in model string

https://github.com/BerriAI/litellm/issues/7743

* test: load vertex creds
2025-01-14 21:47:11 -08:00
Ishaan Jaff
4c608a1a32 fix uvloop critical fix 2025-01-10 13:47:18 -08:00
Ishaan Jaff
2507c275f6
(proxy perf improvement) - use uvloop for higher RPS (10%-20% higher RPS) (#7662)
* uvicorn use uvloop

* fix uvloop==0.21.0

* add uvloop to pyproject

* test_completion_response_ratelimit_headers
2025-01-09 18:11:20 -08:00
Ishaan Jaff
3158dcf88b
(Security fix) - Upgrade to fastapi==0.115.5 (#7447)
* fix upgrade fast api

* bump fastapi

* update a proxy startup tests

* remove unused test file

* update tests

* bump fast api
2024-12-28 17:08:19 -08:00
Ishaan Jaff
4f49f836aa
(security fix) - bump fast api, fastapi-sso, python-multipart - fix snyk vulnerabilities (#7417)
* bump fast api, fastapi-sso, python-multipart

* bump cryptography

* bump deps

* bump proxy_logging_guardrails_model_info_tests

* reduce num changes

* undo change

* undo change

* undo fast api change
2024-12-25 19:56:35 -08:00
Krish Dholakia
3671829e39
Complete 'requests' library removal (#7350)
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 12s
* refactor: initial commit moving watsonx_text to base_llm_http_handler + clarifying new provider directory structure

* refactor(watsonx/completion/handler.py): move to using base llm http handler

removes 'requests' library usage

* fix(watsonx_text/transformation.py): fix result transformation

migrates to transformation.py, for usage with base llm http handler

* fix(streaming_handler.py): migrate watsonx streaming to transformation.py

ensures streaming works with base llm http handler

* fix(streaming_handler.py): fix streaming linting errors and remove watsonx conditional logic

* fix(watsonx/): fix chat route post completion route refactor

* refactor(watsonx/embed): refactor watsonx to use base llm http handler for embedding calls as well

* refactor(base.py): remove requests library usage from litellm

* build(pyproject.toml): remove requests library usage

* fix: fix linting errors

* fix: fix linting errors

* fix(types/utils.py): fix validation errors for modelresponsestream

* fix(replicate/handler.py): fix linting errors

* fix(litellm_logging.py): handle modelresponsestream object

* fix(streaming_handler.py): fix modelresponsestream args

* fix: remove unused imports

* test: fix test

* fix: fix test

* test: fix test

* test: fix tests

* test: fix test

* test: fix patch target

* test: fix test
2024-12-22 07:21:25 -08:00
Ishaan Jaff
2a8d64991f
(fix) 'utf-8' codec can't encode characters error on OpenAI (#7018)
* test_openai_multilingual

* pin httpx

* fix openai pyproject

* test_multilingual_requests

* TestOpenAIChatCompletion

* fix test anthropic completion
2024-12-03 20:33:14 -08:00
Ishaan Jaff
c047d51cc8
(feat) add Predicted Outputs for OpenAI (#6594)
* bump openai to openai==1.54.0

* add 'prediction' param

* testing fix bedrock deprecated cohere.command-text-v14

* test test_openai_prediction_param.py

* test_openai_prediction_param_with_caching

* doc Predicted Outputs

* doc Predicted Output
2024-11-04 21:16:57 -08:00
Ishaan Jaff
13e0b3f626
(feat) Support audio, modalities params (#6304)
* add audio, modalities param

* add test for gpt audio models

* add get_supported_openai_params for GPT audio models

* add supported params for audio

* test_audio_output_from_model

* bump openai to openai==1.52.0

* bump openai on pyproject

* fix audio test

* fix test mock_chat_response

* handle audio for Message

* fix handling audio for OAI compatible API endpoints

* fix linting

* fix mock dbrx test
2024-10-18 19:14:25 +05:30
Ishaan Jaff
4e88fd65e1
(feat) openai prompt caching (non streaming) - add prompt_tokens_details in usage response (#6039)
* add prompt_tokens_details in usage response

* use _prompt_tokens_details as a param in Usage

* fix linting errors

* fix type error

* fix ci/cd deps

* bump deps for openai

* bump deps openai

* fix llm translation testing

* fix llm translation embedding
2024-10-03 23:31:10 +05:30
Ishaan Jaff
ded40e4d41 bump openai to 1.45.0 2024-09-12 14:18:15 -07:00
Ishaan Jaff
ac6f15ae2c fix pynacl error 2024-08-24 09:48:45 -07:00
Krrish Dholakia
1008f24b16 build(requirements.txt): bump openai version 2024-08-07 08:08:37 -07:00
dependabot[bot]
9fe5d9caa6
Bump azure-identity from 1.16.0 to 1.16.1
Bumps [azure-identity](https://github.com/Azure/azure-sdk-for-python) from 1.16.0 to 1.16.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-python/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/esrp_release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-python/compare/azure-identity_1.16.0...azure-identity_1.16.1)

---
updated-dependencies:
- dependency-name: azure-identity
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-07-12 01:28:58 +00:00
dependabot[bot]
0daa9e94a6
build(deps): bump zipp from 3.18.2 to 3.19.1
Bumps [zipp](https://github.com/jaraco/zipp) from 3.18.2 to 3.19.1.
- [Release notes](https://github.com/jaraco/zipp/releases)
- [Changelog](https://github.com/jaraco/zipp/blob/main/NEWS.rst)
- [Commits](https://github.com/jaraco/zipp/compare/v3.18.2...v3.19.1)

---
updated-dependencies:
- dependency-name: zipp
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-07-09 19:26:57 +00:00
Ishaan Jaff
f4eb04640c bump: version 1.41.9 → 1.41.10 2024-07-06 15:30:27 -07:00
James Braza
a30157d659 Ran 'poetry lock --no-update' 2024-07-02 13:02:07 -04:00
Ishaan Jaff
6091c7798e use cache tools as dep 2024-06-24 17:31:59 -07:00
Krrish Dholakia
40ca30813e test(test_python_38.py): add coverage for non-gen settings config.yaml flow
Adds testing for this https://github.com/BerriAI/litellm/issues/4324
2024-06-20 20:29:10 -07:00
dependabot[bot]
ee729a5d1d
build(deps): bump urllib3 from 2.2.1 to 2.2.2
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.2.1 to 2.2.2.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.2.1...2.2.2)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-17 23:06:47 +00:00
Krrish Dholakia
478d8c8200 build(poetry.lock): updating poetry.lock 2024-06-15 11:03:16 -07:00
James Braza
917e8678db Added mypy to the dev Poetry group 2024-06-11 14:43:32 -07:00
sha-ahammed
575d94f5ef fix: poetry lock --no-update 2024-06-06 21:09:13 +05:30
sha-ahammed
76aab91d63 Revert "fix: Authentication Error, No module named 'nacl'."
This reverts commit 499933f943.
2024-06-06 20:56:16 +05:30
sha-ahammed
499933f943 fix: Authentication Error, No module named 'nacl'. 2024-06-05 16:46:27 +05:30