Commit graph

17823 commits

Author SHA1 Message Date
Jacob Hagstedt P Suorra
44708214ea
Update the dockerignore to minimise the amount of data transfered to the docker context (#5863)
Co-authored-by: Jacob Hagstedt <wcgs@novonordisk.com>
2024-09-24 07:16:17 -07:00
Ishaan Jaff
9a5115d5ea bump: version 1.47.3 → 1.48.0 2024-09-23 18:57:00 -07:00
Ishaan Jaff
765e3cd931 fix linting 2024-09-23 18:39:32 -07:00
Ishaan Jaff
b7cbdcdf74 fix imports 2024-09-23 18:27:41 -07:00
Ishaan Jaff
52835e5434 bump: version 1.47.2 → 1.47.3 2024-09-23 18:10:42 -07:00
Ishaan Jaff
36114f234c ui new build 2024-09-23 18:10:12 -07:00
Ishaan Jaff
4f9f505ebe
docker - handle debian issue on docker builds (#5752) 2024-09-23 17:58:22 -07:00
Ishaan Jaff
39b68b2914
[Feat] Admin UI - Add Service Accounts (#5855)
* service_account_settings on config

* add service account checks

* call service_account_checks

* add testing for service accounts

* add service_account on ui
2024-09-23 17:50:05 -07:00
Ishaan Jaff
391b107909
[Feat UI sso] store 'provider' in user metadata (#5856)
* store sso provider in user metadata

* store user metadata

* store user auth_provider in user metadata

* add "metadata" for LiteLLM_UserTable

* fix sso test
2024-09-23 17:49:36 -07:00
Ishaan Jaff
922c8ac758
[Feat-Proxy] add service accounts backend (#5852)
* service_account_settings on config

* add service account checks

* call service_account_checks

* add testing for service accounts
2024-09-23 16:37:02 -07:00
Ishaan Jaff
5337440ff9
[Feat] SSO - add provider in the OpenID field for custom sso (#5849)
* service_account_settings on config

* include provider in OpenID for custom sso

* add GENERIC_PROVIDER_ATTRIBUTE to docs

* use correct naming scheme
2024-09-23 16:34:30 -07:00
Ishaan Jaff
c19592e502 ui new build 2024-09-23 13:17:40 -07:00
Ishaan Jaff
47d36e3c79
ui networking list all teams (#5851) 2024-09-23 13:16:18 -07:00
Ishaan Jaff
d9e798ecda
[Testing-Proxy] Add E2E Admin UI testing (#5845)
* add working ui e2e testing

* ui test

* ui playwright testing

* install python on ui testing

* add playwright testing

* fix ui testing

* fix ui testing

* add redis vars for testing

* fix playwright testing

* fix playwright testing

* rename ui testing

* move e2e ui testing
2024-09-23 11:34:42 -07:00
Ishaan Jaff
6b9b469686 testing - nvidia nim api use mock testing 2024-09-23 08:48:13 -07:00
Ishaan Jaff
37be0088ff fix prometheus track input and output tokens (#5780) 2024-09-23 08:19:22 -07:00
Krrish Dholakia
2a8eb492a1 test(test_otel.py): fix test 2024-09-23 08:10:06 -07:00
Ishaan Jaff
030b2e1bae ui new build 2024-09-23 07:56:23 -07:00
Krrish Dholakia
16c8549b77 docs(virtual_keys.md): add enable/disable virtual keys to docs + refactor sidebar 2024-09-21 22:20:39 -07:00
Krrish Dholakia
b6bf994084 bump: version 1.47.1 → 1.47.2 2024-09-21 21:49:14 -07:00
Krish Dholakia
2488e4b45f
Cost tracking improvements (#5828)
* feat(litellm_logging.py): update standard logging payload to include debug information for cost failures

Also includes fixes for cohere rerank cost tracking + databricks llama2 model cost tracking

 Easier to repro cost failures and improve reliability in prod

* fix(proxy_server.py): emit cost failure debug info for slack alerting

Improves debug information for cost tracking failures, on slack alerting
2024-09-21 21:47:50 -07:00
Krish Dholakia
8039b95aaf
LiteLLM Minor Fixes & Improvements (09/21/2024) (#5819)
* fix(router.py): fix error message

* Litellm disable keys (#5814)

* build(schema.prisma): allow blocking/unblocking keys

Fixes https://github.com/BerriAI/litellm/issues/5328

* fix(key_management_endpoints.py): fix pop

* feat(auth_checks.py): allow admin to enable/disable virtual keys

Closes https://github.com/BerriAI/litellm/issues/5328

* docs(vertex.md): add auth section for vertex ai

Addresses - https://github.com/BerriAI/litellm/issues/5768#issuecomment-2365284223

* build(model_prices_and_context_window.json): show which models support prompt_caching

Closes https://github.com/BerriAI/litellm/issues/5776

* fix(router.py): allow setting default priority for requests

* fix(router.py): add 'retry-after' header for concurrent request limit errors

Fixes https://github.com/BerriAI/litellm/issues/5783

* fix(router.py): correctly raise and use retry-after header from azure+openai

Fixes https://github.com/BerriAI/litellm/issues/5783

* fix(user_api_key_auth.py): fix valid token being none

* fix(auth_checks.py): fix model dump for cache management object

* fix(user_api_key_auth.py): pass prisma_client to obj

* test(test_otel.py): update test for new key check

* test: fix test
2024-09-21 18:51:53 -07:00
Ishaan Jaff
1ca638973f mark test_completion_sagemaker_prompt_template_non_stream as flaky 2024-09-21 18:14:58 -07:00
Ishaan Jaff
b987d99486 handle hf rate limit error 2024-09-21 18:00:18 -07:00
Ishaan Jaff
664e3ed590 ci/cd run again 2024-09-21 17:48:34 -07:00
Ishaan Jaff
16b0d38c11
fix re-add virtual key auth checks on vertex ai pass thru endpoints (#5827) 2024-09-21 17:34:10 -07:00
Ishaan Jaff
e4f309d0e7 bump: version 1.47.0 → 1.47.1 2024-09-21 16:59:42 -07:00
Ishaan Jaff
1333ab5ac7
[fix-sso] Allow internal user viewer to view usage routes (#5825)
* use /user/list endpoint on admin ui

* sso insert user with role when user does not exist

* add sso sign in test

* linting fix

* rename self serve doc

* add doc for self serve flow

* test - sso sign in default values

* add test for /user/list endpoint

* allow internal user viewer to view usage tab
2024-09-21 16:58:52 -07:00
Ishaan Jaff
39e872c7eb
fix premium user check for tags on keys (#5826) 2024-09-21 16:56:33 -07:00
Ishaan Jaff
d100b32573
[SSO-UI] Set new sso users as internal_view role users (#5824)
* use /user/list endpoint on admin ui

* sso insert user with role when user does not exist

* add sso sign in test

* linting fix

* rename self serve doc

* add doc for self serve flow

* test - sso sign in default values

* add test for /user/list endpoint
2024-09-21 16:43:52 -07:00
Ishaan Jaff
a9caba33ef
[Feat] Allow setting custom arize endpoint (#5709)
* set arize endpoint

* docs arize endpoint

* fix arize endpoint
2024-09-21 13:12:00 -07:00
Ishaan Jaff
16176da9fa mark test_async_completion_azure_caching as flaky 2024-09-21 11:41:25 -07:00
Ishaan Jaff
5f1c1aaa04
[Feat] Prometheus - show status code and class type on prometheus (#5806)
* prom - show status code and class type on prom

* log exception_class name on prometheus metrics

* prometheus track error code and status
2024-09-21 11:36:39 -07:00
Ishaan Jaff
711932294c
[Feat] Add testing for prometheus failure metrics (#5823)
* prom - show status code and class type on prom

* log exception_class name on prometheus metrics

* prometheus track error code and status

* add bad model

* add prometheus failure metric test

* remove outdated file

* fix litellm_proxy_total_requests_metric

* add prometheus metrics testing
2024-09-21 11:36:29 -07:00
Ishaan Jaff
1973ae8fb8
[Feat] Allow setting supports_vision for Custom OpenAI endpoints + Added testing (#5821)
* add test for using images with custom openai endpoints

* run all otel tests

* update name of test

* add custom openai model to test config

* add test for setting supports_vision=True for model

* fix test guardrails aporia

* docs supports vison

* fix yaml

* fix yaml

* docs supports vision

* fix bedrock guardrail test

* fix cohere rerank test

* update model_group doc string

* add better prints on test
2024-09-21 11:35:55 -07:00
Yurii Kostyukov
4069942dd8
Fixed DeepSeek input and output tokens (#5718)
* Fixed deepseek input and output tokens

See https://platform.deepseek.com/api-docs/quick_start/pricing/

* Returned 4096
2024-09-21 08:22:27 -07:00
superpoussin22
acfb060bf1
Correct casing (#5817)
* Update Dockerfile

correct casing

* Update Dockerfile.database

correct casing

* Update Dockerfile.alpine

correct casing

* Update Dockerfile.non_root

correct casing
2024-09-21 08:21:11 -07:00
Ishaan Jaff
1d630b61ad
[Feat] Add fireworks AI embedding (#5812)
* add fireworks embedding models

* add fireworks ai

* fireworks ai embeddings support

* is_fireworks_embedding_model

* working fireworks embeddings

* fix health check * models

* fix embedding get optional params

* fix linting errors

* fix pick_cheapest_chat_model_from_llm_provider

* add fireworks ai litellm provider

* docs fireworks embedding models

* fixes for when azure ad token  is passed
2024-09-20 22:23:28 -07:00
Krrish Dholakia
d349d501c8 docs(proxy/configs.md): add CONFIG_FILE_PATH tutorial to docs 2024-09-20 22:04:16 -07:00
Krrish Dholakia
7ca9165d59 bump: version 1.46.8 → 1.47.0 2024-09-20 21:51:18 -07:00
Krish Dholakia
7ed6938a3f
LiteLLM Minor Fixes & Improvements (09/20/2024) (#5807)
* fix(vertex_llm_base.py): Handle api_base = ""

Fixes https://github.com/BerriAI/litellm/issues/5798

* fix(o1_transformation.py): handle stream_options not being supported

https://github.com/BerriAI/litellm/issues/5803

* docs(routing.md): fix docs

Closes https://github.com/BerriAI/litellm/issues/5808

* perf(internal_user_endpoints.py): reduce db calls for getting team_alias for a key

Use the list gotten earlier in `/user/info` endpoint

 Reduces ui keys tab load time to 800ms (prev. 28s+)

* feat(proxy_server.py): support CONFIG_FILE_PATH as env var

Closes https://github.com/BerriAI/litellm/issues/5744

* feat(get_llm_provider_logic.py): add `litellm_proxy/` as a known openai-compatible route

simplifies calling litellm proxy

Reduces confusion when calling models on litellm proxy from litellm sdk

* docs(litellm_proxy.md): cleanup docs

* fix(internal_user_endpoints.py): fix pydantic obj

* test(test_key_generate_prisma.py): fix test
2024-09-20 20:21:32 -07:00
Krish Dholakia
c9ceab0f1e
refactor: cleanup root of repo (#5813) 2024-09-20 20:17:35 -07:00
Krrish Dholakia
dad3964207 build(schema.prisma): add column 'blocked' for litellm keys
enables blocking/unblocking litellm keys
2024-09-20 19:40:45 -07:00
Ishaan Jaff
cf7dcd9168
[Feat-Proxy] Allow using custom sso handler (#5809)
* update internal user doc string

* add readme on location of /sso routes

* add custom_sso_handler

* docs custom sso

* use secure=True for cookies
2024-09-20 19:14:33 -07:00
Ishaan Jaff
0a18b6539c
use .debug for update_database() (#5810) 2024-09-20 18:52:51 -07:00
Ishaan Jaff
b98b5abfb0 fix model cost map fireworks embeddings 2024-09-20 18:33:22 -07:00
Ishaan Jaff
be3fec8bfb add fireworks_ai-embedding-models 2024-09-20 17:56:58 -07:00
Ishaan Jaff
9558cbd115 add fireworks embedding pricing 2024-09-20 17:41:28 -07:00
Ishaan Jaff
036fce8f18
[Fix] Tag Based Routing not work with wildcard routing (#5805)
* allow using tag routing for free

* only enforce tags for teams / keys
2024-09-20 14:05:56 -07:00
Krish Dholakia
3933fba41f
LiteLLM Minor Fixes & Improvements (09/19/2024) (#5793)
* fix(model_prices_and_context_window.json): add cost tracking for more vertex llama3.1 model

8b and 70b models

* fix(proxy/utils.py): handle data being none on pre-call hooks

* fix(proxy/): create views on initial proxy startup

fixes base case, where user starts proxy for first time

 Fixes https://github.com/BerriAI/litellm/issues/5756

* build(config.yml): fix vertex version for test

* feat(ui/): support enabling/disabling slack alerting

Allows admin to turn on/off slack alerting through ui

* feat(rerank/main.py): support langfuse logging

* fix(proxy/utils.py): fix linting errors

* fix(langfuse.py): log clean metadata

* test(tests): replace deprecated openai model
2024-09-20 08:19:52 -07:00