phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity
17890 commits 1603 branches 645 tags 320 MiB
Commit graph

17890 commits

This branch
This branch
All branches
Author SHA1 Message Date
John HU
8c7e357a23
Add gemini-1.5-pro-002 and gemini-1.5-flash-002 (#5879) 2024-09-25 19:31:37 -07:00
Ishaan Jaff
a8dd495eae
[Feat] add fireworks llama 3.2 models + cost tracking (#5905) 
* add fireworks llama 3.2 vision models

* add new llama3.2 models

* docs add new llama 3.2 vision models
 2024-09-25 17:59:46 -07:00
Ishaan Jaff
4bdeefd7e4
docs service accounts (#5900) 2024-09-25 15:46:13 -07:00
Ishaan Jaff
2444d76fda fix - add stricter type check for OTEL when args[0] is not dict 2024-09-25 13:48:50 -07:00
Ishaan Jaff
7cbcf538c6
[Feat] Improve OTEL Tracking - Require all Redis Cache reads to be logged on OTEL (#5881) 
* fix use previous internal usage caching logic

* fix test_dual_cache_uses_redis

* redis track event_metadata in service logging

* show otel error on _get_parent_otel_span_from_kwargs

* track parent otel span on internal usage cache

* update_request_status

* fix internal usage cache

* fix linting

* fix test internal usage cache

* fix linting error

* show event metadata in redis set

* fix test_get_team_redis

* fix test_get_team_redis

* test_proxy_logging_setup
 2024-09-25 10:57:08 -07:00
Ishaan Jaff
4ec4d02474
[Feat-Router] Allow setting which environment to use a model on (#5892) 
* add check deployment_is_active_for_environment

* add test for test_init_router_with_supported_environments

* show good example config for environments

* docs clean up config.yaml

* docs cleanup

* docs configs

* docs specfic env
 2024-09-25 10:12:06 -07:00
Krish Dholakia
4911979c61
feat(vertex): Use correct provider for response_schema support check (#5815) (#5829) 
* Fixed gemini-1.5-flash pricing

* (models): Added missing gemini experimental models + fixed pricing for gemini-1.5-pro-exp-0827

* Added gemini/gemini-1.5-flash-001 model

* Updated supports_response_schema to true for gemini flash 1.5 models

* feat(vertex): Use correct provider for response_schema support check

Co-authored-by: F1bos <44951186+F1bos@users.noreply.github.com>
 2024-09-25 08:53:21 -07:00
Ishaan Jaff
2516360ceb docs show all configs 2024-09-25 06:37:38 -07:00
Ishaan Jaff
a8bb2f476c docs show relevant litellm_settings 2024-09-25 06:36:10 -07:00
Krrish Dholakia
b2e80ecb8e docs(user_keys.md): add docs on configurable clientside auth credentials 
Allow easy switching of finetuned models
 2024-09-24 22:44:39 -07:00
Ishaan Jaff
becb4acc24 bump: version 1.48.0 → 1.48.1 2024-09-24 22:26:46 -07:00
Ishaan Jaff
b468c58fe5 chore - handle case when otel metadata field value is None 2024-09-24 22:24:12 -07:00
Ishaan Jaff
2000e8cde9
[Perf Fix] Don't always read from Redis by Default (#5877) 
* fix use previous internal usage caching logic

* fix test_dual_cache_uses_redis
 2024-09-24 21:34:18 -07:00
Ishaan Jaff
3ccdb42d26
[Fix] OTEL - Don't log messages when callback settings disable message logging (#5875) 
* fix otel dont log messages

* otel fix redis failure hook logging
 2024-09-24 18:29:52 -07:00
Krish Dholakia
d37c8b5c6b
LiteLLM Minor Fixes & Improvements (09/23/2024) (#5842) (#5858) 
* LiteLLM Minor Fixes & Improvements (09/23/2024)  (#5842)

* feat(auth_utils.py): enable admin to allow client-side credentials to be passed

Makes it easier for devs to experiment with finetuned fireworks ai models

* feat(router.py): allow setting configurable_clientside_auth_params for a model

Closes https://github.com/BerriAI/litellm/issues/5843

* build(model_prices_and_context_window.json): fix anthropic claude-3-5-sonnet max output token limit

Fixes https://github.com/BerriAI/litellm/issues/5850

* fix(azure_ai/): support content list for azure ai

Fixes https://github.com/BerriAI/litellm/issues/4237

* fix(litellm_logging.py): always set saved_cache_cost

Set to 0 by default

* fix(fireworks_ai/cost_calculator.py): add fireworks ai default pricing

handles calling 405b+ size models

* fix(slack_alerting.py): fix error alerting for failed spend tracking

Fixes regression with slack alerting error monitoring

* fix(vertex_and_google_ai_studio_gemini.py): handle gemini no candidates in streaming chunk error

* docs(bedrock.md): add llama3-1 models

* test: fix tests

* fix(azure_ai/chat): fix transformation for azure ai calls
 2024-09-24 15:01:31 -07:00
Ishaan Jaff
4df9aca45e add debian fixes to non root image 2024-09-24 07:24:59 -07:00
Ishaan Jaff
e2d48d7e45
ui add deepseek provider (#5857) 2024-09-24 07:16:51 -07:00
Jacob Hagstedt P Suorra
44708214ea
Update the dockerignore to minimise the amount of data transfered to the docker context (#5863) 
Co-authored-by: Jacob Hagstedt <wcgs@novonordisk.com>
 2024-09-24 07:16:17 -07:00
Ishaan Jaff
9a5115d5ea bump: version 1.47.3 → 1.48.0 2024-09-23 18:57:00 -07:00
Ishaan Jaff
765e3cd931 fix linting 2024-09-23 18:39:32 -07:00
Ishaan Jaff
b7cbdcdf74 fix imports 2024-09-23 18:27:41 -07:00
Ishaan Jaff
52835e5434 bump: version 1.47.2 → 1.47.3 2024-09-23 18:10:42 -07:00
Ishaan Jaff
36114f234c ui new build 2024-09-23 18:10:12 -07:00
Ishaan Jaff
4f9f505ebe
docker - handle debian issue on docker builds (#5752) 2024-09-23 17:58:22 -07:00
Ishaan Jaff
39b68b2914
[Feat] Admin UI - Add Service Accounts (#5855) 
* service_account_settings on config

* add service account checks

* call service_account_checks

* add testing for service accounts

* add service_account on ui
 2024-09-23 17:50:05 -07:00
Ishaan Jaff
391b107909
[Feat UI sso] store 'provider' in user metadata (#5856) 
* store sso provider in user metadata

* store user metadata

* store user auth_provider in user metadata

* add "metadata" for LiteLLM_UserTable

* fix sso test
 2024-09-23 17:49:36 -07:00
Ishaan Jaff
922c8ac758
[Feat-Proxy] add service accounts backend (#5852) 
* service_account_settings on config

* add service account checks

* call service_account_checks

* add testing for service accounts
 2024-09-23 16:37:02 -07:00
Ishaan Jaff
5337440ff9
[Feat] SSO - add provider in the OpenID field for custom sso (#5849) 
* service_account_settings on config

* include provider in OpenID for custom sso

* add GENERIC_PROVIDER_ATTRIBUTE to docs

* use correct naming scheme
 2024-09-23 16:34:30 -07:00
Ishaan Jaff
c19592e502 ui new build 2024-09-23 13:17:40 -07:00
Ishaan Jaff
47d36e3c79
ui networking list all teams (#5851) 2024-09-23 13:16:18 -07:00
Ishaan Jaff
d9e798ecda
[Testing-Proxy] Add E2E Admin UI testing (#5845) 
* add working ui e2e testing

* ui test

* ui playwright testing

* install python on ui testing

* add playwright testing

* fix ui testing

* fix ui testing

* add redis vars for testing

* fix playwright testing

* fix playwright testing

* rename ui testing

* move e2e ui testing
 2024-09-23 11:34:42 -07:00
Ishaan Jaff
6b9b469686 testing - nvidia nim api use mock testing 2024-09-23 08:48:13 -07:00
Ishaan Jaff
37be0088ff fix prometheus track input and output tokens (#5780) 2024-09-23 08:19:22 -07:00
Krrish Dholakia
2a8eb492a1 test(test_otel.py): fix test 2024-09-23 08:10:06 -07:00
Ishaan Jaff
030b2e1bae ui new build 2024-09-23 07:56:23 -07:00
Krrish Dholakia
16c8549b77 docs(virtual_keys.md): add enable/disable virtual keys to docs + refactor sidebar 2024-09-21 22:20:39 -07:00
Krrish Dholakia
b6bf994084 bump: version 1.47.1 → 1.47.2 2024-09-21 21:49:14 -07:00
Krish Dholakia
2488e4b45f
Cost tracking improvements (#5828) 
* feat(litellm_logging.py): update standard logging payload to include debug information for cost failures

Also includes fixes for cohere rerank cost tracking + databricks llama2 model cost tracking

 Easier to repro cost failures and improve reliability in prod

* fix(proxy_server.py): emit cost failure debug info for slack alerting

Improves debug information for cost tracking failures, on slack alerting
 2024-09-21 21:47:50 -07:00
Krish Dholakia
8039b95aaf
LiteLLM Minor Fixes & Improvements (09/21/2024) (#5819) 
* fix(router.py): fix error message

* Litellm disable keys (#5814)

* build(schema.prisma): allow blocking/unblocking keys

Fixes https://github.com/BerriAI/litellm/issues/5328

* fix(key_management_endpoints.py): fix pop

* feat(auth_checks.py): allow admin to enable/disable virtual keys

Closes https://github.com/BerriAI/litellm/issues/5328

* docs(vertex.md): add auth section for vertex ai

Addresses - https://github.com/BerriAI/litellm/issues/5768#issuecomment-2365284223

* build(model_prices_and_context_window.json): show which models support prompt_caching

Closes https://github.com/BerriAI/litellm/issues/5776

* fix(router.py): allow setting default priority for requests

* fix(router.py): add 'retry-after' header for concurrent request limit errors

Fixes https://github.com/BerriAI/litellm/issues/5783

* fix(router.py): correctly raise and use retry-after header from azure+openai

Fixes https://github.com/BerriAI/litellm/issues/5783

* fix(user_api_key_auth.py): fix valid token being none

* fix(auth_checks.py): fix model dump for cache management object

* fix(user_api_key_auth.py): pass prisma_client to obj

* test(test_otel.py): update test for new key check

* test: fix test
 2024-09-21 18:51:53 -07:00
Ishaan Jaff
1ca638973f mark test_completion_sagemaker_prompt_template_non_stream as flaky 2024-09-21 18:14:58 -07:00
Ishaan Jaff
b987d99486 handle hf rate limit error 2024-09-21 18:00:18 -07:00
Ishaan Jaff
664e3ed590 ci/cd run again 2024-09-21 17:48:34 -07:00
Ishaan Jaff
16b0d38c11
fix re-add virtual key auth checks on vertex ai pass thru endpoints (#5827) 2024-09-21 17:34:10 -07:00
Ishaan Jaff
e4f309d0e7 bump: version 1.47.0 → 1.47.1 2024-09-21 16:59:42 -07:00
Ishaan Jaff
1333ab5ac7
[fix-sso] Allow internal user viewer to view usage routes (#5825) 
* use /user/list endpoint on admin ui

* sso insert user with role when user does not exist

* add sso sign in test

* linting fix

* rename self serve doc

* add doc for self serve flow

* test - sso sign in default values

* add test for /user/list endpoint

* allow internal user viewer to view usage tab
 2024-09-21 16:58:52 -07:00
Ishaan Jaff
39e872c7eb
fix premium user check for tags on keys (#5826) 2024-09-21 16:56:33 -07:00
Ishaan Jaff
d100b32573
[SSO-UI] Set new sso users as internal_view role users (#5824) 
* use /user/list endpoint on admin ui

* sso insert user with role when user does not exist

* add sso sign in test

* linting fix

* rename self serve doc

* add doc for self serve flow

* test - sso sign in default values

* add test for /user/list endpoint
 2024-09-21 16:43:52 -07:00
Ishaan Jaff
a9caba33ef
[Feat] Allow setting custom arize endpoint (#5709) 
* set arize endpoint

* docs arize endpoint

* fix arize endpoint
 2024-09-21 13:12:00 -07:00
Ishaan Jaff
16176da9fa mark test_async_completion_azure_caching as flaky 2024-09-21 11:41:25 -07:00
Ishaan Jaff
5f1c1aaa04
[Feat] Prometheus - show status code and class type on prometheus (#5806) 
* prom - show status code and class type on prom

* log exception_class name on prometheus metrics

* prometheus track error code and status
 2024-09-21 11:36:39 -07:00