Krrish Dholakia
|
a4bea47a2d
|
fix(router.py): log rejected router requests to langfuse
Fixes issue where rejected requests weren't being logged
|
2024-06-24 17:52:01 -07:00 |
|
Krrish Dholakia
|
f5fbdf0fee
|
fix(router.py): use user-defined model_input_tokens for pre-call filter checks
|
2024-06-24 17:25:26 -07:00 |
|
Krish Dholakia
|
0454c0781a
|
Merge branch 'main' into litellm_azure_content_filter_fallbacks
|
2024-06-22 21:28:29 -07:00 |
|
Krish Dholakia
|
961e7ac95d
|
Merge branch 'main' into litellm_dynamic_tpm_limits
|
2024-06-22 19:14:59 -07:00 |
|
Krrish Dholakia
|
2c7a80d08d
|
fix(router.py): check if azure returns 'content_filter' response + fallback available -> fallback
Exception maps azure content filter response exceptions
|
2024-06-22 19:10:15 -07:00 |
|
Krrish Dholakia
|
068e8dff5b
|
feat(dynamic_rate_limiter.py): passing base case
|
2024-06-21 22:46:46 -07:00 |
|
Steven Osborn
|
0ab6a18516
|
Print content window fallbacks on startup to help verify configuration
|
2024-06-21 19:43:26 -07:00 |
|
Krrish Dholakia
|
2545da777b
|
feat(dynamic_rate_limiter.py): initial commit for dynamic rate limiting
Closes https://github.com/BerriAI/litellm/issues/4124
|
2024-06-21 18:41:31 -07:00 |
|
Krrish Dholakia
|
06b297a6e8
|
fix(router.py): fix set_client init to check if custom_llm_provider is azure not if in model name
fixes issue where 'azure_ai/' was being init as azureopenai client
|
2024-06-21 17:09:20 -07:00 |
|
Krish Dholakia
|
f86290584a
|
Merge pull request #4290 from BerriAI/litellm_specific_deployment
feat(router.py): allow user to call specific deployment via id
|
2024-06-20 20:36:13 -07:00 |
|
Krrish Dholakia
|
5729eb5168
|
fix(user_api_key_auth.py): ensure user has access to fallback models
for client side fallbacks, checks if user has access to fallback models
|
2024-06-20 16:02:19 -07:00 |
|
Ishaan Jaff
|
cdc1e952ac
|
router - add doc string
|
2024-06-20 14:36:51 -07:00 |
|
Ishaan Jaff
|
b6066d1ece
|
feat - set custom routing strategy
|
2024-06-20 13:49:44 -07:00 |
|
Krrish Dholakia
|
16889b8478
|
feat(router.py): allow user to call specific deployment via id
Allows easier health checks for specific deployments by just passing in model id
|
2024-06-19 13:02:46 -07:00 |
|
Krrish Dholakia
|
14b66c3daa
|
fix(router.py): support multiple orgs in 1 model definition
Closes https://github.com/BerriAI/litellm/issues/3949
|
2024-06-18 19:36:58 -07:00 |
|
Krrish Dholakia
|
3d9ef689e7
|
fix(vertex_httpx.py): check if model supports system messages before sending separately
|
2024-06-17 17:32:38 -07:00 |
|
Krish Dholakia
|
28a52fe5fb
|
Merge pull request #4207 from BerriAI/litellm_content_policy_fallbacks
feat(router.py): support content policy fallbacks
|
2024-06-14 18:55:11 -07:00 |
|
Krrish Dholakia
|
6f715b4782
|
feat(router.py): support content policy fallbacks
Closes https://github.com/BerriAI/litellm/issues/2632
|
2024-06-14 17:15:44 -07:00 |
|
Ishaan Jaff
|
bd5d1be1f6
|
feat - send llm exception alert on acompletion, aembedding etc
|
2024-06-14 10:11:24 -07:00 |
|
Ishaan Jaff
|
a0ecc6f414
|
fix - send alert on router level exceptions
|
2024-06-14 08:41:12 -07:00 |
|
Ishaan Jaff
|
490f5265ac
|
fix model hub not loading
|
2024-06-12 19:38:31 -07:00 |
|
Ishaan Jaff
|
d628bf0bd3
|
fix azure falbacks test
|
2024-06-10 21:50:54 -07:00 |
|
Ishaan Jaff
|
dfe874c9e5
|
test - client side fallbacks
|
2024-06-10 15:00:36 -07:00 |
|
Ishaan Jaff
|
a9006b965f
|
fix - support fallbacks as list
|
2024-06-10 14:32:28 -07:00 |
|
Krrish Dholakia
|
6306914e56
|
fix(types/router.py): modelgroupinfo to handle mode being None and supported_openai_params not being a list
|
2024-06-08 20:13:45 -07:00 |
|
Krish Dholakia
|
471be6670c
|
Merge pull request #4049 from BerriAI/litellm_cleanup_traceback
refactor: replace 'traceback.print_exc()' with logging library
|
2024-06-07 08:03:22 -07:00 |
|
Krish Dholakia
|
1742141fb6
|
Merge pull request #4046 from BerriAI/litellm_router_order
feat(router.py): enable settting 'order' for a deployment in model list
|
2024-06-06 16:37:03 -07:00 |
|
Krish Dholakia
|
677e0255c8
|
Merge branch 'main' into litellm_cleanup_traceback
|
2024-06-06 16:32:08 -07:00 |
|
Krrish Dholakia
|
b590e6607c
|
feat(scheduler.py): support redis caching for req. prioritization
enables req. prioritization to work across multiple instances of litellm
|
2024-06-06 14:19:21 -07:00 |
|
Krrish Dholakia
|
6cca5612d2
|
refactor: replace 'traceback.print_exc()' with logging library
allows error logs to be in json format for otel logging
|
2024-06-06 13:47:43 -07:00 |
|
Krrish Dholakia
|
a7dcf25722
|
feat(router.py): enable settting 'order' for a deployment in model list
Allows user to control which model gets called first in model group
|
2024-06-06 09:46:51 -07:00 |
|
Krrish Dholakia
|
f3d78532f9
|
feat(assistants/main.py): add assistants api streaming support
|
2024-06-04 16:30:35 -07:00 |
|
Krish Dholakia
|
7311e82f47
|
Merge pull request #3992 from BerriAI/litellm_router_default_request_timeout
fix(router.py): use `litellm.request_timeout` as default for router clients
|
2024-06-03 21:37:38 -07:00 |
|
Krish Dholakia
|
5ee3b0f30f
|
Merge pull request #3996 from BerriAI/litellm_azure_assistants_api_support
feat(assistants/main.py): Azure Assistants API support
|
2024-06-03 21:05:03 -07:00 |
|
Krrish Dholakia
|
7163bce37b
|
feat(assistants/main.py): Closes https://github.com/BerriAI/litellm/issues/3993
|
2024-06-03 18:47:05 -07:00 |
|
Krrish Dholakia
|
1de5235ba0
|
fix(router.py): use litellm.request_timeout as default for router clients
|
2024-06-03 14:19:53 -07:00 |
|
Krrish Dholakia
|
a019fd05e3
|
fix(router.py): fix should_retry logic for authentication errors
|
2024-06-03 13:12:00 -07:00 |
|
Ishaan Jaff
|
2ce5dc0dfd
|
ci/cd run again
|
2024-06-01 21:19:32 -07:00 |
|
Ishaan Jaff
|
309a66692f
|
fix test_rate_limit[usage-based-routing-True-3-2]
|
2024-06-01 21:18:23 -07:00 |
|
Ishaan Jaff
|
373a41ca6d
|
fix async_function_with_retries
|
2024-06-01 19:00:22 -07:00 |
|
Ishaan Jaff
|
054456c50e
|
Merge pull request #3963 from BerriAI/litellm_set_allowed_fail_policy
[FEAT]- set custom AllowedFailsPolicy on litellm.Router
|
2024-06-01 17:57:11 -07:00 |
|
Ishaan Jaff
|
fb49d036fb
|
Merge pull request #3962 from BerriAI/litellm_return_num_rets_max_exceptions
[Feat] return `num_retries` and `max_retries` in exceptions
|
2024-06-01 17:48:38 -07:00 |
|
Ishaan Jaff
|
d4378143f1
|
fix current_attempt, num_retries not defined
|
2024-06-01 17:42:37 -07:00 |
|
Ishaan Jaff
|
eb203c051a
|
feat - set custom AllowedFailsPolicy
|
2024-06-01 17:26:21 -07:00 |
|
Ishaan Jaff
|
cfc55b39a9
|
fix - return in LITELLM_EXCEPTION_TYPES
|
2024-06-01 17:05:33 -07:00 |
|
Ishaan Jaff
|
286d42a881
|
feat - add num retries and max retries in exception
|
2024-06-01 16:53:00 -07:00 |
|
Krrish Dholakia
|
7715267989
|
fix(router.py): simplify scheduler
move the scheduler poll queuing logic into the router class, making it easier to use
|
2024-06-01 16:09:57 -07:00 |
|
Krish Dholakia
|
8375e9621c
|
Merge pull request #3954 from BerriAI/litellm_simple_request_prioritization
feat(scheduler.py): add request prioritization scheduler
|
2024-05-31 23:29:09 -07:00 |
|
Krrish Dholakia
|
381247a095
|
fix(router.py): fix param
|
2024-05-31 21:52:23 -07:00 |
|
Krrish Dholakia
|
e49325b234
|
fix(router.py): fix cooldown logic for usage-based-routing-v2 pre-call-checks
|
2024-05-31 21:32:01 -07:00 |
|