Ishaan Jaff
|
b8f8b6c31b
|
fix router test
|
2024-05-04 17:58:54 -07:00 |
|
Ishaan Jaff
|
f2a537d143
|
router - use retry policy
|
2024-05-04 17:04:51 -07:00 |
|
Krrish Dholakia
|
7715a9d333
|
fix(bedrock.py): convert httpx.timeout to boto3 valid timeout
Closes https://github.com/BerriAI/litellm/issues/3398
|
2024-05-03 16:24:21 -07:00 |
|
Ishaan Jaff
|
c1d156ecf8
|
fix update router logic
|
2024-05-03 08:48:11 -07:00 |
|
Krrish Dholakia
|
b22c604c8c
|
feat(router.py): add 'get_model_info' helper function to get the model info for a specific model, based on it's id
|
2024-05-02 17:53:09 -07:00 |
|
Krrish Dholakia
|
c09097d108
|
fix(router.py): fix update routing strategy
|
2024-05-01 09:51:11 -07:00 |
|
Krish Dholakia
|
ec3e35730b
|
Merge pull request #3376 from BerriAI/litellm_routing_logic
fix(router.py): unify retry timeout logic across sync + async function_with_retries
|
2024-04-30 19:58:45 -07:00 |
|
Krrish Dholakia
|
b4ec16fd5f
|
fix(test_router_fallbacks.py): fix tests
|
2024-04-30 18:48:39 -07:00 |
|
Krrish Dholakia
|
a12878b0f8
|
fix(router.py): cooldown deployments, for 401 errors
|
2024-04-30 17:54:00 -07:00 |
|
Krish Dholakia
|
6e84e2080d
|
Merge pull request #3370 from BerriAI/litellm_latency_buffer
fix(lowest_latency.py): allow setting a buffer for getting values within a certain latency threshold
|
2024-04-30 16:01:47 -07:00 |
|
Krrish Dholakia
|
3a901fc343
|
fix(router.py): return routing args as dict
|
2024-04-30 15:39:14 -07:00 |
|
Krrish Dholakia
|
b93d1381c6
|
fix(router.py): fix linting issue
|
2024-04-30 15:35:16 -07:00 |
|
Krrish Dholakia
|
66c5b5b9dd
|
fix(router.py): only check /v1 for azure ai studio models
Fixes https://github.com/BerriAI/litellm/issues/3346
|
2024-04-30 15:29:50 -07:00 |
|
Krrish Dholakia
|
17c3f110c1
|
fix(router.py): unify retry timeout logic across sync + async function_with_retries
|
2024-04-30 15:23:19 -07:00 |
|
Krrish Dholakia
|
92796d5422
|
build(ui): allow user to modify 'lowest_latency_buffer' on UI
|
2024-04-30 13:54:43 -07:00 |
|
Krrish Dholakia
|
eea8a37ec9
|
fix(router.py): raise better exception when no deployments are available
Fixes https://github.com/BerriAI/litellm/issues/3355
|
2024-04-29 18:48:04 -07:00 |
|
Krrish Dholakia
|
14f9a6f679
|
fix(router.py): fix high-traffic bug for usage-based-routing-v2
|
2024-04-29 16:48:01 -07:00 |
|
Krrish Dholakia
|
c39f8f3ef1
|
fix(lowest_tpm_rpm_v2.py): add more detail to 'No deployments available' error message
|
2024-04-29 15:04:37 -07:00 |
|
Krrish Dholakia
|
146e5d04ef
|
test: fix test - set num_retries=0
|
2024-04-27 21:02:19 -07:00 |
|
Krrish Dholakia
|
e322b8a1f8
|
fix(router.py): fix trailing slash handling for api base which contains /v1
|
2024-04-27 17:36:28 -07:00 |
|
Krrish Dholakia
|
4f49a8142f
|
fix(router.py): set initial value of default litellm params to none
|
2024-04-27 17:22:50 -07:00 |
|
Krrish Dholakia
|
500a3e1360
|
fix(router.py): fix router should retry logic
|
2024-04-27 15:59:38 -07:00 |
|
Krrish Dholakia
|
c70fbd0654
|
fix(router.py): fix router should_retry
|
2024-04-27 15:13:20 -07:00 |
|
Krrish Dholakia
|
71d63c33da
|
fix(router.py): fix sync should_retry logic
|
2024-04-27 14:48:07 -07:00 |
|
Krish Dholakia
|
26bacef87b
|
Merge branch 'main' into litellm_default_router_retries
|
2024-04-27 11:21:57 -07:00 |
|
Krrish Dholakia
|
069d1f863d
|
fix(router.py): add /v1/ if missing to base url, for openai-compatible api's
Fixes https://github.com/BerriAI/litellm/issues/2279
|
2024-04-26 17:05:07 -07:00 |
|
Krrish Dholakia
|
ca4918b9a7
|
fix(router.py): support verify_ssl flag
Fixes https://github.com/BerriAI/litellm/issues/3162#issuecomment-2075273807
|
2024-04-26 15:38:01 -07:00 |
|
Krrish Dholakia
|
f1b2405fe0
|
fix(router.py): fix default retry logic
|
2024-04-25 11:57:27 -07:00 |
|
Ishaan Jaff
|
c0b554169c
|
Revert "fix(router.py): fix max retries on set_client"
This reverts commit 821844c1a3.
|
2024-04-24 23:19:14 -07:00 |
|
Krrish Dholakia
|
399d6647e6
|
fix(router.py): fix default retry on router
|
2024-04-24 23:06:53 -07:00 |
|
Krrish Dholakia
|
2154ec624b
|
fix(router.py): fix max retries on set_client
|
2024-04-24 22:03:01 -07:00 |
|
Ishaan Jaff
|
ad637c15ce
|
Merge pull request #3283 from BerriAI/litellm_debug_lowest_latency
[Fix] Add better observability for debugging lowest latency routing
|
2024-04-24 20:42:52 -07:00 |
|
Ishaan Jaff
|
5dae1cf303
|
fix - set latency stats in kwargs
|
2024-04-24 20:13:45 -07:00 |
|
Krrish Dholakia
|
1988ce3247
|
feat(router.py): support mock testing fallbacks flag
allow user to test if fallbacks work as expected with a `mock_testing_fallbacks = True` flag set during a call
|
2024-04-24 20:13:10 -07:00 |
|
Krrish Dholakia
|
609793df83
|
feat(router.py): support mock testing fallbacks flag on router
|
2024-04-24 17:33:00 -07:00 |
|
Krrish Dholakia
|
f4bd85a489
|
fix(proxy_server.py): fix /config/update/
allows updating router config via UI and having the change be propogated across all proxy instances by persisting config changes to the db
|
2024-04-24 16:42:42 -07:00 |
|
Krrish Dholakia
|
50f5241a4e
|
build(add-fallbacks-on-UI): allows admin to add fallbacks on the UI
|
2024-04-24 15:40:02 -07:00 |
|
Ishaan Jaff
|
5ff0bad6a4
|
feat - update deployments
|
2024-04-24 09:53:42 -07:00 |
|
Ishaan Jaff
|
634139ba59
|
fix - updating router settings from DB
|
2024-04-23 12:07:58 -07:00 |
|
Krrish Dholakia
|
f1f08af785
|
fix(proxy_server.py): handle router being initialized without a model list
|
2024-04-23 10:52:28 -07:00 |
|
Krrish Dholakia
|
cce1aefdfb
|
fix(router.py): add random shuffle and tpm-based shuffle for async shuffle logic
|
2024-04-22 12:58:59 -07:00 |
|
Krrish Dholakia
|
9cf8817dad
|
fix(router.py): async simple-shuffle support
|
2024-04-20 15:01:12 -07:00 |
|
Krrish Dholakia
|
485ad73133
|
fix(router.py): improve debug logsd
|
2024-04-20 13:12:12 -07:00 |
|
Krrish Dholakia
|
9f6e90e17d
|
test(test_router_max_parallel_requests.py): more extensive testing for setting max parallel requests
|
2024-04-20 12:56:54 -07:00 |
|
Krrish Dholakia
|
a9108cbdc2
|
fix(router.py): add if router caching setup on info logs
|
2024-04-20 12:34:09 -07:00 |
|
Krrish Dholakia
|
27a32e930e
|
fix(router.py): fix init line for self.default_max_parallel_requests
|
2024-04-20 12:08:21 -07:00 |
|
Krrish Dholakia
|
22d3121f48
|
fix(router.py): calculate max_parallel_requests from given tpm limits
use the azure formula to calculate rpm -> max_parallel_requests based on a deployment's tpm limits
|
2024-04-20 10:43:18 -07:00 |
|
Krish Dholakia
|
1c6f6592ea
|
Merge pull request #3153 from BerriAI/litellm_usage_based_routing_v2_improvements
usage based routing v2 improvements - unit testing + *NEW* async + sync 'pre_call_checks'
|
2024-04-18 22:16:16 -07:00 |
|
Krrish Dholakia
|
5bb73dc9c0
|
fix(router.py): instrument pre-call-checks for all openai endpoints
|
2024-04-18 21:54:25 -07:00 |
|
Krrish Dholakia
|
376ee4e9d7
|
fix(test_lowest_tpm_rpm_routing_v2.py): unit testing for usage-based-routing-v2
|
2024-04-18 21:38:00 -07:00 |
|