Krrish Dholakia
|
32d1d64b63
|
refactor(lowest_latency.py): fix linting issue
|
2024-01-09 12:59:42 +05:30 |
|
Krrish Dholakia
|
832c10b402
|
feat(lowest_latency.py): support expanded time window for latency based routing
uses a 1hr avg. of latency for deployments, to determine which to route to
https://github.com/BerriAI/litellm/issues/1361
|
2024-01-09 12:59:42 +05:30 |
|
Krrish Dholakia
|
9dc2bc227b
|
refactor(lowest_latency.py): fix linting error
|
2024-01-09 12:58:58 +05:30 |
|
Krrish Dholakia
|
9ddcdc4716
|
feat(lowest_latency.py): support expanded time window for latency based routing
uses a 1hr avg. of latency for deployments, to determine which to route to
https://github.com/BerriAI/litellm/issues/1361
|
2024-01-09 12:58:58 +05:30 |
|
Krrish Dholakia
|
2ab31bcaf8
|
fix(lowest_tpm_rpm.py): handle null case for text/message input
|
2024-01-02 12:24:29 +05:30 |
|
Krrish Dholakia
|
a37a18ca80
|
feat(router.py): add support for retry/fallbacks for async embedding calls
|
2024-01-02 11:54:28 +05:30 |
|
Krrish Dholakia
|
dff4c172d0
|
refactor(test_router_caching.py): move tpm/rpm routing tests to separate file
|
2024-01-02 11:10:11 +05:30 |
|
Krrish Dholakia
|
a83e2e07cf
|
fix(router.py): correctly raise no model available error
https://github.com/BerriAI/litellm/issues/1289
|
2024-01-01 21:22:42 +05:30 |
|
Krrish Dholakia
|
027218c3f0
|
test(test_lowest_latency_routing.py): add more tests
|
2023-12-30 17:41:42 +05:30 |
|
Krrish Dholakia
|
f2d0d5584a
|
fix(router.py): fix latency based routing
|
2023-12-30 17:25:40 +05:30 |
|
Krrish Dholakia
|
b66cf0aa43
|
fix(lowest_tpm_rpm_routing.py): broaden scope of get deployment logic
|
2023-12-30 13:27:50 +05:30 |
|
Krrish Dholakia
|
38f55249e1
|
fix(router.py): support retry and fallbacks for atext_completion
|
2023-12-30 11:19:32 +05:30 |
|
Krrish Dholakia
|
a34de56289
|
fix(router.py): handle initial scenario for tpm/rpm routing
|
2023-12-30 07:28:45 +05:30 |
|
Krrish Dholakia
|
2fc264ca04
|
fix(router.py): fix int logic
|
2023-12-29 20:41:56 +05:30 |
|
Krrish Dholakia
|
cf91e49c87
|
refactor(lowest_tpm_rpm.py): move tpm/rpm based routing to a separate file for better testing
|
2023-12-29 18:33:43 +05:30 |
|
Krrish Dholakia
|
54d7bc2cc3
|
test(test_least_busy_router.py): add better testing for least busy routing
|
2023-12-29 17:16:00 +05:30 |
|
Krrish Dholakia
|
678bbfa9be
|
fix(least_busy.py): support consistent use of model id instead of deployment name
|
2023-12-29 17:05:26 +05:30 |
|
Krrish Dholakia
|
4905929de3
|
refactor: add black formatting
|
2023-12-25 14:11:20 +05:30 |
|
Krrish Dholakia
|
4bf875d3ed
|
fix(router.py): fix least-busy routing
|
2023-12-08 20:29:49 -08:00 |
|