ishaan-jaff
|
ea3fe0124c
|
(feat) improve litellm.Router logging
|
2024-01-19 12:28:51 -08:00 |
|
Krrish Dholakia
|
b0a8976684
|
fix(router.py): support http and https proxys
|
2024-01-18 09:58:41 -08:00 |
|
ishaan-jaff
|
d80ae8b9bb
|
(feat) set Azure vision enhancement params using os.environ
|
2024-01-17 21:23:40 -08:00 |
|
ishaan-jaff
|
fcc387358a
|
(feat) litellm router - Azure, use base_url when set
|
2024-01-17 10:24:30 -08:00 |
|
Krrish Dholakia
|
da2af9f45e
|
fix(router.py): bump httpx pool limits
|
2024-01-11 12:51:29 +05:30 |
|
Krrish Dholakia
|
f288b12411
|
fix(lowest_latency.py): add back tpm/rpm checks, configurable time window
|
2024-01-10 20:52:01 +05:30 |
|
Krrish Dholakia
|
94ba5e46aa
|
fix(router.py): azure client init fix
|
2024-01-08 14:56:57 +05:30 |
|
Krrish Dholakia
|
cf409dd86e
|
refactor: trigger dockerbuild
|
2024-01-08 14:42:28 +05:30 |
|
Ishaan Jaff
|
d4a1f0efdd
|
Merge pull request #1356 from BerriAI/litellm_improve_proxy_logs
[Feat] Improve Proxy Logging
|
2024-01-08 14:41:01 +05:30 |
|
Krrish Dholakia
|
4e008ab09f
|
fix(router.py): increasing connection pool limits for azure router
|
2024-01-08 14:39:49 +05:30 |
|
ishaan-jaff
|
3df31ee42d
|
(feat) verbose logs + fallbacks - working well
|
2024-01-08 12:33:09 +05:30 |
|
ishaan-jaff
|
27b128ed90
|
(feat) log what model is being used as a fallback
|
2024-01-08 09:41:24 +05:30 |
|
ishaan-jaff
|
d6cc002987
|
(feat) move litellm router - to use logging.debug, logging.info
|
2024-01-08 09:31:29 +05:30 |
|
ishaan-jaff
|
c4ff2fcb4a
|
(fix) improve logging when no fallbacks found
|
2024-01-08 08:53:40 +05:30 |
|
Krrish Dholakia
|
bd3a9cba21
|
perf(router.py): don't use asyncio.wait for - just pass it to the completion call for timeouts
|
2024-01-06 17:05:55 +05:30 |
|
Krrish Dholakia
|
17c9f541a6
|
fix(router.py): don't retry malformed / content policy violating errors (400 status code)
https://github.com/BerriAI/litellm/issues/1317 , https://github.com/BerriAI/litellm/issues/1316
|
2024-01-04 22:23:51 +05:30 |
|
ishaan-jaff
|
fde5c7f158
|
(fix) proxy - cloudflare + Azure bug [non-streaming]
|
2024-01-04 10:24:51 +05:30 |
|
Krrish Dholakia
|
01c042fdc6
|
feat(router.py): add support for retry/fallbacks for async embedding calls
|
2024-01-02 11:54:28 +05:30 |
|
Krrish Dholakia
|
b82e20e517
|
fix(router.py): fix model name passed through
|
2024-01-02 11:15:30 +05:30 |
|
Krrish Dholakia
|
4f988058a1
|
refactor(test_router_caching.py): move tpm/rpm routing tests to separate file
|
2024-01-02 11:10:11 +05:30 |
|
Krrish Dholakia
|
4eae0c9a0d
|
fix(router.py): correctly raise no model available error
https://github.com/BerriAI/litellm/issues/1289
|
2024-01-01 21:22:42 +05:30 |
|
Krrish Dholakia
|
d3dee9b20c
|
test(test_lowest_latency_routing.py): add more tests
|
2023-12-30 17:41:42 +05:30 |
|
Krrish Dholakia
|
25ee96271e
|
fix(router.py): fix latency based routing
|
2023-12-30 17:25:40 +05:30 |
|
Krrish Dholakia
|
2cea8b0e83
|
fix(router.py): periodically re-initialize azure/openai clients to solve max conn issue
|
2023-12-30 15:48:34 +05:30 |
|
Krrish Dholakia
|
402d59d0ff
|
fix(lowest_tpm_rpm_routing.py): broaden scope of get deployment logic
|
2023-12-30 13:27:50 +05:30 |
|
Krrish Dholakia
|
e1925d0e29
|
fix(router.py): support retry and fallbacks for atext_completion
|
2023-12-30 11:19:32 +05:30 |
|
ishaan-jaff
|
b5e819300b
|
(feat) router, add ModelResponse type hints
|
2023-12-30 10:44:13 +05:30 |
|
Krrish Dholakia
|
a11940f4eb
|
fix(router.py): handle initial scenario for tpm/rpm routing
|
2023-12-30 07:28:45 +05:30 |
|
Krrish Dholakia
|
1933d44cbd
|
fix(router.py): fix int logic
|
2023-12-29 20:41:56 +05:30 |
|
Krrish Dholakia
|
a30f00276b
|
refactor(lowest_tpm_rpm.py): move tpm/rpm based routing to a separate file for better testing
|
2023-12-29 18:33:43 +05:30 |
|
Krrish Dholakia
|
ffe2350428
|
fix(least_busy.py): support consistent use of model id instead of deployment name
|
2023-12-29 17:05:26 +05:30 |
|
Krrish Dholakia
|
90ed04d992
|
fix(router.py): support wait_for for async completion calls
|
2023-12-29 15:27:20 +05:30 |
|
Krrish Dholakia
|
b9dd46de6c
|
feat(router.py): support 'retry_after' param, to set min timeout before retrying a failed request (default 0)
|
2023-12-29 15:18:28 +05:30 |
|
Krrish Dholakia
|
6bd6d68990
|
feat(proxy_server.py): support maxage cache control
|
2023-12-26 17:50:27 +05:30 |
|
Krrish Dholakia
|
8f3732eeeb
|
docs(user_keys.md): docs on passing user keys to litellm proxy
|
2023-12-26 13:55:28 +05:30 |
|
Krrish Dholakia
|
a4633c57c4
|
fix(router.py): accept dynamic api key
|
2023-12-26 13:16:22 +05:30 |
|
Krrish Dholakia
|
79978c44ba
|
refactor: add black formatting
|
2023-12-25 14:11:20 +05:30 |
|
Krrish Dholakia
|
d1dea7c87d
|
fix(utils.py): log user_id to langfuse
|
2023-12-23 12:14:09 +05:30 |
|
Krish Dholakia
|
a52bb3ff1c
|
Merge pull request #1182 from sumanth13131/usage-based-routing-fix
usage_based_routing_fix
|
2023-12-23 11:50:34 +05:30 |
|
Krish Dholakia
|
f509ee4ee1
|
Merge pull request #1195 from AllentDan/fix-routing
fix least_busy router by updating min_traffic
|
2023-12-23 11:45:35 +05:30 |
|
Krrish Dholakia
|
61ab8dd5c1
|
fix(proxy_server.py): handle misformatted json body in chat completion request
|
2023-12-22 12:30:36 +05:30 |
|
Krrish Dholakia
|
c084f04a35
|
fix(router.py): add support for async image generation endpoints
|
2023-12-21 14:38:44 +05:30 |
|
Krrish Dholakia
|
b873833340
|
fix(router.py): only do sync image gen fallbacks for now
The customhttptransport we use for dall-e-2 only works for sync httpx calls, not async. Will need to spend some time writing the async version
n
|
2023-12-20 19:10:59 +05:30 |
|
Krrish Dholakia
|
89b808d767
|
feat(router.py): support async image generation on router
|
2023-12-20 17:24:20 +05:30 |
|
AllentDan
|
4b84f0f049
|
fix least_busy router by updating min_traffic
|
2023-12-20 18:16:00 +08:00 |
|
sumanth
|
64dfd0d9a6
|
fix
|
2023-12-19 13:15:48 +05:30 |
|
Krrish Dholakia
|
00826e195e
|
fix(router.py): init deployment_latency_map even if model_list is empty
|
2023-12-18 17:50:26 -08:00 |
|
Krrish Dholakia
|
d9dfa3defc
|
fix(Dockerfile): support mac
|
2023-12-16 16:01:02 -08:00 |
|
Krrish Dholakia
|
68c53256ab
|
fix(proxy_server.py): setup depedencies on server startup
|
2023-12-16 11:56:11 -08:00 |
|
Krrish Dholakia
|
e76ed6be7d
|
feat(router.py): support caching groups
|
2023-12-15 21:45:51 -08:00 |
|