phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-26 19:24:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
21223 commits 2490 branches 894 tags 548 MiB
Commit graph

681 commits

Author SHA1 Message Date
Krrish Dholakia
cf409dd86e refactor: trigger dockerbuild 2024-01-08 14:42:28 +05:30
Ishaan Jaff
d4a1f0efdd Merge pull request #1356 from BerriAI/litellm_improve_proxy_logs 
[Feat] Improve Proxy Logging
 2024-01-08 14:41:01 +05:30
Krrish Dholakia
4e008ab09f fix(router.py): increasing connection pool limits for azure router 2024-01-08 14:39:49 +05:30
ishaan-jaff
3df31ee42d (feat) verbose logs + fallbacks - working well 2024-01-08 12:33:09 +05:30
ishaan-jaff
27b128ed90 (feat) log what model is being used as a fallback 2024-01-08 09:41:24 +05:30
ishaan-jaff
d6cc002987 (feat) move litellm router - to use logging.debug, logging.info 2024-01-08 09:31:29 +05:30
ishaan-jaff
c4ff2fcb4a (fix) improve logging when no fallbacks found 2024-01-08 08:53:40 +05:30
Krrish Dholakia
bd3a9cba21 perf(router.py): don't use asyncio.wait for - just pass it to the completion call for timeouts 2024-01-06 17:05:55 +05:30
Krrish Dholakia
17c9f541a6 fix(router.py): don't retry malformed / content policy violating errors (400 status code) 
https://github.com/BerriAI/litellm/issues/1317 , https://github.com/BerriAI/litellm/issues/1316
 2024-01-04 22:23:51 +05:30
ishaan-jaff
fde5c7f158 (fix) proxy - cloudflare + Azure bug [non-streaming] 2024-01-04 10:24:51 +05:30
Krrish Dholakia
01c042fdc6 feat(router.py): add support for retry/fallbacks for async embedding calls 2024-01-02 11:54:28 +05:30
Krrish Dholakia
b82e20e517 fix(router.py): fix model name passed through 2024-01-02 11:15:30 +05:30
Krrish Dholakia
4f988058a1 refactor(test_router_caching.py): move tpm/rpm routing tests to separate file 2024-01-02 11:10:11 +05:30
Krrish Dholakia
4eae0c9a0d fix(router.py): correctly raise no model available error 
https://github.com/BerriAI/litellm/issues/1289
 2024-01-01 21:22:42 +05:30
Krrish Dholakia
d3dee9b20c test(test_lowest_latency_routing.py): add more tests 2023-12-30 17:41:42 +05:30
Krrish Dholakia
25ee96271e fix(router.py): fix latency based routing 2023-12-30 17:25:40 +05:30
Krrish Dholakia
2cea8b0e83 fix(router.py): periodically re-initialize azure/openai clients to solve max conn issue 2023-12-30 15:48:34 +05:30
Krrish Dholakia
402d59d0ff fix(lowest_tpm_rpm_routing.py): broaden scope of get deployment logic 2023-12-30 13:27:50 +05:30
Krrish Dholakia
e1925d0e29 fix(router.py): support retry and fallbacks for atext_completion 2023-12-30 11:19:32 +05:30
ishaan-jaff
b5e819300b (feat) router, add ModelResponse type hints 2023-12-30 10:44:13 +05:30
Krrish Dholakia
a11940f4eb fix(router.py): handle initial scenario for tpm/rpm routing 2023-12-30 07:28:45 +05:30
Krrish Dholakia
1933d44cbd fix(router.py): fix int logic 2023-12-29 20:41:56 +05:30
Krrish Dholakia
a30f00276b refactor(lowest_tpm_rpm.py): move tpm/rpm based routing to a separate file for better testing 2023-12-29 18:33:43 +05:30
Krrish Dholakia
ffe2350428 fix(least_busy.py): support consistent use of model id instead of deployment name 2023-12-29 17:05:26 +05:30
Krrish Dholakia
90ed04d992 fix(router.py): support wait_for for async completion calls 2023-12-29 15:27:20 +05:30
Krrish Dholakia
b9dd46de6c feat(router.py): support 'retry_after' param, to set min timeout before retrying a failed request (default 0) 2023-12-29 15:18:28 +05:30
Krrish Dholakia
6bd6d68990 feat(proxy_server.py): support maxage cache control 2023-12-26 17:50:27 +05:30
Krrish Dholakia
8f3732eeeb docs(user_keys.md): docs on passing user keys to litellm proxy 2023-12-26 13:55:28 +05:30
Krrish Dholakia
a4633c57c4 fix(router.py): accept dynamic api key 2023-12-26 13:16:22 +05:30
Krrish Dholakia
79978c44ba refactor: add black formatting 2023-12-25 14:11:20 +05:30
Krrish Dholakia
d1dea7c87d fix(utils.py): log user_id to langfuse 2023-12-23 12:14:09 +05:30
Krish Dholakia
a52bb3ff1c Merge pull request #1182 from sumanth13131/usage-based-routing-fix 
usage_based_routing_fix
 2023-12-23 11:50:34 +05:30
Krish Dholakia
f509ee4ee1 Merge pull request #1195 from AllentDan/fix-routing 
fix least_busy router by updating min_traffic
 2023-12-23 11:45:35 +05:30
Krrish Dholakia
61ab8dd5c1 fix(proxy_server.py): handle misformatted json body in chat completion request 2023-12-22 12:30:36 +05:30
Krrish Dholakia
c084f04a35 fix(router.py): add support for async image generation endpoints 2023-12-21 14:38:44 +05:30
Krrish Dholakia
b873833340 fix(router.py): only do sync image gen fallbacks for now 
The customhttptransport we use for dall-e-2 only works for sync httpx calls, not async. Will need to spend some time writing the async version

n
 2023-12-20 19:10:59 +05:30
Krrish Dholakia
89b808d767 feat(router.py): support async image generation on router 2023-12-20 17:24:20 +05:30
AllentDan
4b84f0f049 fix least_busy router by updating min_traffic 2023-12-20 18:16:00 +08:00
sumanth
64dfd0d9a6 fix 2023-12-19 13:15:48 +05:30
Krrish Dholakia
00826e195e fix(router.py): init deployment_latency_map even if model_list is empty 2023-12-18 17:50:26 -08:00
Krrish Dholakia
d9dfa3defc fix(Dockerfile): support mac 2023-12-16 16:01:02 -08:00
Krrish Dholakia
68c53256ab fix(proxy_server.py): setup depedencies on server startup 2023-12-16 11:56:11 -08:00
Krrish Dholakia
e76ed6be7d feat(router.py): support caching groups 2023-12-15 21:45:51 -08:00
Krrish Dholakia
8e7491c67e fix(router.py): init clients for azure/openai on set_model_list 2023-12-15 20:23:29 -08:00
Krrish Dholakia
6081602bf8 fix(router.py): support openai-compatible endpoints 2023-12-15 14:47:54 -08:00
Krrish Dholakia
07a54977a2 fix(router.py): adding mistral to list of openai-compatible endpoints 2023-12-15 14:24:43 -08:00
ishaan-jaff
5731b3bc23 (feat) router - remove confusing -ModelID-XXXX 2023-12-15 07:07:53 +05:30
Krrish Dholakia
bb5b883316 fix(main.py): support async streaming for text completions endpoint 2023-12-14 13:56:32 -08:00
ishaan-jaff
072fdac48c (feat) caching - add supported call types 2023-12-14 22:27:14 +05:30
ishaan-jaff
18478382bf (feat) add mistral to proxy + router 2023-12-14 18:20:08 +05:30