Ishaan Jaff
3245d8cdce
support all-proxy-models for teams
2024-04-02 16:04:09 -07:00
Ishaan Jaff
b83c452ddd
support all-models-on-proxy
2024-04-02 15:52:54 -07:00
Ishaan Jaff
73ef4780f7
(fix) support all-models alias on backend
2024-04-02 15:12:37 -07:00
Krrish Dholakia
b07788d2a5
fix(openai.py): return logprobs for text completion calls
2024-04-02 14:05:56 -07:00
Krrish Dholakia
0d949d71ab
fix(main.py): support text completion input being a list of strings
...
addresses - https://github.com/BerriAI/litellm/issues/2792 , https://github.com/BerriAI/litellm/issues/2777
2024-04-02 08:50:16 -07:00
Ishaan Jaff
92984a1c6f
Merge pull request #2788 from BerriAI/litellm_support_-_models
...
[Feat] Allow using model = * on proxy config.yaml
2024-04-01 19:46:50 -07:00
Ishaan Jaff
98df2b027b
test test_wildcard_openai_routing
2024-04-01 19:46:07 -07:00
Krrish Dholakia
c3e4af76cf
refactor: fix linting issue
2024-04-01 18:11:38 -07:00
Krrish Dholakia
6467dd4e11
fix(tpm_rpm_limiter.py): fix cache init logic
2024-04-01 18:01:38 -07:00
Krrish Dholakia
52b1538b2e
fix(router.py): support context window fallbacks for pre-call checks
2024-04-01 10:51:54 -07:00
Krrish Dholakia
c9e6b05cfb
test(test_max_tpm_rpm_limiter.py): add unit testing for redis namespaces working for tpm/rpm limits
2024-04-01 10:39:03 -07:00
Krrish Dholakia
f3e47323b9
test(test_max_tpm_rpm_limiter.py): unit tests for key + team based tpm rpm limits on proxy
2024-04-01 08:11:30 -07:00
Ishaan Jaff
ddb35facc0
ci/cd run again
2024-04-01 07:40:05 -07:00
Krrish Dholakia
aebb0e489c
test: fix test
2024-04-01 07:29:56 -07:00
Krrish Dholakia
583e334bd2
fix(utils.py): set redis_usage_cache to none by default
2024-04-01 07:29:56 -07:00
Krish Dholakia
2ca303ec0e
Merge pull request #2748 from BerriAI/litellm_anthropic_tool_calling_list_parsing_fix
...
fix(factory.py): parse list in xml tool calling response (anthropic)
2024-03-30 11:27:02 -07:00
Krrish Dholakia
22d5603778
ci(config.yml): add lunary to circle ci
2024-03-29 22:09:21 -07:00
Vincelwt
1b84dfac91
Merge branch 'main' into main
2024-03-30 13:21:53 +09:00
Krrish Dholakia
cbf35087c7
test(test_key_generate_prisma.py): fix test
2024-03-29 20:30:43 -07:00
Krrish Dholakia
3810b050c1
fix(proxy_server.py): increment cached global proxy spend object
2024-03-29 20:02:31 -07:00
Krrish Dholakia
5280fc809f
fix(proxy_server.py): enforce end user budgets with 'litellm.max_end_user_budget' param
2024-03-29 17:14:40 -07:00
Krrish Dholakia
bbd94f504c
test(test_rules.py): fix assert
2024-03-29 13:12:16 -07:00
Krrish Dholakia
49642a5b00
fix(factory.py): parse list in xml tool calling response (anthropic)
...
improves tool calling outparsing to check if list in response. Also returns the raw response back to the user via `response._hidden_params["original_response"]`, so user can see exactly what anthropic returned
2024-03-29 11:51:26 -07:00
Krrish Dholakia
109cd93a39
fix(sagemaker.py): support model_id consistently. support dynamic args for async calls
2024-03-29 09:05:00 -07:00
Krrish Dholakia
d547944556
fix(sagemaker.py): support 'model_id' param for sagemaker
...
allow passing inference component param to sagemaker in the same format as we handle this for bedrock
2024-03-29 08:43:17 -07:00
Krrish Dholakia
cd53291b62
fix(utils.py): support bedrock mistral streaming
2024-03-29 07:56:10 -07:00
Krrish Dholakia
5a117490ec
fix(proxy_server.py): fix tpm/rpm limiting for jwt auth
...
fixes tpm/rpm limiting for jwt auth and implements unit tests for jwt auth
2024-03-28 21:19:34 -07:00
Krish Dholakia
934a9ac2b4
Merge pull request #2722 from BerriAI/litellm_db_perf_improvement
...
feat(proxy/utils.py): enable updating db in a separate server
2024-03-28 14:56:14 -07:00
Krrish Dholakia
47ca223d0b
fix(lowest_tpm_rpm_routing.py): fix base case where max tpm/rpm is 0
2024-03-28 14:51:31 -07:00
Krrish Dholakia
e8d80509b1
test(test_update_spend.py): allow db_client to be none
2024-03-28 13:44:40 -07:00
Ishaan Jaff
365497e860
(fix) OpenAI img gen endpoints unstable
2024-03-28 12:42:04 -07:00
Ishaan Jaff
6d408dcce7
(fix) test aimg gen on router
2024-03-28 12:27:26 -07:00
Krrish Dholakia
9ef7afd2b4
test(test_completion.py): skip unresponsive endpoint
2024-03-27 20:12:22 -07:00
Krrish Dholakia
9b7383ac67
fix(utils.py): don't run post-call rules on a coroutine function
2024-03-27 13:16:27 -07:00
Krish Dholakia
c1f8d346b8
Merge pull request #2706 from BerriAI/litellm_key_llm_guardrails
...
feat(llm_guard.py): enable key-specific llm guard check
2024-03-26 19:02:11 -07:00
Krish Dholakia
e266142d2b
Merge pull request #2705 from BerriAI/litellm_permissions_table
...
enable new `/team/disable` endpoint
2024-03-26 18:47:34 -07:00
Krrish Dholakia
4488480188
test(test_llm_guard.py): fix test
2024-03-26 18:37:27 -07:00
Krrish Dholakia
f62f642393
test(test_llm_guard.py): fix test
2024-03-26 18:13:15 -07:00
Krrish Dholakia
5b66cb3864
test(test_exceptions.py): handle api instability
2024-03-26 18:06:49 -07:00
Krrish Dholakia
1046a63521
test(test_llm_guard.py): unit testing for key-level llm guard enabling
2024-03-26 17:55:53 -07:00
Krish Dholakia
0ab708e6f1
Merge pull request #2704 from BerriAI/litellm_jwt_auth_improvements_3
...
fix(handle_jwt.py): enable team-based jwt-auth access
2024-03-26 16:06:56 -07:00
Krrish Dholakia
ea8f6672c5
test(test_jwt.py): fix test
2024-03-26 15:22:05 -07:00
Krrish Dholakia
752516df1b
fix(handle_jwt.py): support public key caching ttl param
2024-03-26 14:32:55 -07:00
Krrish Dholakia
4d7f4550e2
test(test_batch_completions.py): handle anthropic overloaded error
2024-03-26 13:55:03 -07:00
Krrish Dholakia
6b1d2551d1
test(test_batch_completions.py): handle overloaded anthropic error
2024-03-26 13:53:18 -07:00
Krrish Dholakia
3a82ff2ef2
fix(utils.py): don't send subsequent chunks if last chunk sent
...
prevents multiple empty finish chunks from being sent
2024-03-26 13:49:42 -07:00
Krrish Dholakia
b4d0a95cff
test(test_router_debug_logs.py): add info statement to log test
2024-03-26 09:54:26 -07:00
Ishaan Jaff
2ecdd92619
Merge pull request #2702 from BerriAI/litellm_cache_flush
...
[Feat] Proxy - /cache/flushall - delete all elements from cache
2024-03-26 09:34:39 -07:00
Krrish Dholakia
2dd2b8a8e3
test(test_streaming.py): add unit testing for custom stream wrapper
2024-03-26 08:57:44 -07:00
Krrish Dholakia
49e8cdbff9
fix(router.py): check for context window error when handling 400 status code errors
...
was causing proxy context window fallbacks to not work as expected
2024-03-26 08:08:15 -07:00
