phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-27 11:43:54 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
8909 commits 2497 branches 897 tags 550 MiB
Commit graph

964 commits

Author SHA1 Message Date
Krrish Dholakia
e9cc6b4cc9 feat(proxy_server.py): enable llm api based prompt injection checks 
run user calls through an llm api to check for prompt injection attacks. This happens in parallel to th
e actual llm call using `async_moderation_hook`
 2024-03-20 22:43:42 -07:00
Krrish Dholakia
feb78b7819 fix(proxy_server.py): fix import 2024-03-20 19:15:06 -07:00
Ishaan Jaff
4ad5ff0230 (feat) better debugging for /cache/ping 2024-03-20 08:30:11 -07:00
Ishaan Jaff
e8f775ee04 (feat) litellm cache ping 2024-03-20 08:24:13 -07:00
Ishaan Jaff
4c38f5a450 (fix) caching - don't require cache password 2024-03-19 20:50:16 -07:00
Krrish Dholakia
5c88a8bcba fix(proxy_server.py): allow user to disable scheduled reset budget task 2024-03-19 20:36:22 -07:00
Krrish Dholakia
499087301c Revert "Merge pull request #2593 from BerriAI/litellm_reset_budget_fix" 
This reverts commit afd363129f, reversing
changes made to c94bc94ad5.
 2024-03-19 20:25:41 -07:00
Krish Dholakia
eeb8b585dc Merge pull request #2593 from BerriAI/litellm_reset_budget_fix 
fix(proxy/utils.py): fix reset budget logic
 2024-03-19 20:17:03 -07:00
Krrish Dholakia
5a49360af2 fix(proxy_server.py): add more debug logs 2024-03-19 19:59:43 -07:00
Krrish Dholakia
809124a698 fix: better debug logs 2024-03-19 19:28:26 -07:00
Ishaan Jaff
854437df0e Merge pull request #2591 from BerriAI/litellm_metrics_endpoint 
[Feat] /metrics endpoint for Prometheus, Grafana
 2024-03-19 18:08:22 -07:00
Ishaan Jaff
acd6691d6d (feat) using prom litellm 2024-03-19 15:49:23 -07:00
Krrish Dholakia
386688e5f9 feat(handle_jwt.py): support authenticating admins into the proxy via jwt's 2024-03-19 15:00:27 -07:00
Ishaan Jaff
3f28e932d9 (v0) prometheus metric 2024-03-19 14:48:38 -07:00
Krrish Dholakia
53ed290ed3 fix(proxy_server.py): expose disable_spend_logs flag in config general settings 
Writing each spend log adds +300ms latency

https://github.com/BerriAI/litellm/issues/1714#issuecomment-1924727281
 2024-03-19 12:08:37 -07:00
Krish Dholakia
f522a5236b Merge pull request #2561 from BerriAI/litellm_batch_writing_db 
fix(proxy/utils.py): move to batch writing db updates
 2024-03-18 21:50:47 -07:00
Krrish Dholakia
e20ee870d2 refactor(proxy_server.py): re-add custom db client logic - prevent regressions 2024-03-18 21:16:28 -07:00
Ishaan Jaff
1a3a19e195 (fix) if litellm-proxy-budget set use it 2024-03-18 20:31:23 -07:00
Krrish Dholakia
6c578db2c2 fix(proxy_server.py): fix spend log update 2024-03-18 20:26:28 -07:00
Ishaan Jaff
d2f3813e54 (fix) show global spend on UI 2024-03-18 18:15:08 -07:00
Krrish Dholakia
1b10123f07 fix(proxy/utils.py): batch writing updates to db 2024-03-18 16:47:02 -07:00
Krrish Dholakia
7f74cf0057 fix(proxy_server.py): don't override cache params on proxy config if set 2024-03-18 12:11:30 -07:00
Krrish Dholakia
23d088a1da fix(proxy_server.py): ignore cache if value is false 2024-03-18 11:15:32 -07:00
Krrish Dholakia
1618751824 fix(proxy/utils.py): move to batch writing db updates 2024-03-16 22:32:00 -07:00
Ishaan Jaff
0da6e78cb7 Merge pull request #2560 from BerriAI/litellm_view_team_based_spend 
Admin UI - view team based spend
 2024-03-16 19:38:17 -07:00
Ishaan Jaff
e6bebfda8f (feat) new ui build 2024-03-16 19:35:00 -07:00
Ishaan Jaff
12c68060c9 (feat) view team based spend 2024-03-16 19:06:16 -07:00
Krish Dholakia
293c32204e Merge pull request #2556 from BerriAI/litellm_aws_secret_manager_support 
fix(utils.py): initial commit for aws secret manager support
 2024-03-16 18:41:58 -07:00
Krrish Dholakia
bad2327b88 fix(utils.py): fix aws secret manager + support key_management_settings 
fixes the aws secret manager implementation and allows the user to set which keys they want to check thr
ough it
 2024-03-16 16:47:50 -07:00
Ishaan Jaff
3cc579dcbb Merge pull request #2559 from BerriAI/litellm_show_spend_correctly 
(fix) admin ui - order spend by date
 2024-03-16 16:43:13 -07:00
Ishaan Jaff
6253b45517 Merge pull request #2557 from BerriAI/litellm_clean_up_health_readiness 
(fix) /health/readiness return success callback names as (str)
 2024-03-16 16:38:01 -07:00
Ishaan Jaff
d6d9d69618 (fix) admin ui - order spend by date 2024-03-16 16:15:10 -07:00
Ishaan Jaff
e11e42a0ec (fix) /health readiness return callback names 2024-03-16 15:56:07 -07:00
Ishaan Jaff
dd17c5bd1a Merge pull request #2553 from BerriAI/litellm_improve_health_readiness_perf 
(feat) Proxy - improve health readiness perf (93% faster)
 2024-03-16 15:38:45 -07:00
Ishaan Jaff
e71a5b4d24 (fix) improve /health/readiness under high load 2024-03-16 14:14:17 -07:00
Krrish Dholakia
e37c97607b fix(proxy_server.py): fix linting errors 2024-03-16 14:02:36 -07:00
Krrish Dholakia
928ea59fac fix(proxy_server.py): rename beta end user blocking endpoints 
separating end user management from user management
 2024-03-16 13:53:10 -07:00
Ishaan Jaff
f6418b9b34 (feat) improve health readiness perf 2024-03-16 13:10:04 -07:00
Krrish Dholakia
be33918438 fix(blocked_user_list.py): check if end user blocked in db 2024-03-16 13:03:52 -07:00
Krrish Dholakia
588b252cab fix(proxy_server.py): write blocked user list to a db table 
lets this persist across workers
 2024-03-16 12:26:29 -07:00
Krrish Dholakia
8ebf4a62dc fix(proxy_server.py): bug fix on getting user obj from cache 2024-03-16 11:07:38 -07:00
Krrish Dholakia
27de1089a6 fix(caching.py): close redis connection pool upon proxy shutdown 2024-03-16 10:39:58 -07:00
ishaan-jaff
27368f9500 (fix) backoff import 2024-03-16 08:53:06 -07:00
ishaan-jaff
f71ea87cf1 (fix) errant print saying "Error importing .." msg 2024-03-15 21:47:52 -07:00
Krish Dholakia
8e6e7f880d Merge pull request #2542 from BerriAI/litellm_redis_perf_improvements 
89% Caching improvement - Async Redis completion calls + batch redis GET requests for a given key + call type
 2024-03-15 18:58:36 -07:00
ishaan-jaff
37be2ba569 (fix) fix error message raised on passing invalid tokens 2024-03-15 14:50:34 -07:00
Krrish Dholakia
8d1c60bfdc feat(batch_redis_get.py): batch redis GET requests for a given key + call type 
reduces the number of GET requests we're making in high-throughput scenarios
 2024-03-15 14:40:11 -07:00
Ishaan Jaff
607292596b Merge pull request #2524 from BerriAI/litellm_fix_update_user 
(fix) - update user error
 2024-03-15 08:51:45 -07:00
ishaan-jaff
2893d289a0 (fix) error cli users see when importing enterprise folder 2024-03-15 08:10:45 -07:00
ishaan-jaff
084fdf5cb9 (fix) - update user error 2024-03-14 19:50:26 -07:00