phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-27 19:54:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
21461 commits 2497 branches 898 tags 550 MiB
Commit graph

153 commits

Author SHA1 Message Date
Lunik
08593fcaab ⚡️ perf: Remove test violation on each stream chunk 
Signed-off-by: Lunik <lunik@tiwabbit.fr>
 2024-05-03 20:51:40 +02:00
Lunik
7945e28356 ✅ ci: Add tests 
Signed-off-by: Lunik <lunik@tiwabbit.fr>
 2024-05-03 20:50:37 +02:00
Lunik
3ca174bc57 ✨ feat: Add Azure Content-Safety Proxy hooks 
Signed-off-by: Lunik <lunik@tiwabbit.fr>
 2024-05-02 23:21:08 +02:00
Krrish Dholakia
31e2d4e6d1 feat(lowest_tpm_rpm_v2.py): move to using redis.incr and redis.mget for getting model usage from redis 
makes routing work across multiple instances
 2024-04-10 14:56:23 -07:00
Krrish Dholakia
e06d43dc90 fix(tpm_rpm_limiter.py): fix cache init logic 2024-04-01 18:01:38 -07:00
Krrish Dholakia
b39bc583bd test(test_max_tpm_rpm_limiter.py): unit tests for key + team based tpm rpm limits on proxy 2024-04-01 08:00:01 -07:00
Krrish Dholakia
555f0af027 fix(tpm_rpm_limiter.py): enable redis caching for tpm/rpm checks on keys/user/teams 
allows tpm/rpm checks to work across instances

https://github.com/BerriAI/litellm/issues/2730
 2024-03-30 20:01:36 -07:00
Krrish Dholakia
737bb3e444 fix(proxy_server.py): fix tpm/rpm limiting for jwt auth 
fixes tpm/rpm limiting for jwt auth and implements unit tests for jwt auth
 2024-03-28 21:19:34 -07:00
Krrish Dholakia
7bc76ddbc3 feat(llm_guard.py): enable key-specific llm guard check 2024-03-26 17:21:51 -07:00
Ishaan Jaff
f0992c2dbd (fix) stop using f strings with logger 2024-03-25 10:47:18 -07:00
Krrish Dholakia
b872644496 fix(prompt_injection_detection.py): fix type check 2024-03-21 08:56:13 -07:00
Krrish Dholakia
5cfabe9a09 fix: fix linting issue 2024-03-21 08:19:09 -07:00
Krrish Dholakia
e9cc6b4cc9 feat(proxy_server.py): enable llm api based prompt injection checks 
run user calls through an llm api to check for prompt injection attacks. This happens in parallel to th
e actual llm call using `async_moderation_hook`
 2024-03-20 22:43:42 -07:00
Krrish Dholakia
feb78b7819 fix(proxy_server.py): fix import 2024-03-20 19:15:06 -07:00
Krrish Dholakia
e9ff51aa70 fix(prompt_injection_detection.py): ensure combinations are actual phrases, not just 1-2 words 
reduces misflagging

https://github.com/BerriAI/litellm/issues/2601
 2024-03-20 19:09:38 -07:00
Krrish Dholakia
3680f16cd7 feat(batch_redis_get.py): batch redis GET requests for a given key + call type 
reduces number of redis requests. 85ms latency improvement over 3 minutes of load (19k requests).
 2024-03-15 14:54:16 -07:00
Krrish Dholakia
8d1c60bfdc feat(batch_redis_get.py): batch redis GET requests for a given key + call type 
reduces the number of GET requests we're making in high-throughput scenarios
 2024-03-15 14:40:11 -07:00
Krrish Dholakia
d2f47ee45b fix(parallel_request_limiter.py): handle metadata being none 2024-03-14 10:02:41 -07:00
Krrish Dholakia
c963e2761b feat(proxy_server.py): retry if virtual key is rate limited 
currently for chat completions
 2024-03-05 19:00:03 -08:00
Krrish Dholakia
f72b84f6e0 fix(parallel_request_limiter.py): handle none scenario 2024-02-26 20:09:06 -08:00
Krrish Dholakia
7fff5119de fix(parallel_request_limiter.py): fix team rate limit enforcement 2024-02-26 18:06:13 -08:00
Krrish Dholakia
5213fd2e1e feat(parallel_request_limiter.py): enforce team based tpm / rpm limits 2024-02-26 16:20:41 -08:00
ishaan-jaff
5ec69a0ca5 (fix) failing parallel_Request_limiter test 2024-02-22 19:16:22 -08:00
ishaan-jaff
b728ded300 (fix) don't double check curr data and time 2024-02-22 18:50:02 -08:00
ishaan-jaff
74d66d5ac5 (feat) tpm/rpm limit by User 2024-02-22 18:44:03 -08:00
Krrish Dholakia
f68b692147 fix(presidio_pii_masking.py): enable user to pass ad hoc recognizer for pii masking 2024-02-20 16:01:15 -08:00
Krrish Dholakia
aa93b02562 fix(presidio_pii_masking.py): enable user to pass their own ad hoc recognizers to presidio 2024-02-20 15:19:31 -08:00
Krrish Dholakia
966abee67f test(test_presidio_pii_masking.py): add more unit tests 2024-02-19 16:30:44 -08:00
Krrish Dholakia
93acda267c feat(presidio_pii_masking.py): allow request level controls for turning on/off pii masking 
https://github.com/BerriAI/litellm/issues/2003
 2024-02-17 11:04:56 -08:00
Krrish Dholakia
7a75682637 docs(enterprise.md): add llama guard tutorial to enterprise docs 2024-02-17 09:25:49 -08:00
Krrish Dholakia
e2b2bb0de1 fix: check key permissions for turning on/off pii masking 2024-02-15 20:16:15 -08:00
Krrish Dholakia
b399aab597 fix(presidio_pii_masking.py): fix conditional check 2024-02-13 22:11:03 -08:00
Krrish Dholakia
9936427669 feat(presidio_pii_masking.py): enable output parsing for pii masking 2024-02-13 21:36:57 -08:00
Krrish Dholakia
c1a60176a5 feat(proxy_server.py): support for pii masking with microsoft presidio 2024-02-10 20:21:12 -08:00
Krrish Dholakia
07aa05bf17 fix(test_parallel_request_limiter.py): use mock responses for streaming 2024-02-08 21:45:38 -08:00
ishaan-jaff
c8b2f0fd5d (fix) parallel_request_limiter debug 2024-02-06 12:43:28 -08:00
Krrish Dholakia
dbf2b0b2c8 fix(utils.py): override default success callbacks with dynamic callbacks if set 2024-02-02 06:21:43 -08:00
Krrish Dholakia
f5c88018dd feat(proxy_server.py): enable cache controls per key + no-store cache flag 2024-01-30 20:46:50 -08:00
Krrish Dholakia
c91ab81fde fix(test_parallel_request_limiter): increase time limit for waiting for success logging event to happen 2024-01-30 13:26:17 -08:00
Krrish Dholakia
e957f41ab7 fix(utils.py): add metadata to logging obj on setup, if exists 2024-01-19 17:29:47 -08:00
Krrish Dholakia
f73a4ae7c2 fix(parallel_request_limiter.py): handle tpm/rpm limits being null 2024-01-19 10:22:27 -08:00
Krrish Dholakia
34c3b33b37 test(test_parallel_request_limiter.py): unit testing for tpm/rpm rate limits 2024-01-18 15:28:28 -08:00
Krrish Dholakia
13b013b28d feat(parallel_request_limiter.py): add support for tpm/rpm limits 2024-01-18 13:52:15 -08:00
Krrish Dholakia
44553bcc3a fix(parallel_request_limiter.py): decrement count for failed llm calls 
https://github.com/BerriAI/litellm/issues/1477
 2024-01-18 12:42:14 -08:00
Krrish Dholakia
79978c44ba refactor: add black formatting 2023-12-25 14:11:20 +05:30
Krrish Dholakia
018405b956 fix(proxy/utils.py): return different exceptions if key is invalid vs. expired 
https://github.com/BerriAI/litellm/issues/1230
 2023-12-25 10:29:44 +05:30
Krrish Dholakia
72e8c84914 build(test_streaming.py): fix linting issues 2023-12-25 07:34:54 +05:30
Krrish Dholakia
a96bac14af fix(proxy_server.py): manage budget at user-level not key-level 
https://github.com/BerriAI/litellm/issues/1220
 2023-12-22 15:10:38 +05:30
Krrish Dholakia
aefaa157e5 feat(proxy_server.py): support max budget on proxy 2023-12-21 16:07:20 +05:30
Krrish Dholakia
1da7d35218 feat(proxy_server.py): enable infinite retries on rate limited requests 2023-12-15 20:03:41 -08:00